From cjplummer at openjdk.org Fri Dec 1 01:58:11 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 1 Dec 2023 01:58:11 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v5] In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 21:27:27 GMT, Serguei Spitsyn wrote: >> This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. >> However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. >> It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. >> It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. >> >> Additionally, the fix includes the following related changes: >> - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: >> >> JvmtiThreadState* state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> MutexLocker mu(JvmtiThreadState_lock); >> state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> JvmtiEventController::enter_interp_only_mode(); >> } >> } >> >> >> - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. >> - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. >> >> Performance mesurements: >> - Without this fix the test provided by the bug submitter gives execution numbers: >> - no ClassLoad events enabled: 3251 ms >> - ClassLoad events are enabled: 40534 ms >> >> - With the fix: >> - no ClassLoad events enabled: 3270 ms >> - ClassLoad events are enabled: 3385 ms >> >> Testing: >> - Ran mach5 tiers 1-6, no regressions are noticed > > Serguei Spitsyn has updated the pull request incrementally with two additional commits since the last revision: > > - review: one more minor correction of a comment > - review: minor correction of a comment Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16686#pullrequestreview-1758873109 From amenkov at openjdk.org Fri Dec 1 04:08:05 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 1 Dec 2023 04:08:05 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: References: Message-ID: <8VMA3XjqiXFbsQTa5IjAouBch2QjdhaRnERtcTTlxpg=.148cbc2c-8e73-4a6e-b980-682fb45b8ae9@github.com> On Thu, 30 Nov 2023 21:11:08 GMT, Chris Plummer wrote: > I wasn't thinking in terms of the scheduler somehow no longer references the virtual thread, but instead the program no longer referencing the scheduler (and also not referencing the virtual thread). AFAIU unfinished unmounted virtual threads are referenced from other objects (they are parked on), so they can't be unreachable even is the application is not referencing them and the scheduler. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16665#issuecomment-1835422117 From stuefe at openjdk.org Fri Dec 1 05:18:26 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 1 Dec 2023 05:18:26 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 11:49:31 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> ~Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated.~ >> >> Therefore, we need to perform `jmethodID` lookup for each method in an old class version that is getting purged, and null out the pointer of that `jmethodID` to break the link from `jmethodID` to the method instance that is about to get deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: > > Restrict cleanup to obsolete methods only I won't be able to review this this week, too snowed in atm. I can take a look next week. We can always just revert the change if needed. Thinking about Skara, I think as long as we have this confusing mixture of rules (hotspot wants 2 reviewers that are Reviewer/Committer, but some jdk libs only want one, but then you need two for desktop I think otherwise Phil gets angry) - we should hard-code the 2-reviewer rule into skara as default since it affects the lion's share of all changes. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1835474161 From cjplummer at openjdk.org Fri Dec 1 05:46:08 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 1 Dec 2023 05:46:08 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: On Thu, 30 Nov 2023 00:26:25 GMT, Alex Menkov wrote: >> The change impelements dumping of unmounted virtual threads data (stack traces and stack references). >> Unmounted vthreads can be detected only by iterating over the heap, but hprof stack trace records (HPROF_FRAME/HPROF_TRACE) should be written before HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT. >> HeapDumper supports segment dump (parallel dump to separate files with subsequent file merge outside of safepoint), the fix switches HeapDumper to always use segment dump: 1st segment contains only non-heap data, other segments are used for dumping heap objects. For serial dumping single-threaded dumping is performed, but 2 segments are created anyway. >> When HeapObjectDumper detects unmounted virtual thread, it writes HPROF_FRAME/HPROF_TRACE records to the 1st segment ("global writer"), and writes thread object (HPROF_GC_ROOT_JAVA_FRAME) and stack references (HPROF_GC_ROOT_JAVA_FRAME/HPROF_GC_ROOT_JNI_LOCAL) to the HeapObjectDumper segment. >> As parallel dumpers may write HPROF_FRAME/HPROF_TRACE concurrently and VMDumper needs to write non-heap data before heap object dumpers can write virtual threads data, writing to global writer is protected with DumperController::_global_writer_lock. >> >> Testing: run tests which perform heap dump (in different scenarios): >> - test/hotspot/jtreg/serviceability >> - test/hotspot/jtreg/runtime/ErrorHandling >> - test/hotspot/jtreg/gc/epsilon >> - test/jdk/sun/tools/jhsdb > > Alex Menkov has updated the pull request incrementally with two additional commits since the last revision: > > - feedback > - prepare_parallel_dum One concern I have is when there is a large number of virtual threads, the dump may take too long and the hprof file gets bloated. My concern mostly comes from the large number of virtual thread stack traces that will be present. Dumping all these hprof records related to unmounted virtual threads could do more harm than good in some instances, and we may want a way for the user to disable it. It would be nice if there could be some data sharing between threads with identical stack traces, but I don't see a way to do that with the current hprof spec. ------------- PR Review: https://git.openjdk.org/jdk/pull/16665#pullrequestreview-1759060258 From jpai at openjdk.org Fri Dec 1 08:02:04 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 1 Dec 2023 08:02:04 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v4] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Sat, 25 Nov 2023 06:23:25 GMT, Jaikiran Pai wrote: > Since serviceability area requires 2 Reviewers, could I get one more Reviewer to review this please? I'm looking for one more Reviewer, please. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16805#issuecomment-1835642089 From sjohanss at openjdk.org Fri Dec 1 10:01:27 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Fri, 1 Dec 2023 10:01:27 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: References: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> Message-ID: <4NJOslObcIY-G1nUAbKeiCWKK-wbhEw94avA2c2cJ7s=.2ff4560f-5184-40ad-982f-96570cc4a9fc@github.com> On Thu, 30 Nov 2023 21:59:41 GMT, Man Cao wrote: >> src/hotspot/share/runtime/cpuTimeCounters.hpp line 59: >> >>> 57: NONCOPYABLE(CPUTimeCounters); >>> 58: >>> 59: static CPUTimeCounters* _instance; >> >> I would prefer if we made the whole class static and got rid of the instance and just made the `_cpu_time_counters` array static. The only drawback I/we (discussed this with Albert as well) can see is that the memory for the array would not be accounted in NMT, but this array will always be very small so should not be a big problem. >> >> Do you see any other concerns? > > I thought it is typically preferred to initialize a singleton object on the heap, rather than using several static variables. It is easier to control the initialization order and timing of an on-heap singleton object than statics. > > Moreover, for this class, `initialize()` could also check `if (UsePerfData)`, and only create the singleton object under `UsePerfData`. This could save some memory when `UsePerfData` is false. I would say it depends on the use-case and here when switching to use static functions to use the instance it felt more like an all-static class. I agree that it would be nice to avoid the additional memory usage if `UsePerfData` is `false` so I'm ok with keeping the instance if we add that. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1411872139 From sjohanss at openjdk.org Fri Dec 1 10:05:25 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Fri, 1 Dec 2023 10:05:25 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 01:43:28 GMT, Jonathan Joo wrote: >> (I just realized that I made a typo in my previous msg; should be *callee* instead.) That is what I have in mind. >> >> >> void CPUTimeCounters::update_counter(name, total) { >> auto counter = get_counter(name); >> auto old_v = counter->get_value(); >> auto diff = total - old_v; >> counter->inc(diff); >> if (counter->is_gc_counter()) { >> counter->inc(diff); >> } >> } > > I'm not sure I understood correctly, could you let me know if this latest commit addresses your comment in the way you were intending? It does. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1411876609 From kevinw at openjdk.org Fri Dec 1 10:06:23 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Fri, 1 Dec 2023 10:06:23 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 16:21:29 GMT, Kevin Walls wrote: > JMX RMI Connections should use a timeout on the Socket connect call by default. > > JMX Connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. > > Changing the base RMI implementation may not be desirable at this time. > > JMX can use a new ClientSocketFactory for RMI which implements the connect timeout, which can recognise a new JMX-specific property `com.sun.management.jmxremote.rmi.tcpConnectTimeout` > (named like the existing com.sun.management.jmxremote... properties) > > Defaulting to a 1 minute timeout on connect has no effect on existing tests, and should go unnoticed unless there really is a significant connection delay. Specifying zero for the new System Property will use the old technique of a connect with no timeout. > > This can be tested, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will very often pass as the connection happens immediately. Thanks, that's right it is a bad interoperability problem! The alternative SocketFactory needs to be available at all clients, so this is not workable. The other possibility was use of RMISocketFactory::setSocketFactory on the JMX client end. This lets us substitute in a preferred socket factory before making a connection, and one which can implement its own timeout. This is awkward as it affects the whole process. For JMX apps such as JConsole etc this is fine, but in other situations it may have side-effects on other RMI activity. At the moment the demand for the timeout is not strong, and the behaviour is what it has been for many years, so I am going to close this without progressing it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16856#issuecomment-1835810085 From kevinw at openjdk.org Fri Dec 1 10:06:23 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Fri, 1 Dec 2023 10:06:23 GMT Subject: Withdrawn: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: <2k_cWi2SndOeX1WGX71-27gjAQTQLwtUANCO6kfI_fk=.162522b0-d92c-416a-b207-5020b8973bd5@github.com> On Tue, 28 Nov 2023 16:21:29 GMT, Kevin Walls wrote: > JMX RMI Connections should use a timeout on the Socket connect call by default. > > JMX Connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. > > Changing the base RMI implementation may not be desirable at this time. > > JMX can use a new ClientSocketFactory for RMI which implements the connect timeout, which can recognise a new JMX-specific property `com.sun.management.jmxremote.rmi.tcpConnectTimeout` > (named like the existing com.sun.management.jmxremote... properties) > > Defaulting to a 1 minute timeout on connect has no effect on existing tests, and should go unnoticed unless there really is a significant connection delay. Specifying zero for the new System Property will use the old technique of a connect with no timeout. > > This can be tested, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will very often pass as the connection happens immediately. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/16856 From kevinw at openjdk.org Fri Dec 1 10:09:15 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Fri, 1 Dec 2023 10:09:15 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 17:57:32 GMT, Kevin Walls wrote: > RMI Connections (in general) should use a timeout on the Socket connect call by default. > > JMX connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. > > src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java > has other timeouts but nothing to control the initial Socket connect. > > Defaulting to a 1 minute timeout on connect has no effect on existing tests for RMI and JMX, and should go unnoticed in applications unless there really is a significant connection delay. Specifying zero for the new System Property sun.rmi.transport.tcp.initialConnectTimeout uses the old code path of a connect with no timeout. > > I have a test, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will often pass as the connection happens immediately. Thanks for the feedback. Additionally, there can be a concern that affecting all RMI activity with the timeout property could produce unintended side-effects. At the moment the demand for the timeout is not strong, and the behaviour is what it has been for many years, so I am going to close this without progressing it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16771#issuecomment-1835814563 From kevinw at openjdk.org Fri Dec 1 10:09:16 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Fri, 1 Dec 2023 10:09:16 GMT Subject: Withdrawn: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: <4unS4JFDIbdxbZVL0RPVLhp6YME98o-OJpUfmwWQ83Y=.b7e2f0d3-bc27-4941-baca-cc47d5e998bf@github.com> On Tue, 21 Nov 2023 17:57:32 GMT, Kevin Walls wrote: > RMI Connections (in general) should use a timeout on the Socket connect call by default. > > JMX connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. > > src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java > has other timeouts but nothing to control the initial Socket connect. > > Defaulting to a 1 minute timeout on connect has no effect on existing tests for RMI and JMX, and should go unnoticed in applications unless there really is a significant connection delay. Specifying zero for the new System Property sun.rmi.transport.tcp.initialConnectTimeout uses the old code path of a connect with no timeout. > > I have a test, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will often pass as the connection happens immediately. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/16771 From sjohanss at openjdk.org Fri Dec 1 10:24:19 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Fri, 1 Dec 2023 10:24:19 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Thu, 30 Nov 2023 21:43:36 GMT, Man Cao wrote: >> @simonis was the original suggester of this counter, so I will defer to his expertise. I do agree that dropping the counter would simplify things, but it also might not hurt to just leave it in. I'm okay with either option! > > Right, see @simonis 's comments at https://github.com/openjdk/jdk/pull/15082#pullrequestreview-1613868256, https://github.com/openjdk/jdk/pull/15082#discussion_r1321703912. > > I initially had similar thought that `gc_total` isn't necessary and provides redundant data. Now I agree with @simonis that the `gc_total` is valuable to users. It saves users from extra work of aggregating different sets of counters for different garbage collectors, and potential mistakes of missing some counters. It is also future-proof that if GC implementation changes that add additional threads, users wouldn't need to change their code to add the counter for additional threads. > > I think the maintenance overhead is quite small for `gc_total` since it is mostly in this class. The benefit to users is worth it. I agree that the counter is valuable if always up-to-date, but if it is out of sync compared to the "concurrent counters" I think it will confuse some users. So if we want to keep it I think we should try to keep it in sync. I suppose adding a lock for updating `gc_total` should be ok. In the safepoint case we should never contend on that lock and for the concurrent updates it should not be a big deal. Basically what I think would be to change `update_counter(...)` to do something like: if (CPUTimeGroups::is_gc_counter(name)) { instance->get_counter(CPUTimeGroups::CPUTimeType::gc_total)->inc(net_cpu_time); } This way we would also be able to remove the publish part above, right? Any other problems with this approach? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1411897115 From jkern at openjdk.org Fri Dec 1 11:39:11 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 1 Dec 2023 11:39:11 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality Message-ID: On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). We propose a different, cleaner way of handling this: - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. - Cache dl handles; repeated opening of a library should return the cached handle. - Increase handle-local ref counter on open, Decrease it on close - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. ------------- Commit messages: - JDK-8320890 Changes: https://git.openjdk.org/jdk/pull/16920/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320890 Stats: 202 lines in 7 files changed: 122 ins; 70 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From alanb at openjdk.org Fri Dec 1 12:16:29 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 1 Dec 2023 12:16:29 GMT Subject: RFR: 8320652: ThreadInfo.isInNative needs to be updated to say what executing native code means [v3] In-Reply-To: References: Message-ID: > This is a docs only change to j.l.management.ThreadInfo::isInNative. > > The method currently specifies that it tests if the thread is "executing native code via the Java Native Interface (JNI)". It would be clearer to say that it tests if the thread is executing a native method, and expand it to include native code invoked using the new foreign linker APIs. For now, I've left out the detail of a downcall handle created with the "critical" linker option. Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: - Extend description - Merge - Change link to downcallHandle - Merge - Simplify wording - Merge - Initial commit ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16791/files - new: https://git.openjdk.org/jdk/pull/16791/files/5e2cc189..4836a9de Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16791&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16791&range=01-02 Stats: 16832 lines in 625 files changed: 12312 ins; 2774 del; 1746 mod Patch: https://git.openjdk.org/jdk/pull/16791.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16791/head:pull/16791 PR: https://git.openjdk.org/jdk/pull/16791 From jpai at openjdk.org Fri Dec 1 12:52:09 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 1 Dec 2023 12:52:09 GMT Subject: RFR: 8320652: ThreadInfo.isInNative needs to be updated to say what executing native code means [v3] In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 12:16:29 GMT, Alan Bateman wrote: >> This is a docs only change to j.l.management.ThreadInfo::isInNative. >> >> The method currently specifies that it tests if the thread is "executing native code via the Java Native Interface (JNI)". It would be clearer to say that it tests if the thread is executing a native method, and expand it to include native code invoked using the new foreign linker APIs. For now, I've left out the detail of a downcall handle created with the "critical" linker option. > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: > > - Extend description > - Merge > - Change link to downcallHandle > - Merge > - Simplify wording > - Merge > - Initial commit Thank you Alan for this latest update in commit `4836a9d` - it's much more clear on what's considered native code. src/java.management/share/classes/java/lang/management/ThreadInfo.java line 552: > 550: * > 551: *

A thread is considered to be executing native code when it is executing a > 552: * native method, executing native code invoked using a {@linkplain Nit - should "native method" be "{@code native} method"? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16791#issuecomment-1836064630 PR Review Comment: https://git.openjdk.org/jdk/pull/16791#discussion_r1412065761 From ayang at openjdk.org Fri Dec 1 14:45:24 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 1 Dec 2023 14:45:24 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: <4NJOslObcIY-G1nUAbKeiCWKK-wbhEw94avA2c2cJ7s=.2ff4560f-5184-40ad-982f-96570cc4a9fc@github.com> References: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> <4NJOslObcIY-G1nUAbKeiCWKK-wbhEw94avA2c2cJ7s=.2ff4560f-5184-40ad-982f-96570cc4a9fc@github.com> Message-ID: On Fri, 1 Dec 2023 09:58:16 GMT, Stefan Johansson wrote: >> I thought it is typically preferred to initialize a singleton object on the heap, rather than using several static variables. It is easier to control the initialization order and timing of an on-heap singleton object than statics. >> >> Moreover, for this class, `initialize()` could also check `if (UsePerfData)`, and only create the singleton object under `UsePerfData`. This could save some memory when `UsePerfData` is false. > > I would say it depends on the use-case and here when switching to use static functions to use the instance it felt more like an all-static class. I agree that it would be nice to avoid the additional memory usage if `UsePerfData` is `false` so I'm ok with keeping the instance if we add that. > It is easier to control the initialization order and timing of an on-heap singleton object than statics. It's generally true, but the init of CPUTimeCounters is not sensitive to ordering. > This could save some memory when UsePerfData is false. True, but the mem savings will be marginal at most, `PerfCounter* _cpu_time_counters[static_cast(CPUTimeGroups::CPUTimeType::COUNT)];` will be 8 * ~12 = ~96 bytes (including future cpu-time-type enums). (I don't have a strong preference here; however, I'd like to make the pros and cons explicit.) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1412193296 From simonis at openjdk.org Fri Dec 1 16:22:23 2023 From: simonis at openjdk.org (Volker Simonis) Date: Fri, 1 Dec 2023 16:22:23 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> References: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> Message-ID: <810qMt__o90-A1Csix4IiygZEpyP09w8tisrwY5mQC4=.b8696cfe-2323-4a6d-884c-47df6568a337@github.com> On Thu, 30 Nov 2023 09:46:03 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Add missing include > > src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp line 905: > >> 903: gc_threads_do(&tttc); >> 904: >> 905: CPUTimeCounters::publish_gc_total_cpu_time(); > > As I suggested in the other comment, maybe we should not keep the total counter, but if we do we need to make sure the destructor of the closure is run before the call to `publish_gc_total_cpu_time()`. Otherwise we will publish a not yet updated value. I still think that a total counter is useful and I'd appreciate if you can keep it. To second what @caoman said before, it is GC agnostic, easy to use even for non GC experts and future proof with regards to implementation changes in the GCs. Please keep it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1412322098 From mchung at openjdk.org Fri Dec 1 17:59:09 2023 From: mchung at openjdk.org (Mandy Chung) Date: Fri, 1 Dec 2023 17:59:09 GMT Subject: RFR: 8320652: ThreadInfo.isInNative needs to be updated to say what executing native code means [v3] In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 12:16:29 GMT, Alan Bateman wrote: >> This is a docs only change to j.l.management.ThreadInfo::isInNative. >> >> The method currently specifies that it tests if the thread is "executing native code via the Java Native Interface (JNI)". It would be clearer to say that it tests if the thread is executing a native method, and expand it to include native code invoked using the new foreign linker APIs. For now, I've left out the detail of a downcall handle created with the "critical" linker option. > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: > > - Extend description > - Merge > - Change link to downcallHandle > - Merge > - Simplify wording > - Merge > - Initial commit The updated spec change looks good. ------------- Marked as reviewed by mchung (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16791#pullrequestreview-1760317046 From amenkov at openjdk.org Fri Dec 1 20:16:47 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 1 Dec 2023 20:16:47 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v5] In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 21:27:27 GMT, Serguei Spitsyn wrote: >> This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. >> However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. >> It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. >> It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. >> >> Additionally, the fix includes the following related changes: >> - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: >> >> JvmtiThreadState* state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> MutexLocker mu(JvmtiThreadState_lock); >> state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> JvmtiEventController::enter_interp_only_mode(); >> } >> } >> >> >> - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. >> - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. >> >> Performance mesurements: >> - Without this fix the test provided by the bug submitter gives execution numbers: >> - no ClassLoad events enabled: 3251 ms >> - ClassLoad events are enabled: 40534 ms >> >> - With the fix: >> - no ClassLoad events enabled: 3270 ms >> - ClassLoad events are enabled: 3385 ms >> >> Testing: >> - Ran mach5 tiers 1-6, no regressions are noticed > > Serguei Spitsyn has updated the pull request incrementally with two additional commits since the last revision: > > - review: one more minor correction of a comment > - review: minor correction of a comment Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16686#pullrequestreview-1760508007 From sspitsyn at openjdk.org Fri Dec 1 20:57:50 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 1 Dec 2023 20:57:50 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v5] In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 21:27:27 GMT, Serguei Spitsyn wrote: >> This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. >> However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. >> It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. >> It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. >> >> Additionally, the fix includes the following related changes: >> - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: >> >> JvmtiThreadState* state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> MutexLocker mu(JvmtiThreadState_lock); >> state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> JvmtiEventController::enter_interp_only_mode(); >> } >> } >> >> >> - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. >> - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. >> >> Performance mesurements: >> - Without this fix the test provided by the bug submitter gives execution numbers: >> - no ClassLoad events enabled: 3251 ms >> - ClassLoad events are enabled: 40534 ms >> >> - With the fix: >> - no ClassLoad events enabled: 3270 ms >> - ClassLoad events are enabled: 3385 ms >> >> Testing: >> - Ran mach5 tiers 1-6, no regressions are noticed > > Serguei Spitsyn has updated the pull request incrementally with two additional commits since the last revision: > > - review: one more minor correction of a comment > - review: minor correction of a comment Chris and Alex, thank you for review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16686#issuecomment-1836761160 From sspitsyn at openjdk.org Fri Dec 1 20:57:52 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 1 Dec 2023 20:57:52 GMT Subject: Integrated: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 11:15:27 GMT, Serguei Spitsyn wrote: > This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. > However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. > It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. > It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. > > Additionally, the fix includes the following related changes: > - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: > > JvmtiThreadState* state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > MutexLocker mu(JvmtiThreadState_lock); > state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > JvmtiEventController::enter_interp_only_mode(); > } > } > > > - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. > - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. > > Performance mesurements: > - Without this fix the test provided by the bug submitter gives execution numbers: > - no ClassLoad events enabled: 3251 ms > - ClassLoad events are enabled: 40534 ms > > - With the fix: > - no ClassLoad events enabled: 3270 ms > - ClassLoad events are enabled: 3385 ms > > Testing: > - Ran mach5 tiers 1-6, no regressions are noticed This pull request has now been integrated. Changeset: 42af8ce1 Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/42af8ce1f6605376fdb69e03df9e22381a54fc36 Stats: 24 lines in 2 files changed: 13 ins; 2 del; 9 mod 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 Reviewed-by: dcubed, cjplummer, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/16686 From manc at openjdk.org Fri Dec 1 21:04:51 2023 From: manc at openjdk.org (Man Cao) Date: Fri, 1 Dec 2023 21:04:51 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Fri, 1 Dec 2023 10:21:31 GMT, Stefan Johansson wrote: >> Right, see @simonis 's comments at https://github.com/openjdk/jdk/pull/15082#pullrequestreview-1613868256, https://github.com/openjdk/jdk/pull/15082#discussion_r1321703912. >> >> I initially had similar thought that `gc_total` isn't necessary and provides redundant data. Now I agree with @simonis that the `gc_total` is valuable to users. It saves users from extra work of aggregating different sets of counters for different garbage collectors, and potential mistakes of missing some counters. It is also future-proof that if GC implementation changes that add additional threads, users wouldn't need to change their code to add the counter for additional threads. >> >> I think the maintenance overhead is quite small for `gc_total` since it is mostly in this class. The benefit to users is worth it. > > I agree that the counter is valuable if always up-to-date, but if it is out of sync compared to the "concurrent counters" I think it will confuse some users. So if we want to keep it I think we should try to keep it in sync. > > I suppose adding a lock for updating `gc_total` should be ok. In the safepoint case we should never contend on that lock and for the concurrent updates it should not be a big deal. Basically what I think would be to change `update_counter(...)` to do something like: > > if (CPUTimeGroups::is_gc_counter(name)) { > > instance->get_counter(CPUTimeGroups::CPUTimeType::gc_total)->inc(net_cpu_time); > } > > > This way we would also be able to remove the publish part above, right? Any other problems with this approach? I think the ideal approach to simplify this is to support Atomic operation on a `PerfCounter`. We could either introduce a `PerfAtomicCounter`/`PerfAtomicLongCounter` class, or perform `Atomic::add()` on the `PerfData::_valuep` pointer. There's already `PerfData::get_address()`, so we might be able to do: Atomic::add((volatile jlong *)(instance->get_counter(CPUTimeGroups::CPUTimeType::gc_total)->get_address()), net_cpu_time); However, a new class `PerfAtomicCounter` is likely cleaner. E.g., we may also want to make `PerfAtomicCounter::sample()` use a CAS. It is probably better to introduce `PerfAtomicCounter` in a separate RFE later. Would the `Atomic::add()` with `PerfData::get_address()` approach be OK for now, or would we rather introduce a lock, or leave the `gc_total` mechanism as-is and address the out-of-sync-ness in a follow-up RFE? IMO the out-of-sync-ness problem is minor, because users are likely to either look at a single `gc_total` counter, or look at each individual GC CPU counter and disregard `gc_total`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1412569208 From cjplummer at openjdk.org Fri Dec 1 21:13:37 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 1 Dec 2023 21:13:37 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v5] In-Reply-To: <81dXSHvLQMGj3s1BcBs8fmJUEoJpaU-5wBRSIjnztMM=.d53f8a2f-8353-49ec-8a9b-695b32f03d20@github.com> References: <81dXSHvLQMGj3s1BcBs8fmJUEoJpaU-5wBRSIjnztMM=.d53f8a2f-8353-49ec-8a9b-695b32f03d20@github.com> Message-ID: On Tue, 28 Nov 2023 23:25:27 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The man page of jcmd will be updated in a separate PR. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Apply man changes @yftsai Can you update the man page output in the PR description? Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1836781584 From cjplummer at openjdk.org Fri Dec 1 21:34:39 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 1 Dec 2023 21:34:39 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v5] In-Reply-To: <81dXSHvLQMGj3s1BcBs8fmJUEoJpaU-5wBRSIjnztMM=.d53f8a2f-8353-49ec-8a9b-695b32f03d20@github.com> References: <81dXSHvLQMGj3s1BcBs8fmJUEoJpaU-5wBRSIjnztMM=.d53f8a2f-8353-49ec-8a9b-695b32f03d20@github.com> Message-ID: On Tue, 28 Nov 2023 23:25:27 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The man page of jcmd will be updated in a separate PR. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Apply man changes src/hotspot/share/code/codeCache.cpp line 1809: > 1807: } > 1808: > 1809: void CodeCache::write_perf_map(const char* filename) { Why not have a `filename == nullptr` indicate that the default should be used. Then you don't need CodeCache::DefaultPerfMapFile. You can just have a private `CodeCache::defaultPerfmapFileName()` method. src/hotspot/share/code/codeCache.hpp line 232: > 230: const char* name() const { return _name; } > 231: };) > 232: Multi-line sections like this should really use `#ifdef LINUX`. src/hotspot/share/runtime/java.cpp line 507: > 505: if (DumpPerfMapAtExit) { > 506: CodeCache::DefaultPerfMapFile file; > 507: CodeCache::write_perf_map(file.name()); It's a bit inconsistent to support a user provided filename for the dcmd but not when using `DumpPerfMapAtExit`. Perhaps add `PerfMapFilename`. I'm not insisting, but something to consider. Would be good for the compiler team to comment on this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15871#discussion_r1412585004 PR Review Comment: https://git.openjdk.org/jdk/pull/15871#discussion_r1412579044 PR Review Comment: https://git.openjdk.org/jdk/pull/15871#discussion_r1412582441 From matsaave at openjdk.org Fri Dec 1 22:17:03 2023 From: matsaave at openjdk.org (Matias Saavedra Silva) Date: Fri, 1 Dec 2023 22:17:03 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 23:24:53 GMT, Ioi Lam wrote: > This is a simple clean up that moves the code for initializing the CDS config states from arguments.cpp to cdsConfig.cpp > > I renamed a few functions, but otherwise the code is unchanged. > > - `get_default_shared_archive_path()` -> `default_archive_path()` > - `GetSharedArchivePath()` -> `static_archive_path()` > - `GetSharedDynamicArchivePath()` -> `dynamic_archive_path()` > > There's also less `#if INCLUDE_CDS` since the entire cdsConfig.cpp file is compiled only if CDS is enabled. This looks good! I think this is a good opportunity to refactor some of the code for better readability so I left some comments below. src/hotspot/share/cds/cdsConfig.cpp line 101: > 99: void CDSConfig::extract_shared_archive_paths(const char* archive_path, > 100: char** base_archive_path, > 101: char** top_archive_path) { Could you align these arguments? src/hotspot/share/cds/cdsConfig.cpp line 125: > 123: } > 124: > 125: void CDSConfig::init_shared_archive_paths() { Now that I see this there is a lot of indentation thanks to the nested conditionals. I don't have much to offer but is there a cleaner way to format this method? Maybe you can extract the code in `if (archives == 1)` into its own method for better readability. src/hotspot/share/runtime/arguments.cpp line 1262: > 1260: } > 1261: > 1262: CDSConfig::check_system_property(key, value); I see this is only called once, do you expect this method to be used again? It may be unnecessary to extract this code into its own method. ------------- PR Review: https://git.openjdk.org/jdk/pull/16868#pullrequestreview-1760626242 PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412613074 PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412616593 PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412610795 From jjoo at openjdk.org Fri Dec 1 22:40:55 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Fri, 1 Dec 2023 22:40:55 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Fri, 1 Dec 2023 21:01:29 GMT, Man Cao wrote: >> I agree that the counter is valuable if always up-to-date, but if it is out of sync compared to the "concurrent counters" I think it will confuse some users. So if we want to keep it I think we should try to keep it in sync. >> >> I suppose adding a lock for updating `gc_total` should be ok. In the safepoint case we should never contend on that lock and for the concurrent updates it should not be a big deal. Basically what I think would be to change `update_counter(...)` to do something like: >> >> if (CPUTimeGroups::is_gc_counter(name)) { >> >> instance->get_counter(CPUTimeGroups::CPUTimeType::gc_total)->inc(net_cpu_time); >> } >> >> >> This way we would also be able to remove the publish part above, right? Any other problems with this approach? > > I think the ideal approach to simplify this is to support Atomic operation on a `PerfCounter`. > We could either introduce a `PerfAtomicCounter`/`PerfAtomicLongCounter` class, or perform `Atomic::add()` on the `PerfData::_valuep` pointer. There's already `PerfData::get_address()`, so we might be able to do: > > > Atomic::add((volatile jlong *)(instance->get_counter(CPUTimeGroups::CPUTimeType::gc_total)->get_address()), net_cpu_time); > > > However, a new class `PerfAtomicCounter` is likely cleaner. E.g., we may also want to make `PerfAtomicCounter::sample()` use a CAS. It is probably better to introduce `PerfAtomicCounter` in a separate RFE later. > > Would the `Atomic::add()` with `PerfData::get_address()` approach be OK for now, or would we rather introduce a lock, or leave the `gc_total` mechanism as-is and address the out-of-sync-ness in a follow-up RFE? > > IMO the out-of-sync-ness problem is minor, because users are likely to either look at a single `gc_total` counter, or look at each individual GC CPU counter and disregard `gc_total`. In the interest of the RDP1 deadline, should we leave improving the sync issues with gc_total to a separate RFE? (Especially given that a "correct" design may take some time to come up with, and that gc_total being slightly out of sync is not a major issue.) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1412631873 From iklam at openjdk.org Sat Dec 2 00:38:58 2023 From: iklam at openjdk.org (Ioi Lam) Date: Sat, 2 Dec 2023 00:38:58 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v2] In-Reply-To: References: Message-ID: > This is a simple clean up that moves the code for initializing the CDS config states from arguments.cpp to cdsConfig.cpp > > I renamed a few functions, but otherwise the code is unchanged. > > - `get_default_shared_archive_path()` -> `default_archive_path()` > - `GetSharedArchivePath()` -> `static_archive_path()` > - `GetSharedDynamicArchivePath()` -> `dynamic_archive_path()` > > There's also less `#if INCLUDE_CDS` since the entire cdsConfig.cpp file is compiled only if CDS is enabled. Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: fixed indentation ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16868/files - new: https://git.openjdk.org/jdk/pull/16868/files/72f3e44c..01dd47bc Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16868&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16868&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16868.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16868/head:pull/16868 PR: https://git.openjdk.org/jdk/pull/16868 From iklam at openjdk.org Sat Dec 2 00:38:58 2023 From: iklam at openjdk.org (Ioi Lam) Date: Sat, 2 Dec 2023 00:38:58 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v2] In-Reply-To: <7tqgQeAidnvr6kp8hkHZ4QPCV_pFbVvWbafTiWzEEbg=.0e728f7b-0c77-4012-bc3d-6cec099b9e68@github.com> References: <7tqgQeAidnvr6kp8hkHZ4QPCV_pFbVvWbafTiWzEEbg=.0e728f7b-0c77-4012-bc3d-6cec099b9e68@github.com> Message-ID: On Wed, 29 Nov 2023 21:53:06 GMT, Calvin Cheung wrote: >> Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: >> >> fixed indentation > > src/hotspot/share/cds/cdsConfig.cpp line 34: > >> 32: #include "logging/log.hpp" >> 33: #include "runtime/arguments.hpp" >> 34: #include "runtime/java.hpp" > > I was able to build with your patch without including `java.hpp`. > The #include java.hpp could also be removed from arguments.cpp. cdsConfig.cpp needs the declaration of `vm_exit_during_initialization()` from java.hpp. Although java.hpp is included by arguments.hpp, we usually try to avoid such indirectly inclusions. Otherwise if arguments.hpp is changed to no longer include java.hpp, then cdsConfig.hpp will fail to compile. I am not sure about arguments.cpp -- if java.hpp is already included by arguments.hpp, do we need to explicitly include it in arguments.cpp? I'll leave that alone in this PR. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412682898 From iklam at openjdk.org Sat Dec 2 00:39:03 2023 From: iklam at openjdk.org (Ioi Lam) Date: Sat, 2 Dec 2023 00:39:03 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v2] In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 22:04:22 GMT, Matias Saavedra Silva wrote: >> Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: >> >> fixed indentation > > src/hotspot/share/cds/cdsConfig.cpp line 101: > >> 99: void CDSConfig::extract_shared_archive_paths(const char* archive_path, >> 100: char** base_archive_path, >> 101: char** top_archive_path) { > > Could you align these arguments? Fixed. > src/hotspot/share/cds/cdsConfig.cpp line 125: > >> 123: } >> 124: >> 125: void CDSConfig::init_shared_archive_paths() { > > Now that I see this there is a lot of indentation thanks to the nested conditionals. I don't have much to offer but is there a cleaner way to format this method? Maybe you can extract the code in `if (archives == 1)` into its own method for better readability. I want to keep the code change minimal while moving code from one file to another. I'll refactor this function in a follow-on PR. That way it will be easier to track the code history. > src/hotspot/share/runtime/arguments.cpp line 1262: > >> 1260: } >> 1261: >> 1262: CDSConfig::check_system_property(key, value); > > I see this is only called once, do you expect this method to be used again? It may be unnecessary to extract this code into its own method. I wanted to move the code from arguments.cpp to cdsConfig.cpp, so I had to put it in a new function. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412683767 PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412683760 PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412683786 From jjoo at openjdk.org Sat Dec 2 01:25:53 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 2 Dec 2023 01:25:53 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: <810qMt__o90-A1Csix4IiygZEpyP09w8tisrwY5mQC4=.b8696cfe-2323-4a6d-884c-47df6568a337@github.com> References: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> <810qMt__o90-A1Csix4IiygZEpyP09w8tisrwY5mQC4=.b8696cfe-2323-4a6d-884c-47df6568a337@github.com> Message-ID: On Fri, 1 Dec 2023 16:19:49 GMT, Volker Simonis wrote: >> src/hotspot/share/gc/parallel/parallelScavengeHeap.cpp line 905: >> >>> 903: gc_threads_do(&tttc); >>> 904: >>> 905: CPUTimeCounters::publish_gc_total_cpu_time(); >> >> As I suggested in the other comment, maybe we should not keep the total counter, but if we do we need to make sure the destructor of the closure is run before the call to `publish_gc_total_cpu_time()`. Otherwise we will publish a not yet updated value. > > I still think that a total counter is useful and I'd appreciate if you can keep it. To second what @caoman said before, it is GC agnostic, easy to use even for non GC experts and future proof with regards to implementation changes in the GCs. Please keep it. Put the closure in a scope, I think that should address the concern. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1412694432 From duke at openjdk.org Sat Dec 2 01:28:59 2023 From: duke at openjdk.org (Yi-Fan Tsai) Date: Sat, 2 Dec 2023 01:28:59 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v6] In-Reply-To: References: Message-ID: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. > > `jcmd PID help Compiler.perfmap` shows the following usage. > > > Compiler.perfmap > Write map file for Linux perf tool. > > Impact: Low > > Syntax : Compiler.perfmap [] > > Arguments: > filename : [optional] Name of the map file (STRING, no default value) > > > The man page of jcmd will be updated in a separate PR. Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: Remove DefaultPerfMapFile ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15871/files - new: https://git.openjdk.org/jdk/pull/15871/files/a7dcf426..6a854920 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=04-05 Stats: 26 lines in 4 files changed: 9 ins; 15 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/15871.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15871/head:pull/15871 PR: https://git.openjdk.org/jdk/pull/15871 From amenkov at openjdk.org Sat Dec 2 01:51:37 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Sat, 2 Dec 2023 01:51:37 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: On Fri, 1 Dec 2023 05:43:41 GMT, Chris Plummer wrote: > One concern I have is when there is a large number of virtual threads, the dump may take too long and the hprof file gets bloated. My concern mostly comes from the large number of virtual thread stack traces that will be present. Dumping all these hprof records related to unmounted virtual threads could do more harm than good in some instances, and we may want a way for the user to disable it. My understanding that information about references is one of the most important things for dump analysis (and that's what the issue about). So we cannot avoid stack unwinding for unmounted virtual threads. As for heapdump file size, each stack trace adds 21 + 53 * frame_number bytes for 64bit system (uncompressed data) So for 10 frames it adds ~550 bytes, for 20 frames ~1.1KB I'm not sure if stack traces are important for analysis, maybe we it makes sense to add an option to not include them in heap dump (for both platform and virtual threads). > > It would be nice if there could be some data sharing between threads with identical stack traces, but I don't see a way to do that with the current hprof spec. Hprof spec says nothing about 1:1 relation between threads and stack traces, so theoretically several HPROF_GC_ROOT_THREAD_OBJ subrecords may refer to the same stack trace, but search for identical stack traces may be expensive. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16665#issuecomment-1836988313 From lmesnik at openjdk.org Sat Dec 2 02:33:40 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Sat, 2 Dec 2023 02:33:40 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: On Thu, 30 Nov 2023 00:26:25 GMT, Alex Menkov wrote: >> The change impelements dumping of unmounted virtual threads data (stack traces and stack references). >> Unmounted vthreads can be detected only by iterating over the heap, but hprof stack trace records (HPROF_FRAME/HPROF_TRACE) should be written before HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT. >> HeapDumper supports segment dump (parallel dump to separate files with subsequent file merge outside of safepoint), the fix switches HeapDumper to always use segment dump: 1st segment contains only non-heap data, other segments are used for dumping heap objects. For serial dumping single-threaded dumping is performed, but 2 segments are created anyway. >> When HeapObjectDumper detects unmounted virtual thread, it writes HPROF_FRAME/HPROF_TRACE records to the 1st segment ("global writer"), and writes thread object (HPROF_GC_ROOT_JAVA_FRAME) and stack references (HPROF_GC_ROOT_JAVA_FRAME/HPROF_GC_ROOT_JNI_LOCAL) to the HeapObjectDumper segment. >> As parallel dumpers may write HPROF_FRAME/HPROF_TRACE concurrently and VMDumper needs to write non-heap data before heap object dumpers can write virtual threads data, writing to global writer is protected with DumperController::_global_writer_lock. >> >> Testing: run tests which perform heap dump (in different scenarios): >> - test/hotspot/jtreg/serviceability >> - test/hotspot/jtreg/runtime/ErrorHandling >> - test/hotspot/jtreg/gc/epsilon >> - test/jdk/sun/tools/jhsdb > > Alex Menkov has updated the pull request incrementally with two additional commits since the last revision: > > - feedback > - prepare_parallel_dum Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16665#pullrequestreview-1760758567 From ccheung at openjdk.org Sat Dec 2 03:38:43 2023 From: ccheung at openjdk.org (Calvin Cheung) Date: Sat, 2 Dec 2023 03:38:43 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v2] In-Reply-To: References: Message-ID: On Sat, 2 Dec 2023 00:38:58 GMT, Ioi Lam wrote: >> This is a simple clean up that moves the code for initializing the CDS config states from arguments.cpp to cdsConfig.cpp >> >> I renamed a few functions, but otherwise the code is unchanged. >> >> - `get_default_shared_archive_path()` -> `default_archive_path()` >> - `GetSharedArchivePath()` -> `static_archive_path()` >> - `GetSharedDynamicArchivePath()` -> `dynamic_archive_path()` >> >> There's also less `#if INCLUDE_CDS` since the entire cdsConfig.cpp file is compiled only if CDS is enabled. > > Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: > > fixed indentation Marked as reviewed by ccheung (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16868#pullrequestreview-1760771168 From ccheung at openjdk.org Sat Dec 2 03:38:45 2023 From: ccheung at openjdk.org (Calvin Cheung) Date: Sat, 2 Dec 2023 03:38:45 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v2] In-Reply-To: References: <7tqgQeAidnvr6kp8hkHZ4QPCV_pFbVvWbafTiWzEEbg=.0e728f7b-0c77-4012-bc3d-6cec099b9e68@github.com> Message-ID: On Sat, 2 Dec 2023 00:32:30 GMT, Ioi Lam wrote: >> src/hotspot/share/cds/cdsConfig.cpp line 34: >> >>> 32: #include "logging/log.hpp" >>> 33: #include "runtime/arguments.hpp" >>> 34: #include "runtime/java.hpp" >> >> I was able to build with your patch without including `java.hpp`. >> The #include java.hpp could also be removed from arguments.cpp. > > cdsConfig.cpp needs the declaration of `vm_exit_during_initialization()` from java.hpp. Although java.hpp is included by arguments.hpp, we usually try to avoid such indirectly inclusions. Otherwise if arguments.hpp is changed to no longer include java.hpp, then cdsConfig.hpp will fail to compile. > > I am not sure about arguments.cpp -- if java.hpp is already included by arguments.hpp, do we need to explicitly include it in arguments.cpp? I'll leave that alone in this PR. Thanks for the explanation. Looks good then. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16868#discussion_r1412714559 From stuefe at openjdk.org Sat Dec 2 07:08:40 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Sat, 2 Dec 2023 07:08:40 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v2] In-Reply-To: References: Message-ID: <_yjGgwhpDdYsUK7I-FrhA_j4Efi326KJqgD-p4GjK8c=.0840954b-b06f-446e-bcfc-f0eb9d7a5855@github.com> On Sat, 2 Dec 2023 00:38:58 GMT, Ioi Lam wrote: >> This is a simple clean up that moves the code for initializing the CDS config states from arguments.cpp to cdsConfig.cpp >> >> I renamed a few functions, but otherwise the code is unchanged. >> >> - `get_default_shared_archive_path()` -> `default_archive_path()` >> - `GetSharedArchivePath()` -> `static_archive_path()` >> - `GetSharedDynamicArchivePath()` -> `dynamic_archive_path()` >> >> There's also less `#if INCLUDE_CDS` since the entire cdsConfig.cpp file is compiled only if CDS is enabled. > > Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: > > fixed indentation Looks good. Did not find any functional difference to the original code. ------------- Marked as reviewed by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16868#pullrequestreview-1760796058 From jjoo at openjdk.org Sat Dec 2 07:37:26 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 2 Dec 2023 07:37:26 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v55] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Only create CPUTimeCounters if supported - Ensure TTTC is destructed before publishing ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/fcf00cfe..242fef84 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=54 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=53-54 Stats: 23 lines in 3 files changed: 11 ins; 0 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Sat Dec 2 07:37:28 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 2 Dec 2023 07:37:28 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: References: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> <4NJOslObcIY-G1nUAbKeiCWKK-wbhEw94avA2c2cJ7s=.2ff4560f-5184-40ad-982f-96570cc4a9fc@github.com> Message-ID: On Fri, 1 Dec 2023 14:42:22 GMT, Albert Mingkun Yang wrote: >> I would say it depends on the use-case and here when switching to use static functions to use the instance it felt more like an all-static class. I agree that it would be nice to avoid the additional memory usage if `UsePerfData` is `false` so I'm ok with keeping the instance if we add that. > >> It is easier to control the initialization order and timing of an on-heap singleton object than statics. > > It's generally true, but the init of CPUTimeCounters is not sensitive to ordering. > >> This could save some memory when UsePerfData is false. > > True, but the mem savings will be marginal at most, `PerfCounter* _cpu_time_counters[static_cast(CPUTimeGroups::CPUTimeType::COUNT)];` will be 8 * ~12 = ~96 bytes (including future cpu-time-type enums). > > (I don't have a strong preference here; however, I'd like to make the pros and cons explicit.) Added the check in the initializer ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1412749097 From dcubed at openjdk.org Sat Dec 2 17:42:45 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Sat, 2 Dec 2023 17:42:45 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 Message-ID: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> In the fix for the following bug: [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread JvmtiThreadState::state_for_while_locked() was changed to return nullptr for attaching JNI threads regardless of whether that JNI thread/JavaThread had a java.lang.Thread object. We should only filter out a JavaThread that's attaching via JNI if it has no java.lang.Thread object. This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. Mach5 Tier8 is in process. I'm going to need @jianglizhou to rerun her testing for: [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread since the test(s) for that fix are not yet integrated in the jdk/jdk repo. ------------- Commit messages: - 8321069: Test failure after JDK-8319935: exiting thread called setup_jvmti_thread_state Changes: https://git.openjdk.org/jdk/pull/16934/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16934&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321069 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16934.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16934/head:pull/16934 PR: https://git.openjdk.org/jdk/pull/16934 From dholmes at openjdk.org Mon Dec 4 00:43:56 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 00:43:56 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 11:49:31 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> ~Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated.~ >> >> Therefore, we need to perform `jmethodID` lookup for each method in an old class version that is getting purged, and null out the pointer of that `jmethodID` to break the link from `jmethodID` to the method instance that is about to get deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: > > Restrict cleanup to obsolete methods only >From the blog: > Yes! The methods are being deallocated for a class loader that is still alive. Okay so why does this happen and is it a reasonable thing to be happening? On the surface it sounds wrong to deallocate anything associated with a live classloader. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1837668001 From dholmes at openjdk.org Mon Dec 4 04:46:38 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 04:46:38 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 08:42:35 GMT, Alan Bateman wrote: >> Please review this simple clarification to the JVM TI spec regarding use of `JAVA_TOOL_OPTIONS` in regards to module options and their format. >> >> I do not believe this clarification needs a CSR request. >> >> Thanks. > > src/hotspot/share/prims/jvmti.xml line 746: > >> 744: JNI_CreateJavaVM (in the JNI Invocation API) will prepend these options to the options supplied >> 745: in its JavaVMInitArgs argument. Note that module related options must be expressed in their >> 746: "option=value" form (not "option value") for JNI_CreateJavaVM to process them correctly. > > This looks okay. I'm just comparing it to the text that we put into the JNI spec: > > "The module related options ... as option strings using their "option=value" format instead of their "option value" format. (Note the required = between "option" and "value".)" > > It uses "format" instead of "form" and also, the bit I think works well, is to point out "required =" to force the reader to re-read the previous sentence and see what the difference is in the formats. Thanks for looking at this @AlanBateman and @sspitsyn . I was trying to refer to the JNI text in a casual way rather than duplicating the actual content from there. This was more a strong hint/suggestion to "go read the JNI spec for details". Happy to change 'form' to 'format'. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16896#discussion_r1413357537 From dholmes at openjdk.org Mon Dec 4 04:51:03 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 04:51:03 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS [v2] In-Reply-To: References: Message-ID: <_aJZET6Y1stdgimipE-cn50sWScKU8VDEXoA1RgRhyU=.e3b01bfe-b5a4-4eab-8b23-73324347dd20@github.com> > Please review this simple clarification to the JVM TI spec regarding use of `JAVA_TOOL_OPTIONS` in regards to module options and their format. > > I do not believe this clarification needs a CSR request. > > Thanks. David Holmes has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Change form -> format - Merge branch 'master' into 8320860-jvmti-spec - 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16896/files - new: https://git.openjdk.org/jdk/pull/16896/files/7c8ca438..5417a109 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16896&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16896&range=00-01 Stats: 9803 lines in 335 files changed: 7601 ins; 1490 del; 712 mod Patch: https://git.openjdk.org/jdk/pull/16896.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16896/head:pull/16896 PR: https://git.openjdk.org/jdk/pull/16896 From jiangli at openjdk.org Mon Dec 4 05:19:36 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 4 Dec 2023 05:19:36 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Sat, 2 Dec 2023 17:15:57 GMT, Daniel D. Daugherty wrote: > In the fix for the following bug: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > JvmtiThreadState::state_for_while_locked() was changed to > return nullptr for attaching JNI threads regardless of whether > that JNI thread/JavaThread had a java.lang.Thread object. > > We should only filter out a JavaThread that's attaching via JNI > if it has no java.lang.Thread object. > > This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. > Mach5 Tier8 is in process. > > I'm going to need @jianglizhou to rerun her testing for: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > since the test(s) for that fix are not yet integrated in the jdk/jdk repo. @dcubed-ojdk Thanks for the notification. I just ran one of our affected test 100 times with JDK-8312174 change rolled back and with both following applied: - https://git.openjdk.org/jdk/pull/16642 - https://github.com/openjdk/jdk/pull/16934 All 100 runs passed without failure. I'm going to run all tests tonight, will report back later tomorrow if I see any issue. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1837858621 From dholmes at openjdk.org Mon Dec 4 06:34:40 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 06:34:40 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Sat, 2 Dec 2023 17:15:57 GMT, Daniel D. Daugherty wrote: > In the fix for the following bug: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > JvmtiThreadState::state_for_while_locked() was changed to > return nullptr for attaching JNI threads regardless of whether > that JNI thread/JavaThread had a java.lang.Thread object. > > We should only filter out a JavaThread that's attaching via JNI > if it has no java.lang.Thread object. > > This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. > Mach5 Tier8 is in process. > > I'm going to need @jianglizhou to rerun her testing for: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > since the test(s) for that fix are not yet integrated in the jdk/jdk repo. Initially I thought this was not the right fix as we should not be exposing an attaching thread that may still have a partially constructed `threadObj`. But this issue shows that we must expose such a thread because the constructor of the `Thread` object can trigger these events on the current thread so it must have a valid JVMTI state! Thanks. src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 90: > 88: // Don't add a JvmtiThreadState to a thread that is exiting or is attaching. > 89: // When a thread is attaching, it may not have a Java level thread object > 90: // created yet. The comment needs adjusting now - suggestion: // Don't add a JvmtiThreadState to a thread that is exiting, or is attaching // and does not yet have a Java level thread object allocated. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16934#pullrequestreview-1761678171 PR Review Comment: https://git.openjdk.org/jdk/pull/16934#discussion_r1413421607 From sjohanss at openjdk.org Mon Dec 4 09:32:11 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 4 Dec 2023 09:32:11 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v53] In-Reply-To: References: <-GX8bATX2hz3YWgnJbhTNEYbi4t8HxfdhYqBP-ulyGg=.0080d7b0-8e43-4b81-b885-1d4a742048cc@github.com> <810qMt__o90-A1Csix4IiygZEpyP09w8tisrwY5mQC4=.b8696cfe-2323-4a6d-884c-47df6568a337@github.com> Message-ID: On Sat, 2 Dec 2023 01:22:33 GMT, Jonathan Joo wrote: >> I still think that a total counter is useful and I'd appreciate if you can keep it. To second what @caoman said before, it is GC agnostic, easy to use even for non GC experts and future proof with regards to implementation changes in the GCs. Please keep it. > > Put the closure in a scope, I think that should address the concern. Yes, scoping it will work. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1413591437 From jbachorik at openjdk.org Mon Dec 4 10:12:57 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Mon, 4 Dec 2023 10:12:57 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 05:15:15 GMT, Thomas Stuefe wrote: >> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: >> >> Restrict cleanup to obsolete methods only > > I won't be able to review this this week, too snowed in atm. I can take a look next week. We can always just revert the change if needed. > > Thinking about Skara, I think as long as we have this confusing mixture of rules (hotspot wants 2 reviewers that are Reviewer/Committer, but some jdk libs only want one, but then you need two for desktop I think otherwise Phil gets angry) - we should hard-code the 2-reviewer rule into skara as default since it affects the lion's share of all changes. @tstuefe I got confused by the Skara tooling. I had a vague memory of some discussions going on about relaxing the requirement of 2 reviewers for some parts to the code base and I thought I was in a good shape seeing the Skara checkbox. ![Screenshot 2023-12-04 at 11 04 00](https://github.com/openjdk/jdk/assets/738413/a5e363ee-a9e0-4121-9677-c059aa299dd4) As for not having a review for the final version - I am not that restless. I specifically dismissed the previous review to avoid incidentally integrating based on a review of a version that was not actual. Then I asked @coleenp to re-do the review on the final bits (https://github.com/openjdk/jdk/pull/16662#issuecomment-1827432032) @dholmes-ora >Okay so why does this happen and is it a reasonable thing to be happening? On the surface it sounds wrong to deallocate anything associated with a live classloader. This is happening for previous versions of retransformed methods. As long as those methods are still on stack they are kept alive. But once they are not executing they are free to be destroyed. And this is where the problem was happening - the previous versions of methods were being destroyed but the associated jmethodIDs were not updated not to point to what became an invalid memory block. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1838221097 From stuefe at openjdk.org Mon Dec 4 11:14:55 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 4 Dec 2023 11:14:55 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: References: Message-ID: <277y6BBHCLkqj7vleST0dY2hg_iZdREWXlQDWqo7dUQ=.700d1ad2-d1de-4f09-b732-32ef5436360a@github.com> On Mon, 4 Dec 2023 00:41:23 GMT, David Holmes wrote: > From the blog: > > > Yes! The methods are being deallocated for a class loader that is still alive. > > Okay so why does this happen and is it a reasonable thing to be happening? On the surface it sounds wrong to deallocate anything associated with a live classloader. This sounds odd to me to. I know that we deallocate the old *byte code* of re-transformed classes; but `Method*` ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1838413238 From coleenp at openjdk.org Mon Dec 4 12:33:12 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Mon, 4 Dec 2023 12:33:12 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: References: Message-ID: <5gROVIKM5P_J15BGoYq0DFgDBuwvdEgWz6KNERecqz4=.32869544-e767-4b72-bad4-7b1ee4055444@github.com> On Wed, 29 Nov 2023 11:49:31 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> ~Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated.~ >> >> Therefore, we need to perform `jmethodID` lookup for each method in an old class version that is getting purged, and null out the pointer of that `jmethodID` to break the link from `jmethodID` to the method instance that is about to get deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: > > Restrict cleanup to obsolete methods only We deallocate the old Method* if nothing is referring to them (ie they're not running or being referenced for some other reason). Look at MetadataOnStackMark. The jmethodIDs to an obsolete method were a dangling pointer and we want to just null them out. The old Method* are attached to the scratch_version of the InstanceKlass so we essentially remove that and walk down to the Methods when none of them are found. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1838538568 From jkern at openjdk.org Mon Dec 4 12:33:26 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 4 Dec 2023 12:33:26 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v2] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: improve handling of nonexisting files ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/e756f496..0f6716db Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=00-01 Stats: 16 lines in 1 file changed: 4 ins; 5 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From coleenp at openjdk.org Mon Dec 4 12:36:58 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Mon, 4 Dec 2023 12:36:58 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: <277y6BBHCLkqj7vleST0dY2hg_iZdREWXlQDWqo7dUQ=.700d1ad2-d1de-4f09-b732-32ef5436360a@github.com> References: <277y6BBHCLkqj7vleST0dY2hg_iZdREWXlQDWqo7dUQ=.700d1ad2-d1de-4f09-b732-32ef5436360a@github.com> Message-ID: On Mon, 4 Dec 2023 11:11:27 GMT, Thomas Stuefe wrote: > Okay so why does this happen and is it a reasonable thing to be happening? On the surface it sounds wrong to deallocate anything associated with a live classloader. If we didn't deallocate these old methods, there would be a memory leak when using class redefinition. It would be a lot simpler if we didn't have to do this though. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1838543829 From alanb at openjdk.org Mon Dec 4 12:38:39 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 4 Dec 2023 12:38:39 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS [v2] In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 04:43:46 GMT, David Holmes wrote: >> src/hotspot/share/prims/jvmti.xml line 746: >> >>> 744: JNI_CreateJavaVM (in the JNI Invocation API) will prepend these options to the options supplied >>> 745: in its JavaVMInitArgs argument. Note that module related options must be expressed in their >>> 746: "option=value" form (not "option value") for JNI_CreateJavaVM to process them correctly. >> >> This looks okay. I'm just comparing it to the text that we put into the JNI spec: >> >> "The module related options ... as option strings using their "option=value" format instead of their "option value" format. (Note the required = between "option" and "value".)" >> >> It uses "format" instead of "form" and also, the bit I think works well, is to point out "required =" to force the reader to re-read the previous sentence and see what the difference is in the formats. > > Thanks for looking at this @AlanBateman and @sspitsyn . I was trying to refer to the JNI text in a casual way rather than duplicating the actual content from there. This was more a strong hint/suggestion to "go read the JNI spec for details". > > Happy to change 'form' to 'format'. I agree with not duplicating the entire sentence but what would you think about replacing the text in the parenthesis with "Note the required = between option and value". Only asking because "(not "option value")" isn't as clear that we want the reader to see that you can't use a space here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16896#discussion_r1413811389 From david.holmes at oracle.com Mon Dec 4 12:41:58 2023 From: david.holmes at oracle.com (David Holmes) Date: Mon, 4 Dec 2023 22:41:58 +1000 Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: <8VMA3XjqiXFbsQTa5IjAouBch2QjdhaRnERtcTTlxpg=.148cbc2c-8e73-4a6e-b980-682fb45b8ae9@github.com> References: <8VMA3XjqiXFbsQTa5IjAouBch2QjdhaRnERtcTTlxpg=.148cbc2c-8e73-4a6e-b980-682fb45b8ae9@github.com> Message-ID: <72662e8e-342a-4812-9f5d-4002d7810dd9@oracle.com> On 1/12/2023 2:08 pm, Alex Menkov wrote: > On Thu, 30 Nov 2023 21:11:08 GMT, Chris Plummer wrote: > >> I wasn't thinking in terms of the scheduler somehow no longer references the virtual thread, but instead the program no longer referencing the scheduler (and also not referencing the virtual thread). > > AFAIU unfinished unmounted virtual threads are referenced from other objects (they are parked on), so they can't be unreachable even is the application is not referencing them and the scheduler. There is (or was - there may be a property that affects this: trackAllThreads?) a scenario where a VT might park on a synchronization object which is not referenced from any other thread. The VT can never be unparked, and the sync object and the VT are reachable only from either other and so both can be GC'd. David ----- > ------------- > > PR Comment: https://git.openjdk.org/jdk/pull/16665#issuecomment-1835422117 From sjohanss at openjdk.org Mon Dec 4 13:10:59 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 4 Dec 2023 13:10:59 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v55] In-Reply-To: References: Message-ID: <1Pwrkk0q5RFSLTrUSYKVIXC1n21Ugot9M5jslG5O_B4=.1a364569-c869-4f4e-a747-2756a4873aa3@github.com> On Sat, 2 Dec 2023 07:37:26 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Only create CPUTimeCounters if supported > - Ensure TTTC is destructed before publishing Marked as reviewed by sjohanss (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1762364095 From sjohanss at openjdk.org Mon Dec 4 13:11:01 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 4 Dec 2023 13:11:01 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Fri, 1 Dec 2023 22:37:58 GMT, Jonathan Joo wrote: >> I think the ideal approach to simplify this is to support Atomic operation on a `PerfCounter`. >> We could either introduce a `PerfAtomicCounter`/`PerfAtomicLongCounter` class, or perform `Atomic::add()` on the `PerfData::_valuep` pointer. There's already `PerfData::get_address()`, so we might be able to do: >> >> >> Atomic::add((volatile jlong *)(instance->get_counter(CPUTimeGroups::CPUTimeType::gc_total)->get_address()), net_cpu_time); >> >> >> However, a new class `PerfAtomicCounter` is likely cleaner. E.g., we may also want to make `PerfAtomicCounter::sample()` use a CAS. It is probably better to introduce `PerfAtomicCounter` in a separate RFE later. >> >> Would the `Atomic::add()` with `PerfData::get_address()` approach be OK for now, or would we rather introduce a lock, or leave the `gc_total` mechanism as-is and address the out-of-sync-ness in a follow-up RFE? >> >> IMO the out-of-sync-ness problem is minor, because users are likely to either look at a single `gc_total` counter, or look at each individual GC CPU counter and disregard `gc_total`. > > In the interest of the RDP1 deadline, should we leave improving the sync issues with gc_total to a separate RFE? (Especially given that a "correct" design may take some time to come up with, and that gc_total being slightly out of sync is not a major issue.) Me and Albert discussed this again and we are ok with handling the `gc_total` sync issue as a follow up. Please create the RFE for that. If that would include needing a `PerfAtomicCounter`, that would a be its own RFE as well. For me I think a lock would be a good enough solution. >From our point of view having the counters out of sync for a long period of time (think a long concurrent mark cycle without any young collections updating the total) is not good since it shows that the counters are not incremented in sync. It would also be nice to avoid the two-step updating of the total time, so please try to find time to work on this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1413848440 From Alan.Bateman at oracle.com Mon Dec 4 13:20:59 2023 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 Dec 2023 13:20:59 +0000 Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: <72662e8e-342a-4812-9f5d-4002d7810dd9@oracle.com> References: <8VMA3XjqiXFbsQTa5IjAouBch2QjdhaRnERtcTTlxpg=.148cbc2c-8e73-4a6e-b980-682fb45b8ae9@github.com> <72662e8e-342a-4812-9f5d-4002d7810dd9@oracle.com> Message-ID: <5efb2178-fec7-4cd1-a752-45514d5538b5@oracle.com> On 04/12/2023 12:41, David Holmes wrote: > On 1/12/2023 2:08 pm, Alex Menkov wrote: >> On Thu, 30 Nov 2023 21:11:08 GMT, Chris Plummer >> wrote: >> >>> I wasn't thinking in terms of the scheduler somehow no longer >>> references the virtual thread, but instead the program no longer >>> referencing the scheduler (and also not referencing the virtual >>> thread). >> >> AFAIU unfinished unmounted virtual threads are referenced from other >> objects (they are parked on), so they can't be unreachable even is >> the application is not referencing them and the scheduler. > > There is (or was - there may be a property that affects this: > trackAllThreads?) a scenario where a VT might park on a > synchronization object which is not referenced from any other thread. > The VT can never be unparked, and the sync object and the VT are > reachable only from either other and so both can be GC'd. That's right, the door is not closed to introducing ephemeral threads in the future. Right now, virtual threads created directly with the Thread API remaining strongly reachable once started until they terminate. Virtual threads created in other containers (e.g. a thread-per-task ExecutorService) are kept reachable by the container. -Alan -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcubed at openjdk.org Mon Dec 4 18:16:52 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Mon, 4 Dec 2023 18:16:52 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 [v2] In-Reply-To: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: > In the fix for the following bug: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > JvmtiThreadState::state_for_while_locked() was changed to > return nullptr for attaching JNI threads regardless of whether > that JNI thread/JavaThread had a java.lang.Thread object. > > We should only filter out a JavaThread that's attaching via JNI > if it has no java.lang.Thread object. > > This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. > Mach5 Tier8 is in process. > > I'm going to need @jianglizhou to rerun her testing for: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > since the test(s) for that fix are not yet integrated in the jdk/jdk repo. Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: dholmes CR - adjust a comment. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16934/files - new: https://git.openjdk.org/jdk/pull/16934/files/5c9eb2ec..0de1484f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16934&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16934&range=00-01 Stats: 3 lines in 1 file changed: 0 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16934.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16934/head:pull/16934 PR: https://git.openjdk.org/jdk/pull/16934 From dcubed at openjdk.org Mon Dec 4 18:16:53 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Mon, 4 Dec 2023 18:16:53 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 05:16:59 GMT, Jiangli Zhou wrote: >> In the fix for the following bug: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> JvmtiThreadState::state_for_while_locked() was changed to >> return nullptr for attaching JNI threads regardless of whether >> that JNI thread/JavaThread had a java.lang.Thread object. >> >> We should only filter out a JavaThread that's attaching via JNI >> if it has no java.lang.Thread object. >> >> This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. >> Mach5 Tier8 is in process. >> >> I'm going to need @jianglizhou to rerun her testing for: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> since the test(s) for that fix are not yet integrated in the jdk/jdk repo. > > @dcubed-ojdk Thanks for the notification. I just ran one of our affected test 100 times with JDK-8312174 change rolled back and with both following applied: > > - https://git.openjdk.org/jdk/pull/16642 > - https://github.com/openjdk/jdk/pull/16934 > > All 100 runs passed without failure. I'm going to run all tests tonight, will report back later tomorrow if I see any issue. @jianglizhou - Thanks for doing the testing. Can you also do a review? We need two reviewers for this change. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839205099 From dcubed at openjdk.org Mon Dec 4 18:16:55 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Mon, 4 Dec 2023 18:16:55 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 [v2] In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 06:31:43 GMT, David Holmes wrote: >> Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: >> >> dholmes CR - adjust a comment. > > Initially I thought this was not the right fix as we should not be exposing an attaching thread that may still have a partially constructed `threadObj`. But this issue shows that we must expose such a thread because the constructor of the `Thread` object can trigger these events on the current thread so it must have a valid JVMTI state! > > Thanks. @dholmes-ora - Thanks for the review. > src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 90: > >> 88: // Don't add a JvmtiThreadState to a thread that is exiting or is attaching. >> 89: // When a thread is attaching, it may not have a Java level thread object >> 90: // created yet. > > The comment needs adjusting now - suggestion: > > // Don't add a JvmtiThreadState to a thread that is exiting, or is attaching > // and does not yet have a Java level thread object allocated. Thanks for the suggested change. I agree and I've applied it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839205636 PR Review Comment: https://git.openjdk.org/jdk/pull/16934#discussion_r1414306493 From chris.plummer at oracle.com Mon Dec 4 18:59:36 2023 From: chris.plummer at oracle.com (Chris Plummer) Date: Mon, 4 Dec 2023 10:59:36 -0800 Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: <5efb2178-fec7-4cd1-a752-45514d5538b5@oracle.com> References: <8VMA3XjqiXFbsQTa5IjAouBch2QjdhaRnERtcTTlxpg=.148cbc2c-8e73-4a6e-b980-682fb45b8ae9@github.com> <72662e8e-342a-4812-9f5d-4002d7810dd9@oracle.com> <5efb2178-fec7-4cd1-a752-45514d5538b5@oracle.com> Message-ID: <68639112-c37b-4a3a-9e8e-6502d691aeb2@oracle.com> On 12/4/23 5:20 AM, Alan Bateman wrote: > > > On 04/12/2023 12:41, David Holmes wrote: >> On 1/12/2023 2:08 pm, Alex Menkov wrote: >>> On Thu, 30 Nov 2023 21:11:08 GMT, Chris Plummer >>> wrote: >>> >>>> I wasn't thinking in terms of the scheduler somehow no longer >>>> references the virtual thread, but instead the program no longer >>>> referencing the scheduler (and also not referencing the virtual >>>> thread). >>> >>> AFAIU unfinished unmounted virtual threads are referenced from other >>> objects (they are parked on), so they can't be unreachable even is >>> the application is not referencing them and the scheduler. >> >> There is (or was - there may be a property that affects this: >> trackAllThreads?) a scenario where a VT might park on a >> synchronization object which is not referenced from any other thread. >> The VT can never be unparked, and the sync object and the VT are >> reachable only from either other and so both can be GC'd. > > That's right, the door is not closed to introducing ephemeral threads > in the future. Right now, virtual threads created directly with the > Thread API remaining strongly reachable once started until they > terminate. Virtual threads created in other containers (e.g. a > thread-per-task ExecutorService) are kept reachable by the container. > > -Alan So does this mean if the application is no longer referencing the ExecutorService, then we can have unreachable virtual threads that have not completed? This is really the point I've been getting at. Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiangli at openjdk.org Mon Dec 4 19:06:00 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 4 Dec 2023 19:06:00 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 [v2] In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 18:16:52 GMT, Daniel D. Daugherty wrote: >> In the fix for the following bug: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> JvmtiThreadState::state_for_while_locked() was changed to >> return nullptr for attaching JNI threads regardless of whether >> that JNI thread/JavaThread had a java.lang.Thread object. >> >> We should only filter out a JavaThread that's attaching via JNI >> if it has no java.lang.Thread object. >> >> This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. >> Mach5 Tier8 is in process. >> >> I'm going to need @jianglizhou to rerun her testing for: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> since the test(s) for that fix are not yet integrated in the jdk/jdk repo. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes CR - adjust a comment. Marked as reviewed by jiangli (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16934#pullrequestreview-1763177973 From jiangli at openjdk.org Mon Dec 4 19:06:05 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 4 Dec 2023 19:06:05 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 05:16:59 GMT, Jiangli Zhou wrote: >> In the fix for the following bug: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> JvmtiThreadState::state_for_while_locked() was changed to >> return nullptr for attaching JNI threads regardless of whether >> that JNI thread/JavaThread had a java.lang.Thread object. >> >> We should only filter out a JavaThread that's attaching via JNI >> if it has no java.lang.Thread object. >> >> This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. >> Mach5 Tier8 is in process. >> >> I'm going to need @jianglizhou to rerun her testing for: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> since the test(s) for that fix are not yet integrated in the jdk/jdk repo. > > @dcubed-ojdk Thanks for the notification. I just ran one of our affected test 100 times with JDK-8312174 change rolled back and with both following applied: > > - https://git.openjdk.org/jdk/pull/16642 > - https://github.com/openjdk/jdk/pull/16934 > > All 100 runs passed without failure. I'm going to run all tests tonight, will report back later tomorrow if I see any issue. > @jianglizhou - Thanks for doing the testing. Can you also do a review? We need two reviewers for this change. Complete test run finished. Checking the results, looks like there's no issue related to JVMTIThreadState change. @dcubed-ojdk Reducing the check to `(thread->threadObj() == nullptr && thread->is_attaching_via_jni())` looks ok. I just rechecked all usages of setup_jvmti_thread_state(). Currently it's used in three cases: - JvmtiDynamicCodeEventCollector::JvmtiDynamicCodeEventCollector() - JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() - JvmtiSampledObjectAllocEventCollector::start() JDK-8319935 ran into issue with JvmtiSampledObjectAllocEventCollector::start() call path. We changed JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample() to avoid sampling if there is no thread obj allocated for the attaching thread. We also changed JvmtiThreadState::state_for_while_locked to handle the attaching case and return null. @dcubed-ojdk and @dholmes-ora, is there a case JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() might also see an attaching thread without the thread obj allocated? JvmtiDynamicCodeEventCollector::JvmtiDynamicCodeEventCollector() call path probably is not affected by this case. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839282331 From cjplummer at openjdk.org Mon Dec 4 19:17:44 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 4 Dec 2023 19:17:44 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: On Sat, 2 Dec 2023 01:48:28 GMT, Alex Menkov wrote: > My understanding that information about references is one of the most important things for dump analysis (and that's what the issue about). So we cannot avoid stack unwinding for unmounted virtual threads. > As for heapdump file size, each stack trace adds 21 + 53 * frame_number bytes for 64bit system (uncompressed data) > So for 10 frames it adds ~550 bytes, for 20 frames ~1.1KB > I'm not sure if stack traces are important for analysis, maybe we it makes sense to add an option to not include them in heap dump (for both platform and virtual threads). My concern was with the memory usage of the stack traces. Yes I agree that including all referenced objects in the dump is important. An option just to leave out the stack traces seems like a good idea. > Hprof spec says nothing about 1:1 relation between threads and stack traces, so theoretically several HPROF_GC_ROOT_THREAD_OBJ subrecords may refer to the same stack trace, but search for identical stack traces may be expensive. I was thinking initially that you couldn't do this because each stack has it's own unique set of locals that are referenced, and the locals were part of the stack trace, but they are not. There is instead a HPROF_GC_ROOT_JAVA_FRAME record for each local reference. It does include the ThreadID, and we could probably get away with multiple Thread records referring to the same stack trace. but now I'm not seeing any indications that hprof stacks reference the locals, which seems odd. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16665#issuecomment-1839303561 From alanb at openjdk.org Mon Dec 4 19:27:59 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 4 Dec 2023 19:27:59 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit Message-ID: When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. For testing the existing ThreadAPI has new test cases. Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. ------------- Commit messages: - Leave onPinned to another PR - Initial commit Changes: https://git.openjdk.org/jdk/pull/16953/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16953&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321270 Stats: 150 lines in 4 files changed: 79 ins; 19 del; 52 mod Patch: https://git.openjdk.org/jdk/pull/16953.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16953/head:pull/16953 PR: https://git.openjdk.org/jdk/pull/16953 From sspitsyn at openjdk.org Mon Dec 4 19:51:22 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 4 Dec 2023 19:51:22 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 [v2] In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: <6d4H0ctgpsMVQRwwmXRHXZqroW7TNBDHkn7TXA4vFKM=.2026416a-1181-4cba-ae6f-ed8f36e51402@github.com> On Mon, 4 Dec 2023 18:16:52 GMT, Daniel D. Daugherty wrote: >> In the fix for the following bug: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> JvmtiThreadState::state_for_while_locked() was changed to >> return nullptr for attaching JNI threads regardless of whether >> that JNI thread/JavaThread had a java.lang.Thread object. >> >> We should only filter out a JavaThread that's attaching via JNI >> if it has no java.lang.Thread object. >> >> This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. >> Mach5 Tier8 is in process. >> >> I'm going to need @jianglizhou to rerun her testing for: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> since the test(s) for that fix are not yet integrated in the jdk/jdk repo. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes CR - adjust a comment. Looks good. Thank you for fixing this! ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16934#pullrequestreview-1763277330 From dcubed at openjdk.org Mon Dec 4 20:56:44 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Mon, 4 Dec 2023 20:56:44 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 [v2] In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 18:13:50 GMT, Daniel D. Daugherty wrote: >> Initially I thought this was not the right fix as we should not be exposing an attaching thread that may still have a partially constructed `threadObj`. But this issue shows that we must expose such a thread because the constructor of the `Thread` object can trigger these events on the current thread so it must have a valid JVMTI state! >> >> Thanks. > > @dholmes-ora - Thanks for the review. > @dcubed-ojdk and @dholmes-ora, is there a case > JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() > might also see an attaching thread without the thread obj allocated? JvmtiVMObjectAllocEventCollector is the code path that we ran into with the internal stress test, but in the case that tripped, we had a thread obj allocated. If we ran the same code path without the thread obj allocated, then we would return `nullptr` which was the goal of your original fix. I don't know if that specific test case is actually reached by any of our tests, but if such a case occurred, it did not result in the guarantee() firing or any other failure mode that I saw. The `guarantee(state != nullptr) failed: exiting thread called setup_jvmti_thread_state` has not been seen in Mach5 Tier[1-8] testing with this fix in place and I didn't see any other test failures that could be tracked back to problems with this code. A JvmtiVMObjectAllocEventCollector object is created in 34 places in jvm.cpp alone and I haven't checked all of those. I only checked out the one use in JVM_GetStackAccessControlContext. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839456136 From dcubed at openjdk.org Mon Dec 4 20:56:46 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Mon, 4 Dec 2023 20:56:46 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 19:01:05 GMT, Jiangli Zhou wrote: >> @dcubed-ojdk Thanks for the notification. I just ran one of our affected test 100 times with JDK-8312174 change rolled back and with both following applied: >> >> - https://git.openjdk.org/jdk/pull/16642 >> - https://github.com/openjdk/jdk/pull/16934 >> >> All 100 runs passed without failure. I'm going to run all tests tonight, will report back later tomorrow if I see any issue. > >> @jianglizhou - Thanks for doing the testing. Can you also do a review? We need two reviewers for this change. > > Complete test run finished. Checking the results, looks like there's no issue related to JVMTIThreadState change. > > @dcubed-ojdk Reducing the check to `(thread->threadObj() == nullptr && thread->is_attaching_via_jni())` looks ok. > > I just rechecked all usages of setup_jvmti_thread_state(). Currently it's used in three cases: > - JvmtiDynamicCodeEventCollector::JvmtiDynamicCodeEventCollector() > - JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() > - JvmtiSampledObjectAllocEventCollector::start() > > JDK-8319935 ran into issue with JvmtiSampledObjectAllocEventCollector::start() call path. We changed JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample() to avoid sampling if there is no thread obj allocated for the attaching thread. We also changed JvmtiThreadState::state_for_while_locked to handle the attaching case and return null. @dcubed-ojdk and @dholmes-ora, is there a case JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() might also see an attaching thread without the thread obj allocated? > > JvmtiDynamicCodeEventCollector::JvmtiDynamicCodeEventCollector() call path probably is not affected by this case. @jianglizhou and @sspitsyn - Thanks for the reviews. In the interest of reducing the noise in the Mach5 CI, I'm going ahead with integrating this fix without waiting for a reply to my comment above. If there are remaining issues, then we'll deal with them in a follow-up bug/RFE. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839457963 From dcubed at openjdk.org Mon Dec 4 20:56:47 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Mon, 4 Dec 2023 20:56:47 GMT Subject: Integrated: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: <7SZFxdpOF-_3CG-ffZPhqEeWUBNFInVY8e79nDUrB7o=.83552a41-5b74-47e4-b90a-f3a6402bb6a9@github.com> On Sat, 2 Dec 2023 17:15:57 GMT, Daniel D. Daugherty wrote: > In the fix for the following bug: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > JvmtiThreadState::state_for_while_locked() was changed to > return nullptr for attaching JNI threads regardless of whether > that JNI thread/JavaThread had a java.lang.Thread object. > > We should only filter out a JavaThread that's attaching via JNI > if it has no java.lang.Thread object. > > This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. > Mach5 Tier8 is in process. > > I'm going to need @jianglizhou to rerun her testing for: > > [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread > > since the test(s) for that fix are not yet integrated in the jdk/jdk repo. This pull request has now been integrated. Changeset: 30b5d427 Author: Daniel D. Daugherty URL: https://git.openjdk.org/jdk/commit/30b5d427350d03ec8b9eb39fbf06fbd1b1f66cd8 Stats: 5 lines in 1 file changed: 1 ins; 0 del; 4 mod 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 Reviewed-by: dholmes, jiangli, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/16934 From sspitsyn at openjdk.org Mon Dec 4 21:36:36 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 4 Dec 2023 21:36:36 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 16:08:32 GMT, Alan Bateman wrote: > When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. > > The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. > > For testing the existing ThreadAPI has new test cases. > > Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. src/java.base/share/classes/java/lang/VirtualThread.java line 116: > 114: private static final int NEW = 0; > 115: private static final int STARTED = 1; > 116: private static final int RUNNING = 2; // runnable-unmounted Just want to make sure this change is intentional. Before the comment was: `// runnable-mounted`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16953#discussion_r1414528008 From cjplummer at openjdk.org Mon Dec 4 21:51:36 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 4 Dec 2023 21:51:36 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v4] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 1 Dec 2023 07:59:49 GMT, Jaikiran Pai wrote: > I'm looking for one more Reviewer, please. You have 2 reviewers already. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16805#issuecomment-1839539742 From sspitsyn at openjdk.org Mon Dec 4 22:06:34 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 4 Dec 2023 22:06:34 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 16:08:32 GMT, Alan Bateman wrote: > When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. > > The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. > > For testing the existing ThreadAPI has new test cases. > > Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. src/hotspot/share/classfile/javaClasses.cpp line 1998: > 1996: case UNPARKED: > 1997: case YIELDING : > 1998: case YIELDED: Is this new state `YIELDED` really needed? It looks like `UNPARKED` is also appeared where `YIELDED` is used. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16953#discussion_r1414561935 From jiangli at openjdk.org Mon Dec 4 22:25:42 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 4 Dec 2023 22:25:42 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 20:52:46 GMT, Daniel D. Daugherty wrote: > In the interest of reducing the noise in the Mach5 CI, I'm going ahead with integrating this fix without waiting for a reply to my comment above. If there are remaining issues, then we'll deal with them in a follow-up bug/RFE. Thumbs up, thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839594297 From jiangli at openjdk.org Mon Dec 4 22:39:46 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 4 Dec 2023 22:39:46 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 20:52:46 GMT, Daniel D. Daugherty wrote: >>> @jianglizhou - Thanks for doing the testing. Can you also do a review? We need two reviewers for this change. >> >> Complete test run finished. Checking the results, looks like there's no issue related to JVMTIThreadState change. >> >> @dcubed-ojdk Reducing the check to `(thread->threadObj() == nullptr && thread->is_attaching_via_jni())` looks ok. >> >> I just rechecked all usages of setup_jvmti_thread_state(). Currently it's used in three cases: >> - JvmtiDynamicCodeEventCollector::JvmtiDynamicCodeEventCollector() >> - JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() >> - JvmtiSampledObjectAllocEventCollector::start() >> >> JDK-8319935 ran into issue with JvmtiSampledObjectAllocEventCollector::start() call path. We changed JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample() to avoid sampling if there is no thread obj allocated for the attaching thread. We also changed JvmtiThreadState::state_for_while_locked to handle the attaching case and return null. @dcubed-ojdk and @dholmes-ora, is there a case JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() might also see an attaching thread without the thread obj allocated? >> >> JvmtiDynamicCodeEventCollector::JvmtiDynamicCodeEventCollector() call path probably is not affected by this case. > > @jianglizhou and @sspitsyn - Thanks for the reviews. > > In the interest of reducing the noise in the Mach5 CI, I'm going ahead with > integrating this fix without waiting for a reply to my comment above. If there > are remaining issues, then we'll deal with them in a follow-up bug/RFE. > > @dcubed-ojdk and @dholmes-ora, is there a case > > JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() > > might also see an attaching thread without the thread obj allocated? > > JvmtiVMObjectAllocEventCollector is the code path that we ran into with the internal stress test, but in the case that tripped, we had a thread obj allocated. If we ran the same code path without the thread obj allocated, then we would return `nullptr` which was the goal of your original fix. > > I don't know if that specific test case is actually reached by any of our tests, but if such a case occurred, it did not result in the guarantee() firing or any other failure mode that I saw. The `guarantee(state != nullptr) failed: exiting thread called setup_jvmti_thread_state` has not been seen in Mach5 Tier[1-8] testing with this fix in place and I didn't see any other test failures that could be tracked back to problems with this code. > > A JvmtiVMObjectAllocEventCollector object is created in 34 places in jvm.cpp alone and I haven't checked all of those. I only checked out the one use in JVM_GetStackAccessControlContext. Thanks. I was wondering if we would need further work to handle `JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector()` with something similar to the JvmtiSampledObjectAllocEventCollector change, if it's possible to trigger the `guarantee(state != nullptr)` for attaching thread with no allocated thread obj. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839626256 From dholmes at openjdk.org Mon Dec 4 23:35:59 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 23:35:59 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: References: Message-ID: On Wed, 29 Nov 2023 11:49:31 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> ~Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated.~ >> >> Therefore, we need to perform `jmethodID` lookup for each method in an old class version that is getting purged, and null out the pointer of that `jmethodID` to break the link from `jmethodID` to the method instance that is about to get deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: > > Restrict cleanup to obsolete methods only Just for the record Coleen's review is marked as "Review applies to [81e31dae](https://git.openjdk.org/jdk/pull/16662/files/81e31daeef4c68352368a90e3ab453ba9b33650c)" - which is not the final version. The skara tooling does not currently support our rules but it remains as always that non-trivial Hotspot changes require two reviewers. Thanks for the additional explanations Coleen. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1839721631 From manc at openjdk.org Mon Dec 4 23:37:50 2023 From: manc at openjdk.org (Man Cao) Date: Mon, 4 Dec 2023 23:37:50 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v48] In-Reply-To: References: <_lEBVrWV8wrVbmhOiu3AAqPJo_xBs718ZtA9V-VSzGM=.253c0ec8-256e-4dee-b125-90be6338e4b8@github.com> Message-ID: On Mon, 4 Dec 2023 13:07:15 GMT, Stefan Johansson wrote: >> In the interest of the RDP1 deadline, should we leave improving the sync issues with gc_total to a separate RFE? (Especially given that a "correct" design may take some time to come up with, and that gc_total being slightly out of sync is not a major issue.) > > Me and Albert discussed this again and we are ok with handling the `gc_total` sync issue as a follow up. Please create the RFE for that. If that would include needing a `PerfAtomicCounter`, that would a be its own RFE as well. For me I think a lock would be a good enough solution. > > From our point of view having the counters out of sync for a long period of time (think a long concurrent mark cycle without any young collections updating the total) is not good since it shows that the counters are not incremented in sync. It would also be nice to avoid the two-step updating of the total time, so please try to find time to work on this. Thanks, opened https://bugs.openjdk.org/browse/JDK-8321304. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1414634977 From dholmes at openjdk.org Mon Dec 4 23:38:33 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 23:38:33 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS [v2] In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 12:35:42 GMT, Alan Bateman wrote: >> Thanks for looking at this @AlanBateman and @sspitsyn . I was trying to refer to the JNI text in a casual way rather than duplicating the actual content from there. This was more a strong hint/suggestion to "go read the JNI spec for details". >> >> Happy to change 'form' to 'format'. > > I agree with not duplicating the entire sentence but what would you think about replacing the text in the parenthesis with "Note the required = between option and value". Only asking because "(not "option value")" isn't as clear that we want the reader to see that you can't use a space here. Okay. I thought it very clear to say "use format X (not format Y)", but sure I can change it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16896#discussion_r1414637979 From dholmes at openjdk.org Mon Dec 4 23:47:12 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 23:47:12 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS [v3] In-Reply-To: References: Message-ID: > Please review this simple clarification to the JVM TI spec regarding use of `JAVA_TOOL_OPTIONS` in regards to module options and their format. > > I do not believe this clarification needs a CSR request. > > Thanks. David Holmes has updated the pull request incrementally with one additional commit since the last revision: reviewer feedback ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16896/files - new: https://git.openjdk.org/jdk/pull/16896/files/5417a109..7cb6f439 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16896&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16896&range=01-02 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16896.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16896/head:pull/16896 PR: https://git.openjdk.org/jdk/pull/16896 From dholmes at openjdk.org Mon Dec 4 23:52:44 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 4 Dec 2023 23:52:44 GMT Subject: RFR: 8321069: JvmtiThreadState::state_for_while_locked() returns nullptr for an attached JNI thread with a java.lang.Thread object after JDK-8319935 [v2] In-Reply-To: References: <9emHn8lurQQb_qILA9O3wa_0DHc6ia-JxMFu5iEoQlE=.e09f743d-ca4d-4de4-ae2f-ccbc26aa79ea@github.com> Message-ID: On Mon, 4 Dec 2023 18:16:52 GMT, Daniel D. Daugherty wrote: >> In the fix for the following bug: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> JvmtiThreadState::state_for_while_locked() was changed to >> return nullptr for attaching JNI threads regardless of whether >> that JNI thread/JavaThread had a java.lang.Thread object. >> >> We should only filter out a JavaThread that's attaching via JNI >> if it has no java.lang.Thread object. >> >> This fix has been tested with Mach5 Tier[1-7] and there are no related test failures. >> Mach5 Tier8 is in process. >> >> I'm going to need @jianglizhou to rerun her testing for: >> >> [JDK-8319935](https://bugs.openjdk.org/browse/JDK-8319935) Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread >> >> since the test(s) for that fix are not yet integrated in the jdk/jdk repo. > > Daniel D. Daugherty has updated the pull request incrementally with one additional commit since the last revision: > > dholmes CR - adjust a comment. We would need to examine all uses of JvmtiVMObjectAllocEventCollector::JvmtiVMObjectAllocEventCollector() to see if they can occur in the context of an attaching thread prior to allocation of the thread object. I suspect not because we allocate immediately after creating the JavaThread. The issue we hit here was events triggered by the execution of the Java code for the Thread constructor. It may be that we are just lucky that the same Java code doesn't lead to the VMObjectAlloc event though. ??? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16934#issuecomment-1839755493 From manc at openjdk.org Tue Dec 5 00:22:50 2023 From: manc at openjdk.org (Man Cao) Date: Tue, 5 Dec 2023 00:22:50 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v55] In-Reply-To: References: Message-ID: On Sat, 2 Dec 2023 07:37:26 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Only create CPUTimeCounters if supported > - Ensure TTTC is destructed before publishing Marked as reviewed by manc (Committer). ------------- PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1763690561 From sspitsyn at openjdk.org Tue Dec 5 00:29:55 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 00:29:55 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected Message-ID: The fix is for a regression caused by: [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 The fix of 8308614 just triggered a known issue: [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option The fix is just a work around with the extra checks of the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. Testing: - In progress: Test with tiers 1-6 ------------- Commit messages: - 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected Changes: https://git.openjdk.org/jdk/pull/16961/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16961&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321219 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16961.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16961/head:pull/16961 PR: https://git.openjdk.org/jdk/pull/16961 From sspitsyn at openjdk.org Tue Dec 5 00:38:33 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 00:38:33 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS [v3] In-Reply-To: References: Message-ID: <1-R788wjIXpzWA2bHtmE9zVic73zNxLCN_N7zG3Cltw=.b4435a94-77ca-4a53-a150-0faaff94f41f@github.com> On Mon, 4 Dec 2023 23:47:12 GMT, David Holmes wrote: >> Please review this simple clarification to the JVM TI spec regarding use of `JAVA_TOOL_OPTIONS` in regards to module options and their format. >> >> I do not believe this clarification needs a CSR request. >> >> Thanks. > > David Holmes has updated the pull request incrementally with one additional commit since the last revision: > > reviewer feedback Marked as reviewed by sspitsyn (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16896#pullrequestreview-1763703748 From jpai at openjdk.org Tue Dec 5 00:57:37 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Tue, 5 Dec 2023 00:57:37 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v4] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 13:00:33 GMT, Jaikiran Pai wrote: >> Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? >> >> As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. >> >> The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). >> >> A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. >> >> tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. > > Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision: > > Alan's suggestion - don't share ServiceLoader Hello Chris, I misunderstood the 2 Reviewer rule for serviceablitiy area then. I thought it was 2 Reviewers (capital R). I'll go ahead and integrate this shortly. Thank you all for the help and reviews. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16805#issuecomment-1839816547 From jpai at openjdk.org Tue Dec 5 01:10:46 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Tue, 5 Dec 2023 01:10:46 GMT Subject: Integrated: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently In-Reply-To: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 06:06:16 GMT, Jaikiran Pai wrote: > Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? > > As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. > > The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). > > A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. > > tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. This pull request has now been integrated. Changeset: 81484d8c Author: Jaikiran Pai URL: https://git.openjdk.org/jdk/commit/81484d8c0520cf55ec58fc7b4c81880e69537674 Stats: 103 lines in 2 files changed: 93 ins; 7 del; 3 mod 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently Reviewed-by: alanb, kevinw ------------- PR: https://git.openjdk.org/jdk/pull/16805 From amenkov at openjdk.org Tue Dec 5 01:23:34 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 5 Dec 2023 01:23:34 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: On Mon, 4 Dec 2023 19:14:59 GMT, Chris Plummer wrote: > > My understanding that information about references is one of the most important things for dump analysis (and that's what the issue about). So we cannot avoid stack unwinding for unmounted virtual threads. > > As for heapdump file size, each stack trace adds 21 + 53 * frame_number bytes for 64bit system (uncompressed data) > > So for 10 frames it adds ~550 bytes, for 20 frames ~1.1KB > > I'm not sure if stack traces are important for analysis, maybe we it makes sense to add an option to not include them in heap dump (for both platform and virtual threads). > > My concern was with the memory usage of the stack traces. Yes I agree that including all referenced objects in the dump is important. An option just to leave out the stack traces seems like a good idea. VM_HeapDumper caches stack traces for platform/carrier and mounted virtual threads only. For unmounted virtual threads ThreadDumper objects are created on the stack (see `VM_HeapDumper::dump_vthread`), so I don't see problems with memory usage even huge number of unmounted vthreads. I think an option to exclude stack traces from heap dump is a separate task. > > > Hprof spec says nothing about 1:1 relation between threads and stack traces, so theoretically several HPROF_GC_ROOT_THREAD_OBJ subrecords may refer to the same stack trace, but search for identical stack traces may be expensive. > > I was thinking initially that you couldn't do this because each stack has it's own unique set of locals that are referenced, and the locals were part of the stack trace, but they are not. There is instead a HPROF_GC_ROOT_JAVA_FRAME record for each local reference. It does include the ThreadID, and we could probably get away with multiple Thread records referring to the same stack trace. I think this possible improvement is out of scope for this PR ------------- PR Comment: https://git.openjdk.org/jdk/pull/16665#issuecomment-1839838068 From cjplummer at openjdk.org Tue Dec 5 01:44:36 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 5 Dec 2023 01:44:36 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: On Tue, 5 Dec 2023 01:20:28 GMT, Alex Menkov wrote: > VM_HeapDumper caches stack traces for platform/carrier and mounted virtual threads only. > For unmounted virtual threads ThreadDumper objects are created on the stack (see `VM_HeapDumper::dump_vthread`), so I don't see problems with memory usage even huge number of unmounted vthreads. > I think an option to exclude stack traces from heap dump is a separate task. I was actually referring to the footprint of the hprof file, not the in process memory usage while producing it. My concern with not doing the option to exclude stack traces now is that it could result in some unusable or unmanageably large heap dumps, or tools simply being overwhelmed by the number of threads. For example, I just looked at the VisualVM threads view, and it just produces a scrollable list of all threads. What happens if there are suddenly 10's of thousands if not millions of threads? If we are lucky is doesn't choke on them and the platform threads are first in the list, but this is the type of thing I'd like to see testing of before pushing this change. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16665#issuecomment-1839855843 From dholmes at openjdk.org Tue Dec 5 04:44:33 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 5 Dec 2023 04:44:33 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 00:23:45 GMT, Serguei Spitsyn wrote: > The fix is for a regression caused by: > [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 > > The fix of 8308614 just triggered a known issue: > [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option > > The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. > > Testing: > - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix > - In progress: Test with tiers 1-6 src/hotspot/share/prims/jvmtiThreadState.cpp line 562: > 560: if (JvmtiThreadState::seen_interp_only_mode() || > 561: JvmtiExport::should_post_field_access() || > 562: JvmtiExport::should_post_field_modification()){ The comment needs updating to explain the extra checks. Can't say I see the connection with [8316283](https://bugs.openjdk.org/browse/JDK-8316283) as no `-Xcomp` is involved in the current failures AFAICS. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16961#discussion_r1414867564 From alanb at openjdk.org Tue Dec 5 07:12:38 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 5 Dec 2023 07:12:38 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS [v3] In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 23:47:12 GMT, David Holmes wrote: >> Please review this simple clarification to the JVM TI spec regarding use of `JAVA_TOOL_OPTIONS` in regards to module options and their format. >> >> I do not believe this clarification needs a CSR request. >> >> Thanks. > > David Holmes has updated the pull request incrementally with one additional commit since the last revision: > > reviewer feedback Thanks, this is very clear now. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16896#pullrequestreview-1764102034 From Alan.Bateman at oracle.com Tue Dec 5 08:16:39 2023 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 5 Dec 2023 08:16:39 +0000 Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: <68639112-c37b-4a3a-9e8e-6502d691aeb2@oracle.com> References: <8VMA3XjqiXFbsQTa5IjAouBch2QjdhaRnERtcTTlxpg=.148cbc2c-8e73-4a6e-b980-682fb45b8ae9@github.com> <72662e8e-342a-4812-9f5d-4002d7810dd9@oracle.com> <5efb2178-fec7-4cd1-a752-45514d5538b5@oracle.com> <68639112-c37b-4a3a-9e8e-6502d691aeb2@oracle.com> Message-ID: <93c3ebce-9608-42bc-b2dc-18af84876a2e@oracle.com> On 04/12/2023 18:59, Chris Plummer wrote: > > So does this mean if the application is no longer referencing the > ExecutorService, then we can have unreachable virtual threads that > have not completed? This is really the point I've been getting at. > Yes, this is possible. It would be an unusual scenario of course. -Alan From sspitsyn at openjdk.org Tue Dec 5 08:32:33 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 08:32:33 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected In-Reply-To: References: Message-ID: <3ON__gPCupcsrNpP0wxI9gi6ur3d0TpcvZ47cFjLM2k=.f9152d39-9a2f-4745-a869-4cd74973eb2f@github.com> On Tue, 5 Dec 2023 04:41:29 GMT, David Holmes wrote: >> The fix is for a regression caused by: >> [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 >> >> The fix of 8308614 just triggered a known issue: >> [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option >> >> The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. >> >> Testing: >> - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix >> - In progress: Test with tiers 1-6 > > src/hotspot/share/prims/jvmtiThreadState.cpp line 562: > >> 560: if (JvmtiThreadState::seen_interp_only_mode() || >> 561: JvmtiExport::should_post_field_access() || >> 562: JvmtiExport::should_post_field_modification()){ > > The comment needs updating to explain the extra checks. > > Can't say I see the connection with [8316283](https://bugs.openjdk.org/browse/JDK-8316283) as no `-Xcomp` is involved in the current failures AFAICS. Thank you for the suggestion. I'll add a comment. The `-Xcomp` flag is not a root cause but a trigger. There is a general issue that a frame can be not deoptimized. The `-Xcomp` is a stress that helps to reproduce the issue as the `-Xcomp` option and `interp_only_mode` works in opposite directions. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16961#discussion_r1415125933 From alanb at openjdk.org Tue Dec 5 08:45:34 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 5 Dec 2023 08:45:34 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 21:33:52 GMT, Serguei Spitsyn wrote: > Just want to make sure this change is intentional. Before the comment was: `// runnable-mounted`. Well spotted, the wrong comment was moved. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16953#discussion_r1415146065 From duke at openjdk.org Tue Dec 5 08:55:34 2023 From: duke at openjdk.org (ExE Boss) Date: Tue, 5 Dec 2023 08:55:34 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 16:08:32 GMT, Alan Bateman wrote: > When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. > > The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. > > For testing the existing ThreadAPI has new test cases. > > Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. src/hotspot/share/classfile/javaClasses.cpp line 1998: > 1996: case UNPARKED: > 1997: case YIELDING : > 1998: case YIELDED: Suggestion: case UNPARKED : case YIELDING : case YIELDED : ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16953#discussion_r1414448245 From alanb at openjdk.org Tue Dec 5 09:08:35 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 5 Dec 2023 09:08:35 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit In-Reply-To: References: Message-ID: <132Ng8DJvDqF0qU-n7S166K-uCMTm88bu_8W9JPNLTU=.4c560882-efa1-46e5-822e-09d7a6d849be@github.com> On Mon, 4 Dec 2023 20:25:56 GMT, ExE Boss wrote: >> When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. >> >> The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. >> >> For testing the existing ThreadAPI has new test cases. >> >> Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. > > src/hotspot/share/classfile/javaClasses.cpp line 1998: > >> 1996: case UNPARKED: >> 1997: case YIELDING : >> 1998: case YIELDED: > > Suggestion: > > case UNPARKED : > case YIELDING : > case YIELDED : I need to check the Hotspot style guide but I think the prevailing convention is no space between the label and the colon. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16953#discussion_r1415177431 From alanb at openjdk.org Tue Dec 5 09:51:50 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 5 Dec 2023 09:51:50 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit [v2] In-Reply-To: References: Message-ID: > When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. > > The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. > > For testing the existing ThreadAPI has new test cases. > > Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Fix comment, remove space between case labels and colon - Merge - Leave onPinned to another PR - Initial commit ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16953/files - new: https://git.openjdk.org/jdk/pull/16953/files/45c6072a..057c9a57 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16953&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16953&range=00-01 Stats: 2035 lines in 49 files changed: 1448 ins; 293 del; 294 mod Patch: https://git.openjdk.org/jdk/pull/16953.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16953/head:pull/16953 PR: https://git.openjdk.org/jdk/pull/16953 From jkern at openjdk.org Tue Dec 5 12:11:46 2023 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 5 Dec 2023 12:11:46 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v3] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: encapsulate everything in os::Aix::dlopen ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/0f6716db..2d32c43b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=01-02 Stats: 175 lines in 2 files changed: 90 ins; 82 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From alanb at openjdk.org Tue Dec 5 12:14:47 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 5 Dec 2023 12:14:47 GMT Subject: Integrated: 8320652: ThreadInfo.isInNative needs to be updated to say what executing native code means In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 10:25:31 GMT, Alan Bateman wrote: > This is a docs only change to j.l.management.ThreadInfo::isInNative. > > The method currently specifies that it tests if the thread is "executing native code via the Java Native Interface (JNI)". It would be clearer to say that it tests if the thread is executing a native method, and expand it to include native code invoked using the new foreign linker APIs. For now, I've left out the detail of a downcall handle created with the "critical" linker option. This pull request has now been integrated. Changeset: 4fbf22b0 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/4fbf22b002dab3c6e7e20ed9c7fa4551b6350082 Stats: 16 lines in 1 file changed: 11 ins; 0 del; 5 mod 8320652: ThreadInfo.isInNative needs to be updated to say what executing native code means Reviewed-by: mchung ------------- PR: https://git.openjdk.org/jdk/pull/16791 From stuefe at openjdk.org Tue Dec 5 13:55:51 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 5 Dec 2023 13:55:51 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v3] In-Reply-To: References: Message-ID: <7V0zHrWeOjnDyHJuq3DFsb-BvaQvZbwE5zIGyxWvGNE=.48a0fc72-c70d-4e21-891a-5f4714bac830@github.com> On Tue, 5 Dec 2023 12:11:46 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > encapsulate everything in os::Aix::dlopen Excellent, this is how I have pictured a good solution. Very nice. A number of remarks, but nothing fundamental. src/hotspot/os/aix/os_aix.cpp line 1137: > 1135: if (ebuf != nullptr && ebuflen > 0) { > 1136: ::strncpy(ebuf, "dll_load: empty filename specified", ebuflen - 1); > 1137: } Are there any cases where we don't hand in the error buffer? If so, I would just assert ebuf and ebuflen. No need for this kind of flexibility. src/hotspot/os/aix/os_aix.cpp line 3051: > 3049: > 3050: // Simulate the library search algorithm of dlopen() (in os::dll_load) > 3051: int os::Aix::stat64x_via_LIBPATH(const char* path, struct stat64x* stat) { - no need to export this, make it filescope static - please return bool, with false = error - please rename it to something like "search_file_in_LIBPATH" src/hotspot/os/aix/os_aix.cpp line 3055: > 3053: return -1; > 3054: > 3055: char *path2 = strdup (path); Please use os::strdup and os::free. If you really intent to use the plain libc versions, use `::strdup` and `::free` to make sure - and indicate to code readers - you use the global libc variants. src/hotspot/os/aix/os_aix.cpp line 3059: > 3057: int idx = strlen(path2) - 1; > 3058: if (path2[idx] == ')') { > 3059: while (path2[idx] != '(' && idx > 0) idx--; ? Why not `strrchr()`? src/hotspot/os/aix/os_aix.cpp line 3067: > 3065: if (path2[0] == '/' || > 3066: (path2[0] == '.' && (path2[1] == '/' || > 3067: (path2[1] == '.' && path2[2] == '/')))) { This complexity is not needed, nor is it sufficient, since it does not handle relative paths ("mydirectory/hallo.so") https://www.ibm.com/docs/en/aix/7.1?topic=d-dlopen-subroutine "If FilePath contains a slash character, FilePath is used directly, and no directories are searched. " So, just scan for a '/' - if you find one, its a path to be opened directly: const bool use_as_filepath = strchr(path2, '/'); src/hotspot/os/aix/os_aix.cpp line 3085: > 3083: strcpy(libpath, env); > 3084: for (token = strtok_r(libpath, ":", &saveptr); token != nullptr; token = strtok_r(nullptr, ":", &saveptr)) { > 3085: sprintf(combined, "%s/%s", token, path2); You can save a lot of pain and manual labor by using `stringStream` here. stringStream combined; combined.print("%s/%s", token, path2); const char* combined_path_string = combined.base(); no need for manual allocation and byte counting. src/hotspot/os/aix/os_aix.cpp line 3099: > 3097: // filled by os::dll_load(). This way we mimic dl handle equality for a library > 3098: // opened a second time, as it is implemented on other platforms. > 3099: void* os::Aix::dlopen(const char* filename, int Flags) { Does not need to be exported, nor does os::AIX::dlclose. Make file scope static. See my remarks in os_posix.cpp. src/hotspot/os/aix/os_aix.cpp line 3103: > 3101: struct stat64x libstat; > 3102: > 3103: if (os::Aix::stat64x_via_LIBPATH(filename, &libstat)) { Please return bool, not unix int -1, this hurts my brain :-) src/hotspot/os/aix/os_aix.cpp line 3108: > 3106: if (result != nullptr) { > 3107: assert(false, "dll_load: Could not stat() file %s, but dlopen() worked; Have to improve stat()", filename); > 3108: } Since this is just assert code, I'd wrap all this stuff in #ifdef ASSERT. No need for needless dlopens otherwise. src/hotspot/os/aix/os_aix.cpp line 3125: > 3123: } > 3124: if (i == g_handletable_used) { > 3125: // library not still loaded. Check if there is space left in array s/still/yet src/hotspot/os/aix/os_aix.cpp line 3131: > 3129: pthread_mutex_unlock(&g_handletable_mutex); > 3130: assert(false, "max_handletable reached"); > 3131: return nullptr; Please, for the sake of release code, hand in an error buffer and fill it with something that makes sense, eg. "too many libraries loaded". The assert is still okay, I guess, since we don't expect it to fire during tests; if it does fire, it may indicate a problem in our handle table logic or a wrong assumption about handle ?quality. src/hotspot/os/aix/os_aix.cpp line 3133: > 3131: return nullptr; > 3132: } > 3133: // library not still loaded and still place in array, so load library s/still/yet src/hotspot/os/aix/os_aix.cpp line 3143: > 3141: g_handletable[i].devid = libstat.st_dev; > 3142: g_handletable[i].refcount = 1; > 3143: } Error handling: on error, call dlerror and return error string inside the error buffer you should hand in. All other platforms do this too. src/hotspot/os/aix/os_aix.cpp line 3150: > 3148: } > 3149: > 3150: int os::Aix::dlclose(void* lib) { can we call lib something better, maybe "handle"? src/hotspot/os/aix/os_aix.cpp line 3165: > 3163: // refcount == 0, so we have to ::dlclose() the lib > 3164: // and delete the entry from the array. > 3165: res = ::dlclose(lib); Handle dlclose error. We expect it to work; if it doesn't, it indicates that something is wrong with the handle logic, e.g. an invalid or already closed handle had been handed in. So, assert. src/hotspot/os/aix/os_aix.hpp line 185: > 183: // opened a second time, as it is implemented on other platforms. > 184: static void* dlopen(const char* filename, int Flags); > 185: static int dlclose(void* lib); Remove; should not be exported. src/hotspot/os/posix/os_posix.cpp line 735: > 733: l_path = ""; > 734: } > 735: int res = AIX_ONLY(os::Aix)::dlclose(lib); Lets do this cleaner, and in the style of hotspot coding elsewhere: - introduce a new function "os::pd_dll_unload(handle, errorbuf, errbuflen)". Add it to os.hpp, but somewhere non-public. The implementations will live in os_aix.cpp, os_bsd.cpp and os_linux.cpp. - make os::Aix::dlclose -> os::pd_dll_unload; the only difference is that you should fill in error buffer with either ::dlerror or, if you have errors in handle table, a text describing that error - on all other posix platforms (os_bsd.cpp + os_linux.cpp), implement a minimal version of os::pd_dll_unload() that calls ::dlunload, and on error calls ::dlerror and copies the string into errbuf - Here, call os::pd_dll_unload instead of ::dlclose/os::aix::dlclose - change the JFR code below to not use ::dlerror but the string returned from the buffer ------------- Changes requested by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16920#pullrequestreview-1762354122 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415568694 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415577023 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415588986 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415577568 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415585089 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415594396 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415625152 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415595301 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415596399 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415597594 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415599081 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415601350 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415607920 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415612828 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415619700 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415625511 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415638462 From stuefe at openjdk.org Tue Dec 5 13:55:54 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 5 Dec 2023 13:55:54 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v2] In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 12:33:26 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > improve handling of nonexisting files src/hotspot/os/aix/os_aix.cpp line 203: > 201: constexpr int max_handletable = 1024; > 202: static int g_handletable_used = 0; > 203: static struct handletableentry g_handletable[max_handletable] = {{0,0,0,0}}; style nits: - we usually write the * behind type, not before var name - `{{0,0}}` -> insert spaces src/hotspot/os/aix/os_aix.cpp line 1159: > 1157: result = ::dlopen(filename, dflags); > 1158: if (result != nullptr) { > 1159: assert(false, "dll_load: Could not stat() file %s, but dlopen() worked; Have to improve stat()", filename); use assert(result != nullptr) and remove condition ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1413843503 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1413846111 From stuefe at openjdk.org Tue Dec 5 13:55:56 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 5 Dec 2023 13:55:56 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v3] In-Reply-To: <7V0zHrWeOjnDyHJuq3DFsb-BvaQvZbwE5zIGyxWvGNE=.48a0fc72-c70d-4e21-891a-5f4714bac830@github.com> References: <7V0zHrWeOjnDyHJuq3DFsb-BvaQvZbwE5zIGyxWvGNE=.48a0fc72-c70d-4e21-891a-5f4714bac830@github.com> Message-ID: <4QEE7t_IctCIU9HwHs8qYxvjAxykfQDx00mYzavj9y0=.be9dc6ea-1dfc-4c7e-aa88-2cdc21a0e486@github.com> On Tue, 5 Dec 2023 13:21:35 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> encapsulate everything in os::Aix::dlopen > > src/hotspot/os/aix/os_aix.cpp line 3133: > >> 3131: return nullptr; >> 3132: } >> 3133: // library not still loaded and still place in array, so load library > > s/still/yet No need to be this verbose either, especially since the comment is somewhat misleading. "create entry at end of table" implies that we have a dynamically growing table and allocate new entries. Proposal: "Library not yet loaded; load it, then store its handle in handle table". ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1415605856 From eastigeevich at openjdk.org Tue Dec 5 16:17:41 2023 From: eastigeevich at openjdk.org (Evgeny Astigeevich) Date: Tue, 5 Dec 2023 16:17:41 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v5] In-Reply-To: References: <81dXSHvLQMGj3s1BcBs8fmJUEoJpaU-5wBRSIjnztMM=.d53f8a2f-8353-49ec-8a9b-695b32f03d20@github.com> Message-ID: On Fri, 1 Dec 2023 21:25:19 GMT, Chris Plummer wrote: >> Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: >> >> Apply man changes > > src/hotspot/share/code/codeCache.cpp line 1809: > >> 1807: } >> 1808: >> 1809: void CodeCache::write_perf_map(const char* filename) { > > Why not have a `filename == nullptr` indicate that the default should be used. Then you don't need CodeCache::DefaultPerfMapFile. You can just have a private `CodeCache::defaultPerfmapFileName()` method. Hi Chris, The current design of `write_perf_map` provides a clean and explicit interface. The purpose of the function is evident from its signature: to write a perf map into a specified file. This explicitness makes the code more readable and self-documenting. It reduces the need for developers to go to the implementation to figure out: what is the meaning of `nullptr`; where a filename will be taken from. It also serves as a contract between the caller and the function itself. By explicitly requiring a filename, the function sets clear expectations for the caller. I think `CodeCache::write_default_perf_map` hiding the filename of the default perf map might not be a good idea because it makes impossible to get the filename used in it. I prefer either method `CodeCache::defaultPerfmapFileName()` or class `CodeCache::DefaultPerfmapFileName`. The class is simpler to implement than the method (like it was earlier). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15871#discussion_r1415892122 From ddong at openjdk.org Tue Dec 5 16:36:50 2023 From: ddong at openjdk.org (Denghui Dong) Date: Tue, 5 Dec 2023 16:36:50 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC Message-ID: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> Hi, Could I have a review of this patch? In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. Best, Denghui ------------- Commit messages: - 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC Changes: https://git.openjdk.org/jdk/pull/16976/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16976&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321404 Stats: 53 lines in 3 files changed: 51 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16976.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16976/head:pull/16976 PR: https://git.openjdk.org/jdk/pull/16976 From jjoo at openjdk.org Tue Dec 5 19:47:02 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 5 Dec 2023 19:47:02 GMT Subject: Integrated: 8315149: Add hsperf counters for CPU time of internal GC threads In-Reply-To: References: Message-ID: On Mon, 31 Jul 2023 01:50:07 GMT, Jonathan Joo wrote: > 8315149: Add hsperf counters for CPU time of internal GC threads This pull request has now been integrated. Changeset: 9e570105 Author: Jonathan Joo Committer: Man Cao URL: https://git.openjdk.org/jdk/commit/9e570105c30a6e462d08931e2010cef9cd5a6031 Stats: 449 lines in 19 files changed: 444 ins; 4 del; 1 mod 8315149: Add hsperf counters for CPU time of internal GC threads Co-authored-by: Man Cao Co-authored-by: Stefan Johansson Reviewed-by: simonis, manc, sjohanss ------------- PR: https://git.openjdk.org/jdk/pull/15082 From cjplummer at openjdk.org Tue Dec 5 20:15:42 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 5 Dec 2023 20:15:42 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v5] In-Reply-To: References: <81dXSHvLQMGj3s1BcBs8fmJUEoJpaU-5wBRSIjnztMM=.d53f8a2f-8353-49ec-8a9b-695b32f03d20@github.com> Message-ID: On Tue, 5 Dec 2023 16:14:31 GMT, Evgeny Astigeevich wrote: >> src/hotspot/share/code/codeCache.cpp line 1809: >> >>> 1807: } >>> 1808: >>> 1809: void CodeCache::write_perf_map(const char* filename) { >> >> Why not have a `filename == nullptr` indicate that the default should be used. Then you don't need CodeCache::DefaultPerfMapFile. You can just have a private `CodeCache::defaultPerfmapFileName()` method. > > Hi Chris, > The current design of `write_perf_map` provides a clean and explicit interface. The purpose of the function is evident from its signature: to write a perf map into a specified file. This explicitness makes the code more readable and self-documenting. It reduces the need for developers to go to the implementation to figure out: what is the meaning of `nullptr`; where a filename will be taken from. It also serves as a contract between the caller and the function itself. By explicitly requiring a filename, the function sets clear expectations for the caller. > > I think `CodeCache::write_default_perf_map` hiding the filename of the default perf map might not be a good idea because it makes impossible to get the filename used in it. I prefer either method `CodeCache::defaultPerfmapFileName()` or class `CodeCache::DefaultPerfmapFileName`. The class is simpler to implement than the method (like it was earlier). The default filename was already "hidden" before these changes, so at the very least things are not being made any worse, but I don't see why any users `write_perf_map` would ever need the default filename. I just felt that adding and exporting a class whose only purpose is to provide the default name seemed like unnecessary overkill. I'm not so sure having a public CodeCache::defaultPerfmapFileName() API and two `write_perf_map` APIs isn't overkill also. There is nothing wrong with a null filename argument signally to use some default name. You can also have the filename arg default to `nullptr`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15871#discussion_r1416228456 From matsaave at openjdk.org Tue Dec 5 20:30:34 2023 From: matsaave at openjdk.org (Matias Saavedra Silva) Date: Tue, 5 Dec 2023 20:30:34 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v2] In-Reply-To: References: Message-ID: On Sat, 2 Dec 2023 00:38:58 GMT, Ioi Lam wrote: >> This is a simple clean up that moves the code for initializing the CDS config states from arguments.cpp to cdsConfig.cpp >> >> I renamed a few functions, but otherwise the code is unchanged. >> >> - `get_default_shared_archive_path()` -> `default_archive_path()` >> - `GetSharedArchivePath()` -> `static_archive_path()` >> - `GetSharedDynamicArchivePath()` -> `dynamic_archive_path()` >> >> There's also less `#if INCLUDE_CDS` since the entire cdsConfig.cpp file is compiled only if CDS is enabled. > > Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: > > fixed indentation Thanks for addressing my comments. Approved! ------------- Marked as reviewed by matsaave (Committer). PR Review: https://git.openjdk.org/jdk/pull/16868#pullrequestreview-1766049580 From sspitsyn at openjdk.org Tue Dec 5 20:33:40 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 20:33:40 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit [v2] In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 09:51:50 GMT, Alan Bateman wrote: >> When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. >> >> The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. >> >> For testing the existing ThreadAPI has new test cases. >> >> Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: > > - Fix comment, remove space between case labels and colon > - Merge > - Leave onPinned to another PR > - Initial commit The fix looks good to me. Also, it can be more safe to run mach5 tier6 as well. ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16953#pullrequestreview-1766053224 From dholmes at openjdk.org Tue Dec 5 21:51:45 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 5 Dec 2023 21:51:45 GMT Subject: RFR: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS [v3] In-Reply-To: References: Message-ID: <62te1Db7lHLJXxN6NJEVqN0zpBvX-CDwD0ffU0jp4BE=.1990e916-0f63-4864-9975-ad02af6b68ba@github.com> On Mon, 4 Dec 2023 23:47:12 GMT, David Holmes wrote: >> Please review this simple clarification to the JVM TI spec regarding use of `JAVA_TOOL_OPTIONS` in regards to module options and their format. >> >> I do not believe this clarification needs a CSR request. >> >> Thanks. > > David Holmes has updated the pull request incrementally with one additional commit since the last revision: > > reviewer feedback Thanks Alan. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16896#issuecomment-1841667405 From dholmes at openjdk.org Tue Dec 5 21:51:47 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 5 Dec 2023 21:51:47 GMT Subject: Integrated: 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 04:30:28 GMT, David Holmes wrote: > Please review this simple clarification to the JVM TI spec regarding use of `JAVA_TOOL_OPTIONS` in regards to module options and their format. > > I do not believe this clarification needs a CSR request. > > Thanks. This pull request has now been integrated. Changeset: c8fa7581 Author: David Holmes URL: https://git.openjdk.org/jdk/commit/c8fa7581006183d0dabe902c40ab8d7304dfd002 Stats: 6 lines in 1 file changed: 4 ins; 0 del; 2 mod 8320860: add-opens/add-exports require '=' in JAVA_TOOL_OPTIONS Reviewed-by: sspitsyn, alanb ------------- PR: https://git.openjdk.org/jdk/pull/16896 From sspitsyn at openjdk.org Tue Dec 5 22:51:47 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 22:51:47 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v2] In-Reply-To: References: Message-ID: > This is a trivial fix for a regression caused by: > [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 > > The fix of 8308614 just triggered a known issue: > [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option > > The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. > > Testing: > - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix > - In progress: Test with tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: extended comment to cover the watchpoint extra checks ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16961/files - new: https://git.openjdk.org/jdk/pull/16961/files/c08484ac..7d47fd12 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16961&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16961&range=00-01 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16961.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16961/head:pull/16961 PR: https://git.openjdk.org/jdk/pull/16961 From dcubed at openjdk.org Tue Dec 5 23:03:33 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Tue, 5 Dec 2023 23:03:33 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v2] In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 22:51:47 GMT, Serguei Spitsyn wrote: >> This is a trivial fix for a regression caused by: >> [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 >> >> The fix of 8308614 just triggered a known issue: >> [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option >> >> The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. >> >> Testing: >> - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix >> - In progress: Test with tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: extended comment to cover the watchpoint extra checks Thumbs up. This is a trivial fix. You'll need to fix the whitespace complaint before integration. ------------- Marked as reviewed by dcubed (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16961#pullrequestreview-1766264031 From iklam at openjdk.org Tue Dec 5 23:13:11 2023 From: iklam at openjdk.org (Ioi Lam) Date: Tue, 5 Dec 2023 23:13:11 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v3] In-Reply-To: References: Message-ID: > This is a simple clean up that moves the code for initializing the CDS config states from arguments.cpp to cdsConfig.cpp > > I renamed a few functions, but otherwise the code is unchanged. > > - `get_default_shared_archive_path()` -> `default_archive_path()` > - `GetSharedArchivePath()` -> `static_archive_path()` > - `GetSharedDynamicArchivePath()` -> `dynamic_archive_path()` > > There's also less `#if INCLUDE_CDS` since the entire cdsConfig.cpp file is compiled only if CDS is enabled. Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: - Merge branch 'master' into 8320935-move-cds-config-code-from-arguments-cpp - fixed indentation - code alignment - step4 - step3 - step2 - step1 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16868/files - new: https://git.openjdk.org/jdk/pull/16868/files/01dd47bc..a080edeb Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16868&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16868&range=01-02 Stats: 84382 lines in 1756 files changed: 39063 ins; 38780 del; 6539 mod Patch: https://git.openjdk.org/jdk/pull/16868.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16868/head:pull/16868 PR: https://git.openjdk.org/jdk/pull/16868 From sspitsyn at openjdk.org Tue Dec 5 23:31:33 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 23:31:33 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v2] In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 22:51:47 GMT, Serguei Spitsyn wrote: >> This is a trivial fix for a regression caused by: >> [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 >> >> The fix of 8308614 just triggered a known issue: >> [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option >> >> The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. >> >> Testing: >> - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix >> - In progress: Test with tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: extended comment to cover the watchpoint extra checks Dan, thank you a lot for quick review! I'll fix the whitespace issue. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16961#issuecomment-1841792398 From sspitsyn at openjdk.org Tue Dec 5 23:36:46 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 23:36:46 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v3] In-Reply-To: References: Message-ID: <4ac4Z72qGn_-S7p43eNHZRgx4mmXxGNgXR1f7W06aQE=.8b328e1f-6378-4637-a76c-17c581f31a24@github.com> > This is a trivial fix for a regression caused by: > [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 > > The fix of 8308614 just triggered a known issue: > [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option > > The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. > > Testing: > - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix > - In progress: Test with tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: fixed trailing whitespace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16961/files - new: https://git.openjdk.org/jdk/pull/16961/files/7d47fd12..34da9c6e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16961&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16961&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16961.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16961/head:pull/16961 PR: https://git.openjdk.org/jdk/pull/16961 From sspitsyn at openjdk.org Tue Dec 5 23:44:41 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 5 Dec 2023 23:44:41 GMT Subject: Integrated: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected In-Reply-To: References: Message-ID: <2dccnNqjKvOXXz9hLRAwDsrAt9wr-K6XwjH0yTNC4V0=.4343c65c-3829-45da-bf7e-c37f71451ead@github.com> On Tue, 5 Dec 2023 00:23:45 GMT, Serguei Spitsyn wrote: > This is a trivial fix for a regression caused by: > [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 > > The fix of 8308614 just triggered a known issue: > [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option > > The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. > > Testing: > - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix > - In progress: Test with tiers 1-6 This pull request has now been integrated. Changeset: 905137d4 Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/905137d4065eb40bef6946bdc6bb688d6018a89d Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected Reviewed-by: dcubed ------------- PR: https://git.openjdk.org/jdk/pull/16961 From dholmes at openjdk.org Wed Dec 6 02:28:45 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 6 Dec 2023 02:28:45 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v2] In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 23:01:20 GMT, Daniel D. Daugherty wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: extended comment to cover the watchpoint extra checks > > Thumbs up. This is a trivial fix. > > You'll need to fix the whitespace complaint before integration. @dcubed-ojdk I would not consider this a trivial fix at all - the need to add the additional conditions is not at all obvious! And even if they were, that would make this a small/simple fix, not "trivial" as defined for the "one review needed" rule. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16961#issuecomment-1841982060 From dholmes at openjdk.org Wed Dec 6 02:35:49 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 6 Dec 2023 02:35:49 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v3] In-Reply-To: <4ac4Z72qGn_-S7p43eNHZRgx4mmXxGNgXR1f7W06aQE=.8b328e1f-6378-4637-a76c-17c581f31a24@github.com> References: <4ac4Z72qGn_-S7p43eNHZRgx4mmXxGNgXR1f7W06aQE=.8b328e1f-6378-4637-a76c-17c581f31a24@github.com> Message-ID: <64Mn3SR0rOVpZMb6CRVJwKfJIySscp5cBy9hyJkMEs4=.faea7c80-3c92-4d5c-9baf-d80e3b2714c0@github.com> On Tue, 5 Dec 2023 23:36:46 GMT, Serguei Spitsyn wrote: >> This is a trivial fix for a regression caused by: >> [8308614](https://bugs.openjdk.org/browse/JDK-8308614) Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 >> >> The fix of 8308614 just triggered a known issue: >> [8316283](https://bugs.openjdk.org/browse/JDK-8316283) field watch events are not always posted with -Xcomp option >> >> The fix is just a work around with the extra checks with the `JvmtiExport::should_post_field_access()` and `JvmtiExport::should_post_field_modification()`. >> >> Testing: >> - The test `runtime/jni/FastGetField/FastGetField.java` does not fail anymore with this fix >> - In progress: Test with tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > fixed trailing whitespace src/hotspot/share/prims/jvmtiThreadState.cpp line 561: > 559: // it is an important optimization to create JvmtiThreadState objects lazily. > 560: // This optimization is disabled when watchpoint capabilities are present. It is to > 561: // work around a bug with virtual thread frames which can be not deoptimized in time. Suggestion: "This optimization is *also* disabled when ..." The phrase "which can be not deoptimized in time." is unclear. Are we racing with deoptimization? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16961#discussion_r1416579588 From dholmes at openjdk.org Wed Dec 6 02:49:35 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 6 Dec 2023 02:49:35 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit [v2] In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 09:51:50 GMT, Alan Bateman wrote: >> When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. >> >> The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. >> >> For testing the existing ThreadAPI has new test cases. >> >> Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: > > - Fix comment, remove space between case labels and colon > - Merge > - Leave onPinned to another PR > - Initial commit I find the switch from `RUNNABLE` to `UNPARKED` somewhat unnecessary. I don't see any problem with `RUNNABLE` that needed fixing. I found the distinction and transition between `RUNNABLE` and `RUNNING` to be very clear. ?? ------------- PR Review: https://git.openjdk.org/jdk/pull/16953#pullrequestreview-1766538038 From dholmes at openjdk.org Wed Dec 6 03:00:38 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 6 Dec 2023 03:00:38 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit [v2] In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 09:51:50 GMT, Alan Bateman wrote: >> When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. >> >> The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. >> >> For testing the existing ThreadAPI has new test cases. >> >> Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: > > - Fix comment, remove space between case labels and colon > - Merge > - Leave onPinned to another PR > - Initial commit Sorry I've deleted my earlier embarassingly stupid comment. The key point of this change is to split RUNNABLE into two states to indicated "runnable after yielding" and "runnable after being unparked". ------------- PR Comment: https://git.openjdk.org/jdk/pull/16953#issuecomment-1842005657 From dholmes at openjdk.org Wed Dec 6 04:03:31 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 6 Dec 2023 04:03:31 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC In-Reply-To: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> Message-ID: On Tue, 5 Dec 2023 16:31:24 GMT, Denghui Dong wrote: > Hi, > > Could I have a review of this patch? > > In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. > > This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. > > Best, > Denghui @D-D-H adding a new manageable flag requires a CSR request to be approved. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16976#issuecomment-1842046395 From dholmes at openjdk.org Wed Dec 6 04:11:33 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 6 Dec 2023 04:11:33 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC In-Reply-To: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> Message-ID: On Tue, 5 Dec 2023 16:31:24 GMT, Denghui Dong wrote: > Hi, > > Could I have a review of this patch? > > In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. > > This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. > > Best, > Denghui Functional changes seem fine, but I think the test is in the wrong place as it is not a dcmd test. Perhaps just place it in test/hotspot/jtreg/serviceability/HeapDump? ------------- Changes requested by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16976#pullrequestreview-1766597251 From iklam at openjdk.org Wed Dec 6 05:25:37 2023 From: iklam at openjdk.org (Ioi Lam) Date: Wed, 6 Dec 2023 05:25:37 GMT Subject: RFR: 8320935: Move CDS config initialization code to cdsConfig.cpp [v3] In-Reply-To: References: Message-ID: On Sat, 2 Dec 2023 03:36:05 GMT, Calvin Cheung wrote: >> Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains seven additional commits since the last revision: >> >> - Merge branch 'master' into 8320935-move-cds-config-code-from-arguments-cpp >> - fixed indentation >> - code alignment >> - step4 >> - step3 >> - step2 >> - step1 > > Marked as reviewed by ccheung (Reviewer). Thanks @calvinccheung @matias9927 @tstuefe for the review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16868#issuecomment-1842107804 From iklam at openjdk.org Wed Dec 6 05:28:41 2023 From: iklam at openjdk.org (Ioi Lam) Date: Wed, 6 Dec 2023 05:28:41 GMT Subject: Integrated: 8320935: Move CDS config initialization code to cdsConfig.cpp In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 23:24:53 GMT, Ioi Lam wrote: > This is a simple clean up that moves the code for initializing the CDS config states from arguments.cpp to cdsConfig.cpp > > I renamed a few functions, but otherwise the code is unchanged. > > - `get_default_shared_archive_path()` -> `default_archive_path()` > - `GetSharedArchivePath()` -> `static_archive_path()` > - `GetSharedDynamicArchivePath()` -> `dynamic_archive_path()` > > There's also less `#if INCLUDE_CDS` since the entire cdsConfig.cpp file is compiled only if CDS is enabled. This pull request has now been integrated. Changeset: 4c96aac9 Author: Ioi Lam URL: https://git.openjdk.org/jdk/commit/4c96aac9c0aa450b0b6859ded8dfff856222ad58 Stats: 696 lines in 8 files changed: 346 ins; 327 del; 23 mod 8320935: Move CDS config initialization code to cdsConfig.cpp Reviewed-by: ccheung, matsaave, stuefe ------------- PR: https://git.openjdk.org/jdk/pull/16868 From ddong at openjdk.org Wed Dec 6 05:50:02 2023 From: ddong at openjdk.org (Denghui Dong) Date: Wed, 6 Dec 2023 05:50:02 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v2] In-Reply-To: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> Message-ID: <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> > Hi, > > Could I have a review of this patch? > > In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. > > This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. > > Best, > Denghui Denghui Dong has updated the pull request incrementally with one additional commit since the last revision: change the location of test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16976/files - new: https://git.openjdk.org/jdk/pull/16976/files/d62a507f..442b7f47 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16976&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16976&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16976.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16976/head:pull/16976 PR: https://git.openjdk.org/jdk/pull/16976 From dholmes at openjdk.org Wed Dec 6 05:57:33 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 6 Dec 2023 05:57:33 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v2] In-Reply-To: <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> Message-ID: On Wed, 6 Dec 2023 05:50:02 GMT, Denghui Dong wrote: >> Hi, >> >> Could I have a review of this patch? >> >> In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. >> >> This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. >> >> Best, >> Denghui > > Denghui Dong has updated the pull request incrementally with one additional commit since the last revision: > > change the location of test Seems fine. Thanks ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16976#pullrequestreview-1766708495 From alanb at openjdk.org Wed Dec 6 07:58:35 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 6 Dec 2023 07:58:35 GMT Subject: RFR: 8321270: Virtual Thread.yield consumes parking permit [v2] In-Reply-To: References: Message-ID: On Wed, 6 Dec 2023 02:58:19 GMT, David Holmes wrote: > The key point of this change is to split RUNNABLE into two states to indicated "runnable after yielding" and "runnable after being unparked". That's right. This comes up with upcoming changes for monitors too - in that case it is important that a virtual thread that continues after blocking on monitorenter doesn't consume its parking permit. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16953#issuecomment-1842310534 From jbachorik at openjdk.org Wed Dec 6 10:38:53 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Wed, 6 Dec 2023 10:38:53 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v11] In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 23:32:52 GMT, David Holmes wrote: > The skara tooling does not currently support our rules but it remains as always that non-trivial Hotspot changes require two reviewers. Thanks, I will keep this in mind. And I apologise for not following the process, though not intentionally. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1842611183 From sspitsyn at openjdk.org Wed Dec 6 19:53:44 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 6 Dec 2023 19:53:44 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v2] In-Reply-To: References: Message-ID: On Tue, 5 Dec 2023 23:01:20 GMT, Daniel D. Daugherty wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: extended comment to cover the watchpoint extra checks > > Thumbs up. This is a trivial fix. > > You'll need to fix the whitespace complaint before integration. > @dcubed-ojdk I would not consider this a trivial fix at all - the need to add the additional conditions is not at all obvious! > And even if they were, that would make this a small/simple fix, not "trivial" as defined for the "one review needed" rule. Sorry, David. It was kind of obvious to me as this tweak is a work around the field watch related regression. Of course, it would better to wait for you to finish review but it was not clear if you are going to complete it or not. As you know, it is very uncomfortable to do last minute push even if it is trivial. :( ------------- PR Comment: https://git.openjdk.org/jdk/pull/16961#issuecomment-1843587781 From sspitsyn at openjdk.org Wed Dec 6 20:18:47 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 6 Dec 2023 20:18:47 GMT Subject: RFR: 8321219: runtime/jni/FastGetField: assert(is_interpreted_frame()) failed: interpreted frame expected [v3] In-Reply-To: <64Mn3SR0rOVpZMb6CRVJwKfJIySscp5cBy9hyJkMEs4=.faea7c80-3c92-4d5c-9baf-d80e3b2714c0@github.com> References: <4ac4Z72qGn_-S7p43eNHZRgx4mmXxGNgXR1f7W06aQE=.8b328e1f-6378-4637-a76c-17c581f31a24@github.com> <64Mn3SR0rOVpZMb6CRVJwKfJIySscp5cBy9hyJkMEs4=.faea7c80-3c92-4d5c-9baf-d80e3b2714c0@github.com> Message-ID: On Wed, 6 Dec 2023 02:33:21 GMT, David Holmes wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> fixed trailing whitespace > > src/hotspot/share/prims/jvmtiThreadState.cpp line 561: > >> 559: // it is an important optimization to create JvmtiThreadState objects lazily. >> 560: // This optimization is disabled when watchpoint capabilities are present. It is to >> 561: // work around a bug with virtual thread frames which can be not deoptimized in time. > > Suggestion: "This optimization is *also* disabled when ..." > > The phrase "which can be not deoptimized in time." is unclear. Are we racing with deoptimization? Than you for the suggestion. In fact, I did not finish with the scalability related optimizations and will continue in 23. Will correct this comment as you suggest when there is any chance. > The phrase "which can be not deoptimized in time." is unclear. Are we racing with deoptimization? Yes, this comment can be not fully correct as you noted. I do not fully understand optimization vs deoptimization mechanisms. I've already spent a lot of time trying to isolate this deoptimization issue but still need to continue this work in 23. Good news is that it can be reliably reproducible but only with a full run any of 4-6 tier. It is not reproducible locally yet. My understanding is that the deoptimization needs some time to happen. We mark frames as needed to deoptimize and they are really deoptimized upon return of the execution control. However, there can be some subtle details depending on the execution path. There can be more then one bug in this area. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16961#discussion_r1417929323 From cjplummer at openjdk.org Thu Dec 7 01:18:58 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 7 Dec 2023 01:18:58 GMT Subject: RFR: 8321485: remove serviceability/attach/ConcAttachTest.java from problemlist on macosx Message-ID: The fix provided by [JDK-8320931](https://bugs.openjdk.org/browse/JDK-8320931) plus the cleanup of all the extra /tmp subdirs has fixed [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866). This test can now be removed from the problem list for OSX. Verification was done by running all of seviceability/dcmd 500 times on macosx-aarch64 and macosx-x64. I also ran serviceability/dcmd 100 times and all of serviceability/ 5 times. I also verified that none of the test tasks took an unusually long time. ------------- Commit messages: - Remove ConcAttachTest.java from the problem list on OSX. Changes: https://git.openjdk.org/jdk/pull/17007/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17007&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321485 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/17007.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17007/head:pull/17007 PR: https://git.openjdk.org/jdk/pull/17007 From jpai at openjdk.org Thu Dec 7 01:39:33 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Thu, 7 Dec 2023 01:39:33 GMT Subject: RFR: 8321485: remove serviceability/attach/ConcAttachTest.java from problemlist on macosx In-Reply-To: References: Message-ID: On Wed, 6 Dec 2023 23:09:35 GMT, Chris Plummer wrote: > The fix provided by [JDK-8320931](https://bugs.openjdk.org/browse/JDK-8320931) plus the cleanup of all the extra /tmp subdirs has fixed [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866). This test can now be removed from the problem list for OSX. Verification was done by running all of seviceability/dcmd 500 times on macosx-aarch64 and macosx-x64. I also ran serviceability/dcmd 100 times and all of serviceability/ 5 times. I also verified that none of the test tasks took an unusually long time. Looks good to me. ------------- Marked as reviewed by jpai (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17007#pullrequestreview-1768947921 From amenkov at openjdk.org Thu Dec 7 02:01:38 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 7 Dec 2023 02:01:38 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: <4Y2GpBm3ColCUahM7GgHzoptvwrp3prbBoEO-gSQVbQ=.c27a4b9f-6e93-47a7-8396-49d8ed7b6016@github.com> On Tue, 5 Dec 2023 01:41:33 GMT, Chris Plummer wrote: > I was actually referring to the footprint of the hprof file, not the in process memory usage while producing it. > > My concern with not doing the option to exclude stack traces now is that it could result in some unusable or unmanageably large heap dumps, or tools simply being overwhelmed by the number of threads. For example, I just looked at the VisualVM threads view, and it just produces a scrollable list of all threads. What happens if there are suddenly 10's of thousands if not millions of threads? If we are lucky is doesn't choke on them and the platform threads are first in the list, but this is the type of thing I'd like to see testing of before pushing this change. Heap dumps are usually big. I think stack traces would not add much (comparing to the size of heapdump itself). Also heap dumper supports compression. It works perfectly fine for identical stack traces. I did some experiments with VisualVM and Eclipse MAT using a heapdump which contains 5K virtual threads. VisualVM has a bug which causes failure populating thread list (I filed a bug for it). I fixed the bug locally and VisualVM was able to generate the list. VisualVM is not ready to work with bug number of threads - it generates the whole list of the threads with stack traces and locals before show it (as table rows or as html) and the generation takes long time. I'd say this is VisualVM's UI issue. I generated heap dump without stack traces - it doesn't help much. Eclipse MAT handles 5K vthreads with no problem (no noticeable lags with and without stack traces). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16665#issuecomment-1844052655 From amenkov at openjdk.org Thu Dec 7 02:15:34 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 7 Dec 2023 02:15:34 GMT Subject: RFR: 8321485: remove serviceability/attach/ConcAttachTest.java from problemlist on macosx In-Reply-To: References: Message-ID: On Wed, 6 Dec 2023 23:09:35 GMT, Chris Plummer wrote: > The fix provided by [JDK-8320931](https://bugs.openjdk.org/browse/JDK-8320931) plus the cleanup of all the extra /tmp subdirs has fixed [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866). This test can now be removed from the problem list for OSX. Verification was done by running all of seviceability/dcmd 500 times on macosx-aarch64 and macosx-x64. I also ran serviceability/dcmd 100 times and all of serviceability/ 5 times. I also verified that none of the test tasks took an unusually long time. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/17007#pullrequestreview-1768989093 From sspitsyn at openjdk.org Thu Dec 7 07:08:47 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 7 Dec 2023 07:08:47 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable Message-ID: This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. The deadlocking scenario is well described by Patricio in a bug report comment. In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. New test was developed by Patricio: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` The test is very nice as it reliably in 100% reproduces the deadlock without the fix. The test is never failing with this fix. Testing: - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` - tested with mach5 tiers 1-6 ------------- Commit messages: - added @summary to new test SuspendWithInterruptLock.java - add new test SuspendWithInterruptLock.java - 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable Changes: https://git.openjdk.org/jdk/pull/17011/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8311218 Stats: 183 lines in 15 files changed: 178 ins; 0 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From alanb at openjdk.org Thu Dec 7 11:44:46 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 7 Dec 2023 11:44:46 GMT Subject: Integrated: 8321270: Virtual Thread.yield consumes parking permit In-Reply-To: References: Message-ID: On Mon, 4 Dec 2023 16:08:32 GMT, Alan Bateman wrote: > When a virtual thread continues after Thread.yield it currently consumes thread's parking permit. This is an oversight, the parking permit should only be consumed when continuing after park. > > The changes are straight-forward. The internal "RUNNABLE" state is replaced with UNPARKED and YIELDED state, effectively encoding the previous state. So for the most part, it's just replacing the usages of RUNNABLE. The additional states require refactoring tryGetStackTrace, this is the method that is used for Thread::getStackTrace when the virtual thread is unmounted. It is also changed to not not swallow the REE if the reesubmit fails (tryStackTrace has to resubmit as the task for the thread may run, or the thread unparked, while "suspended" and sampling its stack trace). The changes are a subset of larger changes in the loom repo, future PRs will propose to bring in other changes to get main line up to date. > > For testing the existing ThreadAPI has new test cases. > > Testing: test1-5. This includes the JVMTI tests as it maps the thread states to JVMTI thread states. This pull request has now been integrated. Changeset: 29d7a223 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/29d7a22348e43cba253d0483c4c05922368f6b8a Stats: 158 lines in 4 files changed: 79 ins; 19 del; 60 mod 8321270: Virtual Thread.yield consumes parking permit Reviewed-by: sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/16953 From dchuyko at openjdk.org Thu Dec 7 16:31:22 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Thu, 7 Dec 2023 16:31:22 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v12] In-Reply-To: References: Message-ID: <6NtI6fuDxn2uXl9aud0ztXfS5VSMd7_g6_rQrLyeWP4=.cabafba9-06c0-4c49-a39a-48e61e46ee4b@github.com> > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 30 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - jcheck - Unnecessary import - force_update->refresh - ... and 20 more: https://git.openjdk.org/jdk/compare/a7f60164...21fe715e ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=11 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From eastigeevich at openjdk.org Thu Dec 7 19:57:21 2023 From: eastigeevich at openjdk.org (Evgeny Astigeevich) Date: Thu, 7 Dec 2023 19:57:21 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v5] In-Reply-To: References: <81dXSHvLQMGj3s1BcBs8fmJUEoJpaU-5wBRSIjnztMM=.d53f8a2f-8353-49ec-8a9b-695b32f03d20@github.com> Message-ID: On Tue, 5 Dec 2023 20:13:11 GMT, Chris Plummer wrote: >> Hi Chris, >> The current design of `write_perf_map` provides a clean and explicit interface. The purpose of the function is evident from its signature: to write a perf map into a specified file. This explicitness makes the code more readable and self-documenting. It reduces the need for developers to go to the implementation to figure out: what is the meaning of `nullptr`; where a filename will be taken from. It also serves as a contract between the caller and the function itself. By explicitly requiring a filename, the function sets clear expectations for the caller. >> >> I think `CodeCache::write_default_perf_map` hiding the filename of the default perf map might not be a good idea because it makes impossible to get the filename used in it. I prefer either method `CodeCache::defaultPerfmapFileName()` or class `CodeCache::DefaultPerfmapFileName`. The class is simpler to implement than the method (like it was earlier). > > The default filename was already "hidden" before these changes, so at the very least things are not being made any worse, but I don't see why any users `write_perf_map` would ever need the default filename. I just felt that adding and exporting a class whose only purpose is to provide the default name seemed like unnecessary overkill. I'm not so sure having a public CodeCache::defaultPerfmapFileName() API and two `write_perf_map` APIs isn't overkill also. There is nothing wrong with a null filename argument signally to use some default name. You can also have the filename arg default to `nullptr`. Ok, let's have: void CodeCache::write_perf_map(const char* filename = nullptr); without any additional classes or funcitons. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15871#discussion_r1419537894 From duke at openjdk.org Thu Dec 7 22:49:55 2023 From: duke at openjdk.org (Yi-Fan Tsai) Date: Thu, 7 Dec 2023 22:49:55 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v7] In-Reply-To: References: Message-ID: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. > > `jcmd PID help Compiler.perfmap` shows the following usage. > > > Compiler.perfmap > Write map file for Linux perf tool. > > Impact: Low > > Syntax : Compiler.perfmap [] > > Arguments: > filename : [optional] Name of the map file (STRING, no default value) > > > The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) > > > Compiler.perfmap [arguments] (Linux only) > Write map file for Linux perf tool. > > Impact: Low > > arguments: > > ? filename: (Optional) Name of the map file (STRING, no default value) > > If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then > the default filename will be /tmp/perf-12345.map. Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: use default argument of write_perf_map ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15871/files - new: https://git.openjdk.org/jdk/pull/15871/files/6a854920..dbe223c5 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=05-06 Stats: 24 lines in 4 files changed: 8 ins; 13 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/15871.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15871/head:pull/15871 PR: https://git.openjdk.org/jdk/pull/15871 From cjplummer at openjdk.org Thu Dec 7 23:09:17 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 7 Dec 2023 23:09:17 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references [v2] In-Reply-To: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> References: <6SwUX9IYEA0iQlY7rgAIIeC4MINN35OO3unTsOR7BKA=.952bbbb3-7f22-4e2e-b7ad-cf71dbdeb727@github.com> Message-ID: On Thu, 30 Nov 2023 00:26:25 GMT, Alex Menkov wrote: >> The change impelements dumping of unmounted virtual threads data (stack traces and stack references). >> Unmounted vthreads can be detected only by iterating over the heap, but hprof stack trace records (HPROF_FRAME/HPROF_TRACE) should be written before HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT. >> HeapDumper supports segment dump (parallel dump to separate files with subsequent file merge outside of safepoint), the fix switches HeapDumper to always use segment dump: 1st segment contains only non-heap data, other segments are used for dumping heap objects. For serial dumping single-threaded dumping is performed, but 2 segments are created anyway. >> When HeapObjectDumper detects unmounted virtual thread, it writes HPROF_FRAME/HPROF_TRACE records to the 1st segment ("global writer"), and writes thread object (HPROF_GC_ROOT_JAVA_FRAME) and stack references (HPROF_GC_ROOT_JAVA_FRAME/HPROF_GC_ROOT_JNI_LOCAL) to the HeapObjectDumper segment. >> As parallel dumpers may write HPROF_FRAME/HPROF_TRACE concurrently and VMDumper needs to write non-heap data before heap object dumpers can write virtual threads data, writing to global writer is protected with DumperController::_global_writer_lock. >> >> Testing: run tests which perform heap dump (in different scenarios): >> - test/hotspot/jtreg/serviceability >> - test/hotspot/jtreg/runtime/ErrorHandling >> - test/hotspot/jtreg/gc/epsilon >> - test/jdk/sun/tools/jhsdb > > Alex Menkov has updated the pull request incrementally with two additional commits since the last revision: > > - feedback > - prepare_parallel_dum Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16665#pullrequestreview-1771272003 From amenkov at openjdk.org Thu Dec 7 23:21:23 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 7 Dec 2023 23:21:23 GMT Subject: Integrated: 8299426: Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 21:54:06 GMT, Alex Menkov wrote: > The change impelements dumping of unmounted virtual threads data (stack traces and stack references). > Unmounted vthreads can be detected only by iterating over the heap, but hprof stack trace records (HPROF_FRAME/HPROF_TRACE) should be written before HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT. > HeapDumper supports segment dump (parallel dump to separate files with subsequent file merge outside of safepoint), the fix switches HeapDumper to always use segment dump: 1st segment contains only non-heap data, other segments are used for dumping heap objects. For serial dumping single-threaded dumping is performed, but 2 segments are created anyway. > When HeapObjectDumper detects unmounted virtual thread, it writes HPROF_FRAME/HPROF_TRACE records to the 1st segment ("global writer"), and writes thread object (HPROF_GC_ROOT_JAVA_FRAME) and stack references (HPROF_GC_ROOT_JAVA_FRAME/HPROF_GC_ROOT_JNI_LOCAL) to the HeapObjectDumper segment. > As parallel dumpers may write HPROF_FRAME/HPROF_TRACE concurrently and VMDumper needs to write non-heap data before heap object dumpers can write virtual threads data, writing to global writer is protected with DumperController::_global_writer_lock. > > Testing: run tests which perform heap dump (in different scenarios): > - test/hotspot/jtreg/serviceability > - test/hotspot/jtreg/runtime/ErrorHandling > - test/hotspot/jtreg/gc/epsilon > - test/jdk/sun/tools/jhsdb This pull request has now been integrated. Changeset: 354ea4c2 Author: Alex Menkov URL: https://git.openjdk.org/jdk/commit/354ea4c28f1449479f71e89831c64047c50e1a61 Stats: 316 lines in 3 files changed: 152 ins; 78 del; 86 mod 8299426: Heap dump does not contain virtual Thread stack references Reviewed-by: cjplummer, sspitsyn, lmesnik ------------- PR: https://git.openjdk.org/jdk/pull/16665 From amenkov at openjdk.org Fri Dec 8 00:47:46 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 8 Dec 2023 00:47:46 GMT Subject: RFR: 8321563: [BACKOUT] Heap dump does not contain virtual Thread stack references Message-ID: Backout of [JDK-8299426](https://bugs.openjdk.org/browse/JDK-8299426) fix which causes tier1 failures Testing in progress ------------- Commit messages: - Revert "8299426: Heap dump does not contain virtual Thread stack references" Changes: https://git.openjdk.org/jdk/pull/17028/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17028&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321563 Stats: 319 lines in 3 files changed: 81 ins; 155 del; 83 mod Patch: https://git.openjdk.org/jdk/pull/17028.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17028/head:pull/17028 PR: https://git.openjdk.org/jdk/pull/17028 From cjplummer at openjdk.org Fri Dec 8 00:53:13 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 8 Dec 2023 00:53:13 GMT Subject: RFR: 8321563: [BACKOUT] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 00:41:35 GMT, Alex Menkov wrote: > Backout of [JDK-8299426](https://bugs.openjdk.org/browse/JDK-8299426) fix which causes tier1 failures > > Testing in progress David changed [JDK-8321560](https://bugs.openjdk.org/browse/JDK-8321560) so it could be used as the backout CR. You might want to retarget to that CR. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17028#issuecomment-1846330790 From cjplummer at openjdk.org Fri Dec 8 01:01:16 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 8 Dec 2023 01:01:16 GMT Subject: RFR: 8321560: [BACKOUT] 8299426: Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: <9E525qi-s5ET-V0CXlah9pNB6fCZyTVp6muF4aYnSsU=.0143e0ca-64ab-4fdc-9856-a762a7fadff2@github.com> On Fri, 8 Dec 2023 00:41:35 GMT, Alex Menkov wrote: > Backout of [JDK-8299426](https://bugs.openjdk.org/browse/JDK-8299426) fix which causes tier1 failures > > Testing in progress Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/17028#pullrequestreview-1771356047 From sspitsyn at openjdk.org Fri Dec 8 01:16:52 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 8 Dec 2023 01:16:52 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v2] In-Reply-To: References: Message-ID: > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - Resolved merge conflict in VirtualThread.java - added @summary to new test SuspendWithInterruptLock.java - add new test SuspendWithInterruptLock.java - 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable ------------- Changes: https://git.openjdk.org/jdk/pull/17011/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=01 Stats: 183 lines in 15 files changed: 178 ins; 0 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From dholmes at openjdk.org Fri Dec 8 01:19:16 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 8 Dec 2023 01:19:16 GMT Subject: RFR: 8321560: [BACKOUT] 8299426: Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 00:41:35 GMT, Alex Menkov wrote: > Backout of [JDK-8299426](https://bugs.openjdk.org/browse/JDK-8299426) fix which causes tier1 failures > > Testing in progress Backout is good. Failing test is now passing in your CI run. Please integrate immediately. Thanks ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17028#pullrequestreview-1771376079 From amenkov at openjdk.org Fri Dec 8 01:27:21 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 8 Dec 2023 01:27:21 GMT Subject: Integrated: 8321560: [BACKOUT] 8299426: Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 00:41:35 GMT, Alex Menkov wrote: > Backout of [JDK-8299426](https://bugs.openjdk.org/browse/JDK-8299426) fix which causes tier1 failures > > Testing in progress This pull request has now been integrated. Changeset: cb7e3d26 Author: Alex Menkov URL: https://git.openjdk.org/jdk/commit/cb7e3d263a6ed65257b316ffcbbd5e19142eb8d1 Stats: 319 lines in 3 files changed: 81 ins; 155 del; 83 mod 8321560: [BACKOUT] 8299426: Heap dump does not contain virtual Thread stack references Reviewed-by: cjplummer, dholmes ------------- PR: https://git.openjdk.org/jdk/pull/17028 From alanb at openjdk.org Fri Dec 8 06:39:59 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 8 Dec 2023 06:39:59 GMT Subject: RFR: 8320532: Remove Thread/ThreadGroup suspend/resume [v2] In-Reply-To: References: Message-ID: > The deadlock prone Thread/ThreadGroup suspend/resume were deprecated since JDK 1.2, deprecated for removal in Java 14, and re-specified/degraded to throw UnsupportedOperationException unconditionally in Java 19/20. Early in Java 23 seems a fine time to finally remove these methods. > > Corpus analysis of 176 million classes in 485k artifacts found no remaining usages of ThreadGroup.suspend/resume beyond the artifacts that include a copy of java.lang.ThreadGroup (!). It found 87 remaining uses of Thread.suspend and 86 remaining usages of Thread.resume, some of these are tests. > > Thread.suspend/resume have always linked to the "Java Thread Primitive Deprecation" page. This originally explained the reasons why suspend/resume were deprecated. When these methods were degraded to throw UOE we changed the text to explain why the ability to suspend or resume a thread was removed. Now we must change it again. One choice is to re-word to explain why the Java APIs were removed or why the Java APIs don't define a way to suspend/resume threads, the other choice (which I prefer) is to remove the text. > > The method description of java.lang.management.ThreadInfo.isSuspended is tweaked to link to JVMTI SuspendThread instead of Thread.suspend Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Update copyright year - Merge - Clarify ThreadInfo.isSuspended - Initial commit ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16789/files - new: https://git.openjdk.org/jdk/pull/16789/files/cc17ea42..517de309 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16789&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16789&range=00-01 Stats: 106040 lines in 2348 files changed: 55620 ins; 41750 del; 8670 mod Patch: https://git.openjdk.org/jdk/pull/16789.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16789/head:pull/16789 PR: https://git.openjdk.org/jdk/pull/16789 From jpai at openjdk.org Fri Dec 8 06:50:21 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 8 Dec 2023 06:50:21 GMT Subject: RFR: 8320532: Remove Thread/ThreadGroup suspend/resume [v2] In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 06:39:59 GMT, Alan Bateman wrote: >> The deadlock prone Thread/ThreadGroup suspend/resume were deprecated since JDK 1.2, deprecated for removal in Java 14, and re-specified/degraded to throw UnsupportedOperationException unconditionally in Java 19/20. Early in Java 23 seems a fine time to finally remove these methods. >> >> Corpus analysis of 176 million classes in 485k artifacts found no remaining usages of ThreadGroup.suspend/resume beyond the artifacts that include a copy of java.lang.ThreadGroup (!). It found 87 remaining uses of Thread.suspend and 86 remaining usages of Thread.resume, some of these are tests. >> >> Thread.suspend/resume have always linked to the "Java Thread Primitive Deprecation" page. This originally explained the reasons why suspend/resume were deprecated. When these methods were degraded to throw UOE we changed the text to explain why the ability to suspend or resume a thread was removed. Now we must change it again. One choice is to re-word to explain why the Java APIs were removed or why the Java APIs don't define a way to suspend/resume threads, the other choice (which I prefer) is to remove the text. >> >> The method description of java.lang.management.ThreadInfo.isSuspended is tweaked to link to JVMTI SuspendThread instead of Thread.suspend > > Alan Bateman has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: > > - Update copyright year > - Merge > - Clarify ThreadInfo.isSuspended > - Initial commit Marked as reviewed by jpai (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16789#pullrequestreview-1771631210 From alanb at openjdk.org Fri Dec 8 07:13:29 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 8 Dec 2023 07:13:29 GMT Subject: Integrated: 8320532: Remove Thread/ThreadGroup suspend/resume In-Reply-To: References: Message-ID: <-ghU_3TzyD_yGIvb5DVotvLawh17x9VcmtJDsOIp9HA=.2bab4e1f-9733-47e5-bf1a-dacd5eaec136@github.com> On Thu, 23 Nov 2023 09:18:44 GMT, Alan Bateman wrote: > The deadlock prone Thread/ThreadGroup suspend/resume were deprecated since JDK 1.2, deprecated for removal in Java 14, and re-specified/degraded to throw UnsupportedOperationException unconditionally in Java 19/20. Early in Java 23 seems a fine time to finally remove these methods. > > Corpus analysis of 176 million classes in 485k artifacts found no remaining usages of ThreadGroup.suspend/resume beyond the artifacts that include a copy of java.lang.ThreadGroup (!). It found 87 remaining uses of Thread.suspend and 86 remaining usages of Thread.resume, some of these are tests. > > Thread.suspend/resume have always linked to the "Java Thread Primitive Deprecation" page. This originally explained the reasons why suspend/resume were deprecated. When these methods were degraded to throw UOE we changed the text to explain why the ability to suspend or resume a thread was removed. Now we must change it again. One choice is to re-word to explain why the Java APIs were removed or why the Java APIs don't define a way to suspend/resume threads, the other choice (which I prefer) is to remove the text. > > The method description of java.lang.management.ThreadInfo.isSuspended is tweaked to link to JVMTI SuspendThread instead of Thread.suspend This pull request has now been integrated. Changeset: af5c4922 Author: Alan Bateman URL: https://git.openjdk.org/jdk/commit/af5c49226c3416a9c3bdde06cac2076acf9de5c3 Stats: 553 lines in 9 files changed: 116 ins; 430 del; 7 mod 8320532: Remove Thread/ThreadGroup suspend/resume Reviewed-by: dholmes, jpai, sspitsyn, smarks ------------- PR: https://git.openjdk.org/jdk/pull/16789 From dchuyko at openjdk.org Fri Dec 8 11:42:44 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Fri, 8 Dec 2023 11:42:44 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v13] In-Reply-To: References: Message-ID: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 31 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - jcheck - Unnecessary import - ... and 21 more: https://git.openjdk.org/jdk/compare/701bc3bb...1a01cf1c ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=12 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From sspitsyn at openjdk.org Fri Dec 8 11:54:40 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 8 Dec 2023 11:54:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: Message-ID: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17011/files - new: https://git.openjdk.org/jdk/pull/17011/files/ccba940d..18f1752e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=01-02 Stats: 80 lines in 9 files changed: 25 ins; 7 del; 48 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From alanb at openjdk.org Fri Dec 8 12:09:15 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 8 Dec 2023 12:09:15 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Fri, 8 Dec 2023 11:54:40 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods I chatted briefly with @sspitsyn about this. A couple of points: - It shouldn't be necessary to touch mount/unmount as the thread identity is the carrier, not the virtual thread, when executing the "critical code". - toggle_is_in_critical_section needs to detect reentrancy, it is otherwise too early to refactor the Java code, e.g. call threadState while holding the interrupt lock. - All the use-sides will need to use try-finally to more reliably revert the critical section flag when rewinding. - The naming is very problematic, we'll need to replace with methods that are clearly named enter and exit critical section. Ongoing work in this area to support monitors has to introduce some temporary pinning so there will be enter/exitCriticalSection methods, that's a better place for the JVMTI hooks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1847063362 From cjplummer at openjdk.org Fri Dec 8 17:05:23 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 8 Dec 2023 17:05:23 GMT Subject: Integrated: 8321485: remove serviceability/attach/ConcAttachTest.java from problemlist on macosx In-Reply-To: References: Message-ID: On Wed, 6 Dec 2023 23:09:35 GMT, Chris Plummer wrote: > The fix provided by [JDK-8320931](https://bugs.openjdk.org/browse/JDK-8320931) plus the cleanup of all the extra /tmp subdirs has fixed [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866). This test can now be removed from the problem list for OSX. Verification was done by running all of seviceability/dcmd 500 times on macosx-aarch64 and macosx-x64. I also ran serviceability/dcmd 100 times and all of serviceability/ 5 times. I also verified that none of the test tasks took an unusually long time. This pull request has now been integrated. Changeset: 2c2d4d2c Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/2c2d4d2cdef209b6f791d20b01b6aaf6d7a99fde Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8321485: remove serviceability/attach/ConcAttachTest.java from problemlist on macosx Reviewed-by: jpai, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/17007 From amenkov at openjdk.org Fri Dec 8 22:38:22 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 8 Dec 2023 22:38:22 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references Message-ID: Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). Run tier1-3 and heapdump-related tests. ------------- Commit messages: - allow heapdump in safepoints - original fix Changes: https://git.openjdk.org/jdk/pull/17040/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17040&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321565 Stats: 319 lines in 3 files changed: 152 ins; 81 del; 86 mod Patch: https://git.openjdk.org/jdk/pull/17040.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17040/head:pull/17040 PR: https://git.openjdk.org/jdk/pull/17040 From sspitsyn at openjdk.org Fri Dec 8 23:40:12 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 8 Dec 2023 23:40:12 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 22:29:12 GMT, Alex Menkov wrote: > Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): > test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. > > Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). > The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). > > Run tier1-3 and heapdump-related tests. This seems to be the same fix that we REDO, is it right? Also, I'd like to double check on what testing was done for this REDO? ------------- PR Review: https://git.openjdk.org/jdk/pull/17040#pullrequestreview-1773300234 From amenkov at openjdk.org Sat Dec 9 00:42:14 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Sat, 9 Dec 2023 00:42:14 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 23:37:30 GMT, Serguei Spitsyn wrote: > This seems to be the same fix that we REDO, is it right? Also, I'd like to double check on what testing was done for this REDO? It's almost the same (the only differences are deleted asserts in HeapMerger methods). The PR contains 2 commits to simplify re-review. The 1st one ("original fix") is exactly the same commit which was integrated for JDK-8299426 and which was reverted; The 2nd one ("allow heapdump in safepoints") contains the fix for the issue discovered after integration. The fix was tested by running heapdump-related tests: test/hotspot/jtreg/serviceability, test/hotspot/jtreg/runtime/ErrorHandling, test/hotspot/jtreg/gc/epsilon, test/jdk/sun/tools/jhsdb and tier1..3 on all Oracle supported platforms ------------- PR Comment: https://git.openjdk.org/jdk/pull/17040#issuecomment-1848012109 From sspitsyn at openjdk.org Sat Dec 9 08:59:14 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 9 Dec 2023 08:59:14 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 22:29:12 GMT, Alex Menkov wrote: > Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): > test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. > > Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). > The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). > > Run tier1-3 and heapdump-related tests. The REDO fix looks good. Good luck with this one! ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17040#pullrequestreview-1773625377 From dholmes at openjdk.org Mon Dec 11 06:08:15 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 11 Dec 2023 06:08:15 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> On Fri, 8 Dec 2023 22:29:12 GMT, Alex Menkov wrote: > Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): > test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. > > Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). > The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). > > Run tier1-3 and heapdump-related tests. src/hotspot/share/services/heapDumper.cpp line 1984: > 1982: // user space. > 1983: void DumpMerger::merge_file(char* path) { > 1984: assert(!SafepointSynchronize::is_at_safepoint(), "merging happens outside safepoint"); This might fix the failure but why is this restriction in place to begin with? This seems like a very expensive operation to be performing by the VMThread during a safepoint! ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1421975329 From stefank at openjdk.org Mon Dec 11 09:21:21 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Mon, 11 Dec 2023 09:21:21 GMT Subject: RFR: 8321713: Harmonize executeTestJvm with create[Limited]TestJavaProcessBuilder Message-ID: [JDK-8315097](https://bugs.openjdk.org/browse/JDK-8315097): 'Rename createJavaProcessBuilder' changed the name of the ProcessTools helper functions used to create `ProcessBuilder`s used to spawn new java test processes. We now have `createTestJavaProcessBuilder` and `createLimitedTestJavaProcess`. The former prepends jvm options from jtreg, while the latter doesn't. With these functions it is common to see the following pattern in tests: ProcessBuilder pb = ProcessTools.createTestJavaProcessBuilder(...); OutputAnalyzer output = executeProcess(pb); We have a couple of thin wrapper in `ProcessTools` that does exactly this, so that the code can be written as a one-liner: OutputAnalyzer output = ProcessTools.executeTestJvm(); I propose that we name this functions using the same naming scheme we used for `createTestJavaProcessBuilder` and `createLimitedTestJavaProcessBuilder`. That is, we change `executeTestJvm` to `executeTestJava` and add a new `executeLimitedTestJava` function. ------------- Commit messages: - 8321713: Harmonize executeTestJvm with create[Limited]TestJavaProcessBuilder Changes: https://git.openjdk.org/jdk/pull/17049/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17049&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321713 Stats: 217 lines in 88 files changed: 28 ins; 1 del; 188 mod Patch: https://git.openjdk.org/jdk/pull/17049.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17049/head:pull/17049 PR: https://git.openjdk.org/jdk/pull/17049 From kevinw at openjdk.org Mon Dec 11 12:03:22 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Mon, 11 Dec 2023 12:03:22 GMT Subject: RFR: 8311306: Test com/sun/management/ThreadMXBean/ThreadCpuTimeArray.java failed: out of expected range Message-ID: Trivial test update to run in "othervm". This test has failed due to interrupted threads: the interruption interferes with timing, but also the test is designed to fail if interrupted. Other tests in the same directory have the same requirement that threads/sleeps are not interrupted, and they run with othervm and are not known to fail. ------------- Commit messages: - 8311306: Test com/sun/management/ThreadMXBean/ThreadCpuTimeArray.java failed: out of expected range Changes: https://git.openjdk.org/jdk/pull/17054/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17054&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8311306 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/17054.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17054/head:pull/17054 PR: https://git.openjdk.org/jdk/pull/17054 From kevinw at openjdk.org Mon Dec 11 13:09:31 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Mon, 11 Dec 2023 13:09:31 GMT Subject: RFR: 8321729: Remove 'orb' field in RMIConnector Message-ID: Tidyup change, looks like this field was not removed when IIOP was removed from RMIConnector. The field is not required for interoperability: with the field removed, I can still connect an older JMX client to a running app with the updated JDK. Since the JDK9 change https://hg.openjdk.org/jdk9/jdk9/jdk/rev/09daaf1e4c53 did not change serialVersionUID, this update does not either. Tests continue to pass, e.g. javax/management (including remote which has RMI tests) jdk/com/sun/management ------------- Commit messages: - 8321729: Remove orb field in RMIConnector Changes: https://git.openjdk.org/jdk/pull/17055/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17055&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321729 Stats: 6 lines in 1 file changed: 0 ins; 6 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/17055.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17055/head:pull/17055 PR: https://git.openjdk.org/jdk/pull/17055 From stefank at openjdk.org Mon Dec 11 14:03:36 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Mon, 11 Dec 2023 14:03:36 GMT Subject: RFR: 8321713: Harmonize executeTestJvm with create[Limited]TestJavaProcessBuilder [v2] In-Reply-To: References: Message-ID: > [JDK-8315097](https://bugs.openjdk.org/browse/JDK-8315097): 'Rename createJavaProcessBuilder' changed the name of the ProcessTools helper functions used to create `ProcessBuilder`s used to spawn new java test processes. > > We now have `createTestJavaProcessBuilder` and `createLimitedTestJavaProcess`. The former prepends jvm options from jtreg, while the latter doesn't. > > With these functions it is common to see the following pattern in tests: > > ProcessBuilder pb = ProcessTools.createTestJavaProcessBuilder(...); > OutputAnalyzer output = executeProcess(pb); > > > We have a couple of thin wrapper in `ProcessTools` that does exactly this, so that the code can be written as a one-liner: > > OutputAnalyzer output = ProcessTools.executeTestJvm(); > > > I propose that we name this functions using the same naming scheme we used for `createTestJavaProcessBuilder` and `createLimitedTestJavaProcessBuilder`. That is, we change `executeTestJvm` to `executeTestJava` and add a new `executeLimitedTestJava` function. Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: Fix impl and add test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17049/files - new: https://git.openjdk.org/jdk/pull/17049/files/080caef5..ad072e06 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17049&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17049&range=00-01 Stats: 54 lines in 2 files changed: 52 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/17049.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17049/head:pull/17049 PR: https://git.openjdk.org/jdk/pull/17049 From stefank at openjdk.org Mon Dec 11 14:06:43 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Mon, 11 Dec 2023 14:06:43 GMT Subject: RFR: 8321713: Harmonize executeTestJvm with create[Limited]TestJavaProcessBuilder [v3] In-Reply-To: References: Message-ID: > [JDK-8315097](https://bugs.openjdk.org/browse/JDK-8315097): 'Rename createJavaProcessBuilder' changed the name of the ProcessTools helper functions used to create `ProcessBuilder`s used to spawn new java test processes. > > We now have `createTestJavaProcessBuilder` and `createLimitedTestJavaProcess`. The former prepends jvm options from jtreg, while the latter doesn't. > > With these functions it is common to see the following pattern in tests: > > ProcessBuilder pb = ProcessTools.createTestJavaProcessBuilder(...); > OutputAnalyzer output = executeProcess(pb); > > > We have a couple of thin wrapper in `ProcessTools` that does exactly this, so that the code can be written as a one-liner: > > OutputAnalyzer output = ProcessTools.executeTestJvm(); > > > I propose that we name this functions using the same naming scheme we used for `createTestJavaProcessBuilder` and `createLimitedTestJavaProcessBuilder`. That is, we change `executeTestJvm` to `executeTestJava` and add a new `executeLimitedTestJava` function. Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: Test cleanup ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17049/files - new: https://git.openjdk.org/jdk/pull/17049/files/ad072e06..5d488f42 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17049&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17049&range=01-02 Stats: 10 lines in 1 file changed: 1 ins; 8 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/17049.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17049/head:pull/17049 PR: https://git.openjdk.org/jdk/pull/17049 From lkorinth at openjdk.org Mon Dec 11 14:23:19 2023 From: lkorinth at openjdk.org (Leo Korinth) Date: Mon, 11 Dec 2023 14:23:19 GMT Subject: RFR: 8321713: Harmonize executeTestJvm with create[Limited]TestJavaProcessBuilder [v3] In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 14:06:43 GMT, Stefan Karlsson wrote: >> [JDK-8315097](https://bugs.openjdk.org/browse/JDK-8315097): 'Rename createJavaProcessBuilder' changed the name of the ProcessTools helper functions used to create `ProcessBuilder`s used to spawn new java test processes. >> >> We now have `createTestJavaProcessBuilder` and `createLimitedTestJavaProcess`. The former prepends jvm options from jtreg, while the latter doesn't. >> >> With these functions it is common to see the following pattern in tests: >> >> ProcessBuilder pb = ProcessTools.createTestJavaProcessBuilder(...); >> OutputAnalyzer output = executeProcess(pb); >> >> >> We have a couple of thin wrapper in `ProcessTools` that does exactly this, so that the code can be written as a one-liner: >> >> OutputAnalyzer output = ProcessTools.executeTestJvm(); >> >> >> I propose that we name this functions using the same naming scheme we used for `createTestJavaProcessBuilder` and `createLimitedTestJavaProcessBuilder`. That is, we change `executeTestJvm` to `executeTestJava` and add a new `executeLimitedTestJava` function. > > Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: > > Test cleanup Looks good to me. ------------- Marked as reviewed by lkorinth (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17049#pullrequestreview-1775250269 From mbaesken at openjdk.org Mon Dec 11 14:41:15 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 11 Dec 2023 14:41:15 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 11:43:08 GMT, Matthias Baesken wrote: > A colleague contacted IBM about the different behavior of getaddrinfo on AIX (compared to Linux/macOS); maybe we have to adjust the result of the getaddrinfo call on AIX. Haven't heard from them so far, hopefully we get an update soon about the behavior of getaddrinfo on AIX . ------------- PR Comment: https://git.openjdk.org/jdk/pull/16561#issuecomment-1850209051 From rriggs at openjdk.org Mon Dec 11 14:54:14 2023 From: rriggs at openjdk.org (Roger Riggs) Date: Mon, 11 Dec 2023 14:54:14 GMT Subject: RFR: 8321729: Remove 'orb' field in RMIConnector In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 12:50:38 GMT, Kevin Walls wrote: > Tidyup change, looks like this field was not removed when IIOP was removed from RMIConnector. > > The field is not required for interoperability: > with the field removed, I can still connect an older JMX client to a running app with the updated JDK. > > Since the JDK9 change https://hg.openjdk.org/jdk9/jdk9/jdk/rev/09daaf1e4c53 > did not change serialVersionUID, this update does not either. > > Tests continue to pass, e.g. > javax/management (including remote which has RMI tests) > jdk/com/sun/management Tidy! ------------- Marked as reviewed by rriggs (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17055#pullrequestreview-1775335414 From dfuchs at openjdk.org Mon Dec 11 17:56:25 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Mon, 11 Dec 2023 17:56:25 GMT Subject: RFR: 8321729: Remove 'orb' field in RMIConnector In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 12:50:38 GMT, Kevin Walls wrote: > Tidyup change, looks like this field was not removed when IIOP was removed from RMIConnector. > > The field is not required for interoperability: > with the field removed, I can still connect an older JMX client to a running app with the updated JDK. > > Since the JDK9 change https://hg.openjdk.org/jdk9/jdk9/jdk/rev/09daaf1e4c53 > did not change serialVersionUID, this update does not either. > > Tests continue to pass, e.g. > javax/management (including remote which has RMI tests) > jdk/com/sun/management Marked as reviewed by dfuchs (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/17055#pullrequestreview-1775807650 From dchuyko at openjdk.org Mon Dec 11 20:22:15 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Mon, 11 Dec 2023 20:22:15 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v14] In-Reply-To: References: Message-ID: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 32 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - jcheck - ... and 22 more: https://git.openjdk.org/jdk/compare/ce4b257f...8b8b30ae ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=13 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From cjplummer at openjdk.org Mon Dec 11 21:43:36 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 11 Dec 2023 21:43:36 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v7] In-Reply-To: References: Message-ID: On Thu, 7 Dec 2023 22:49:55 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) >> >> >> Compiler.perfmap [arguments] (Linux only) >> Write map file for Linux perf tool. >> >> Impact: Low >> >> arguments: >> >> ? filename: (Optional) Name of the map file (STRING, no default value) >> >> If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then >> the default filename will be /tmp/perf-12345.map. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > use default argument of write_perf_map The test needs a copyright update. Otherwise the changes look good. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/15871#pullrequestreview-1776216918 From duke at openjdk.org Mon Dec 11 22:41:56 2023 From: duke at openjdk.org (Yi-Fan Tsai) Date: Mon, 11 Dec 2023 22:41:56 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. > > `jcmd PID help Compiler.perfmap` shows the following usage. > > > Compiler.perfmap > Write map file for Linux perf tool. > > Impact: Low > > Syntax : Compiler.perfmap [] > > Arguments: > filename : [optional] Name of the map file (STRING, no default value) > > > The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) > > > Compiler.perfmap [arguments] (Linux only) > Write map file for Linux perf tool. > > Impact: Low > > arguments: > > ? filename: (Optional) Name of the map file (STRING, no default value) > > If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then > the default filename will be /tmp/perf-12345.map. Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: Update copyright of PerfMapTest ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15871/files - new: https://git.openjdk.org/jdk/pull/15871/files/dbe223c5..e1b0b162 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=06-07 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/15871.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15871/head:pull/15871 PR: https://git.openjdk.org/jdk/pull/15871 From amenkov at openjdk.org Tue Dec 12 00:22:32 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 12 Dec 2023 00:22:32 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> Message-ID: On Mon, 11 Dec 2023 06:05:52 GMT, David Holmes wrote: >> Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): >> test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. >> >> Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). >> The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). >> >> Run tier1-3 and heapdump-related tests. > > src/hotspot/share/services/heapDumper.cpp line 1984: > >> 1982: // user space. >> 1983: void DumpMerger::merge_file(char* path) { >> 1984: assert(!SafepointSynchronize::is_at_safepoint(), "merging happens outside safepoint"); > > This might fix the failure but why is this restriction in place to begin with? This seems like a very expensive operation to be performing by the VMThread during a safepoint! Main idea of segmented heap dump feature (JDK-8306441, #13667) is to divide heap dump process to 2 phases - dump to segment files (this phase is executed at safepoint) and merge of the segment files (this one is supposed to be executed outside of safepoint, so VM is not blocked in this phase). And this asserts were added to ensure it works as desired. I think merge stage can be performed on the current thread without VMOperation, but I don't know all possible consequences of the change. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1423261940 From amenkov at openjdk.org Tue Dec 12 01:06:29 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 12 Dec 2023 01:06:29 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 14:38:08 GMT, Matthias Baesken wrote: >> A colleague contacted IBM about the different behavior of getaddrinfo on AIX (compared to Linux/macOS); maybe we have to adjust the result of the getaddrinfo call on AIX. > >> A colleague contacted IBM about the different behavior of getaddrinfo on AIX (compared to Linux/macOS); maybe we have to adjust the result of the getaddrinfo call on AIX. > > Haven't heard from them so far, hopefully we get an update soon about the behavior of getaddrinfo on AIX . @MBaesken Just a thought: parseAllowedAddr() needs to parse only numeric addresses. getaddrinfo was used to handle both IPv4 and IPv6 by a single call, but maybe it would be better to reimplement parseAllowedAddr to do 2 inet_pton calls (for AF_INET and AF_INET6) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16561#issuecomment-1851139320 From sspitsyn at openjdk.org Tue Dec 12 01:36:34 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 12 Dec 2023 01:36:34 GMT Subject: RFR: 8311306: Test com/sun/management/ThreadMXBean/ThreadCpuTimeArray.java failed: out of expected range In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 11:33:08 GMT, Kevin Walls wrote: > Trivial test update to run in "othervm". This test has failed due to interrupted threads: the interruption interferes with timing, but also the test is designed to fail if interrupted. > > Other tests in the same directory have the same requirement that threads/sleeps are not interrupted, and they run with othervm and are not known to fail. Looks okay. ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17054#pullrequestreview-1776450680 From yyang at openjdk.org Tue Dec 12 02:16:32 2023 From: yyang at openjdk.org (Yi Yang) Date: Tue, 12 Dec 2023 02:16:32 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 22:29:12 GMT, Alex Menkov wrote: > Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): > test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. > > Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). > The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). > > Run tier1-3 and heapdump-related tests. Marked as reviewed by yyang (Committer). ------------- PR Review: https://git.openjdk.org/jdk/pull/17040#pullrequestreview-1776478407 From dholmes at openjdk.org Tue Dec 12 07:09:34 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 12 Dec 2023 07:09:34 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> Message-ID: On Tue, 12 Dec 2023 00:19:15 GMT, Alex Menkov wrote: >> src/hotspot/share/services/heapDumper.cpp line 1984: >> >>> 1982: // user space. >>> 1983: void DumpMerger::merge_file(char* path) { >>> 1984: assert(!SafepointSynchronize::is_at_safepoint(), "merging happens outside safepoint"); >> >> This might fix the failure but why is this restriction in place to begin with? This seems like a very expensive operation to be performing by the VMThread during a safepoint! > > Main idea of segmented heap dump feature (JDK-8306441, #13667) is to divide heap dump process to 2 phases - dump to segment files (this phase is executed at safepoint) and merge of the segment files (this one is supposed to be executed outside of safepoint, so VM is not blocked in this phase). And this asserts were added to ensure it works as desired. > I think merge stage can be performed on the current thread without VMOperation, but I don't know all possible consequences of the change. Prior to the virtual thread changes what thread is performing the merge? After these changes what thread would do it if not included in the VM operation? The assert is correct - we should not do the merge at the safepoint (or even in the VMThread). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1423523614 From kevinw at openjdk.org Tue Dec 12 10:01:34 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Tue, 12 Dec 2023 10:01:34 GMT Subject: Integrated: 8311306: Test com/sun/management/ThreadMXBean/ThreadCpuTimeArray.java failed: out of expected range In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 11:33:08 GMT, Kevin Walls wrote: > Trivial test update to run in "othervm". This test has failed due to interrupted threads: the interruption interferes with timing, but also the test is designed to fail if interrupted. > > Other tests in the same directory have the same requirement that threads/sleeps are not interrupted, and they run with othervm and are not known to fail. This pull request has now been integrated. Changeset: e1fd663f Author: Kevin Walls URL: https://git.openjdk.org/jdk/commit/e1fd663f224909c09f2f6fab7ad20f7b7864b62b Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod 8311306: Test com/sun/management/ThreadMXBean/ThreadCpuTimeArray.java failed: out of expected range Reviewed-by: sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/17054 From kevinw at openjdk.org Tue Dec 12 10:01:33 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Tue, 12 Dec 2023 10:01:33 GMT Subject: RFR: 8311306: Test com/sun/management/ThreadMXBean/ThreadCpuTimeArray.java failed: out of expected range In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 11:33:08 GMT, Kevin Walls wrote: > Trivial test update to run in "othervm". This test has failed due to interrupted threads: the interruption interferes with timing, but also the test is designed to fail if interrupted. > > Other tests in the same directory have the same requirement that threads/sleeps are not interrupted, and they run with othervm and are not known to fail. Thanks Serguei! ------------- PR Comment: https://git.openjdk.org/jdk/pull/17054#issuecomment-1851707660 From kevinw at openjdk.org Tue Dec 12 10:02:22 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Tue, 12 Dec 2023 10:02:22 GMT Subject: RFR: 8321729: Remove 'orb' field in RMIConnector In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 12:50:38 GMT, Kevin Walls wrote: > Tidyup change, looks like this field was not removed when IIOP was removed from RMIConnector. > > The field is not required for interoperability: > with the field removed, I can still connect an older JMX client to a running app with the updated JDK. > > Since the JDK9 change https://hg.openjdk.org/jdk9/jdk9/jdk/rev/09daaf1e4c53 > did not change serialVersionUID, this update does not either. > > Tests continue to pass, e.g. > javax/management (including remote which has RMI tests) > jdk/com/sun/management Thanks for the reviews! ------------- PR Comment: https://git.openjdk.org/jdk/pull/17055#issuecomment-1851711307 From kevinw at openjdk.org Tue Dec 12 10:05:34 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Tue, 12 Dec 2023 10:05:34 GMT Subject: Integrated: 8321729: Remove 'orb' field in RMIConnector In-Reply-To: References: Message-ID: <2oeA7lalSaM4P8b_2ou3BuuoEcmpn-hLunJHn0rXIDk=.07b627f3-e714-475e-9874-2a27956539df@github.com> On Mon, 11 Dec 2023 12:50:38 GMT, Kevin Walls wrote: > Tidyup change, looks like this field was not removed when IIOP was removed from RMIConnector. > > The field is not required for interoperability: > with the field removed, I can still connect an older JMX client to a running app with the updated JDK. > > Since the JDK9 change https://hg.openjdk.org/jdk9/jdk9/jdk/rev/09daaf1e4c53 > did not change serialVersionUID, this update does not either. > > Tests continue to pass, e.g. > javax/management (including remote which has RMI tests) > jdk/com/sun/management This pull request has now been integrated. Changeset: 6f482406 Author: Kevin Walls URL: https://git.openjdk.org/jdk/commit/6f4824068dbd0631ac73c79de479245e0c53ed81 Stats: 6 lines in 1 file changed: 0 ins; 6 del; 0 mod 8321729: Remove 'orb' field in RMIConnector Reviewed-by: rriggs, dfuchs ------------- PR: https://git.openjdk.org/jdk/pull/17055 From jkern at openjdk.org Tue Dec 12 14:05:48 2023 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 12 Dec 2023 14:05:48 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: followed the proposals ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/2d32c43b..b7676822 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=02-03 Stats: 485 lines in 6 files changed: 329 ins; 149 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From eastigeevich at openjdk.org Tue Dec 12 14:38:22 2023 From: eastigeevich at openjdk.org (Evgeny Astigeevich) Date: Tue, 12 Dec 2023 14:38:22 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: <0g42xTuuwIpZZ8aF4IMVWt6T1tI-ExyeSY31pI6Wv80=.df942a73-d418-4548-b6e1-f344e4896bed@github.com> On Mon, 11 Dec 2023 22:41:56 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) >> >> >> Compiler.perfmap [arguments] (Linux only) >> Write map file for Linux perf tool. >> >> Impact: Low >> >> arguments: >> >> ? filename: (Optional) Name of the map file (STRING, no default value) >> >> If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then >> the default filename will be /tmp/perf-12345.map. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright of PerfMapTest lgtm ------------- Marked as reviewed by eastigeevich (Committer). PR Review: https://git.openjdk.org/jdk/pull/15871#pullrequestreview-1777620332 From amenkov at openjdk.org Tue Dec 12 22:02:35 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 12 Dec 2023 22:02:35 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> Message-ID: <7slUFI0uNgKfMUIsekZHL-sjqHXdvoTwpRVddEC1f-I=.9ad81ada-4865-45c0-98e7-a6c775abaeff@github.com> On Tue, 12 Dec 2023 07:06:07 GMT, David Holmes wrote: > Prior to the virtual thread changes what thread is performing the merge? Prior the change merge is performed on either AttachListener thread (when it's the current thread - special case for tools like jcmd) or on VMThread (if the current thread is not AttachListener thread). Note also that prior the change segmented dumping was conditional - if GC supports multiple workers and if the caller requests parallel dump. HeapDumpAfterFullGC/HeapDumpBeforeFullGC request dumping in 1 thread, so there was no merge in the scenarios. > After these changes what thread would do it if not included in the VM operation? On the current thread. As I wrote before I 'm not sure if it can cause some unexpected consequences in some cases. I tried to do the change - tier1 passed (TestReduceAllocationAndHeapDump.java test calls Runtime.gc() with HeapDumpAfterFullGC, merge is performed on main java thread) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1424619940 From amenkov at openjdk.org Tue Dec 12 22:21:35 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 12 Dec 2023 22:21:35 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: <7slUFI0uNgKfMUIsekZHL-sjqHXdvoTwpRVddEC1f-I=.9ad81ada-4865-45c0-98e7-a6c775abaeff@github.com> References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> <7slUFI0uNgKfMUIsekZHL-sjqHXdvoTwpRVddEC1f-I=.9ad81ada-4865-45c0-98e7-a6c775abaeff@github.com> Message-ID: On Tue, 12 Dec 2023 21:59:12 GMT, Alex Menkov wrote: >> Prior to the virtual thread changes what thread is performing the merge? After these changes what thread would do it if not included in the VM operation? >> >> The assert is correct - we should not do the merge at the safepoint (or even in the VMThread). > >> Prior to the virtual thread changes what thread is performing the merge? > > Prior the change merge is performed on either AttachListener thread (when it's the current thread - special case for tools like jcmd) or on VMThread (if the current thread is not AttachListener thread). > Note also that prior the change segmented dumping was conditional - if GC supports multiple workers and if the caller requests parallel dump. HeapDumpAfterFullGC/HeapDumpBeforeFullGC request dumping in 1 thread, so there was no merge in the scenarios. > >> After these changes what thread would do it if not included in the VM operation? > > On the current thread. As I wrote before I 'm not sure if it can cause some unexpected consequences in some cases. > I tried to do the change - tier1 passed (TestReduceAllocationAndHeapDump.java test calls Runtime.gc() with HeapDumpAfterFullGC, merge is performed on main java thread) Sorry, I was wrong. TestReduceAllocationAndHeapDump.java failed on the assert. I.e. merge is performed on main java thread, but VM is at safepoint ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1424634783 From lmesnik at openjdk.org Tue Dec 12 22:55:22 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Tue, 12 Dec 2023 22:55:22 GMT Subject: RFR: 8321713: Harmonize executeTestJvm with create[Limited]TestJavaProcessBuilder [v3] In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 14:06:43 GMT, Stefan Karlsson wrote: >> [JDK-8315097](https://bugs.openjdk.org/browse/JDK-8315097): 'Rename createJavaProcessBuilder' changed the name of the ProcessTools helper functions used to create `ProcessBuilder`s used to spawn new java test processes. >> >> We now have `createTestJavaProcessBuilder` and `createLimitedTestJavaProcess`. The former prepends jvm options from jtreg, while the latter doesn't. >> >> With these functions it is common to see the following pattern in tests: >> >> ProcessBuilder pb = ProcessTools.createTestJavaProcessBuilder(...); >> OutputAnalyzer output = executeProcess(pb); >> >> >> We have a couple of thin wrapper in `ProcessTools` that does exactly this, so that the code can be written as a one-liner: >> >> OutputAnalyzer output = ProcessTools.executeTestJvm(); >> >> >> I propose that we name this functions using the same naming scheme we used for `createTestJavaProcessBuilder` and `createLimitedTestJavaProcessBuilder`. That is, we change `executeTestJvm` to `executeTestJava` and add a new `executeLimitedTestJava` function. > > Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: > > Test cleanup Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/17049#pullrequestreview-1778566038 From dholmes at openjdk.org Tue Dec 12 23:21:34 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 12 Dec 2023 23:21:34 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> <7slUFI0uNgKfMUIsekZHL-sjqHXdvoTwpRVddEC1f-I=.9ad81ada-4865-45c0-98e7-a6c775abaeff@github.com> Message-ID: On Tue, 12 Dec 2023 22:18:33 GMT, Alex Menkov wrote: > I.e. merge is performed on main java thread, but VM is at safepoint So is the main thread operating in_native whilst doing the merge? I suspect the admonition of not doing the merge at a safepoint actually meant "not a safepoint by the VMThread" as that would cause the whole VM to pause. Even doing it in the VMThread at all can delay the next safepoint, which does not seem good. I'm not familiar with this code in general (and only looked at this because of the previous issue in the CI) but I'm unclear what including virtual thread stack referencves has to do with the merging logic? I would not expect merging to be affected by the current change. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1424670985 From lmesnik at openjdk.org Wed Dec 13 00:04:44 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Wed, 13 Dec 2023 00:04:44 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Fri, 8 Dec 2023 11:54:40 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods Changes requested by lmesnik (Reviewer). src/hotspot/share/prims/jvm.cpp line 4013: > 4011: // Notification from VirtualThread about entering/exiting sync critical section. > 4012: // Needed to avoid deadlocks with JVMTI suspend mechanism. > 4013: JVM_ENTRY(void, JVM_VirtualThreadCriticalLock(JNIEnv* env, jobject vthread, jboolean enter)) the jobject vthread is not used. Can't be the method made static to reduce the number of arguments? It is the performance-critical code, I don't know if it is optimized by C2. src/hotspot/share/runtime/javaThread.hpp line 320: > 318: bool _is_in_VTMS_transition; // thread is in virtual thread mount state transition > 319: bool _is_in_tmp_VTMS_transition; // thread is in temporary virtual thread mount state transition > 320: bool _is_in_critical_section; // thread is in a locking critical section might make sense to add a comment, that his variable Is changed/read only by current thread and no sync is needed. src/java.base/share/classes/java/lang/VirtualThread.java line 1164: > 1162: > 1163: @IntrinsicCandidate > 1164: private native void notifyJvmtiCriticalLock(boolean enter); The name is confusing to me, the CriticalLock looks like it is the section is critical and might be taken by a single thread only. Or it's just unclear what is critical here. However, the purpose is to disable suspend Wouldn't be 'notifyJvmtiSuspendLock notifyJvmtiDisableSuspend' better name here? or comment what critical means here. test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock/SuspendWithInterruptLock.java line 30: > 28: * @requires vm.continuations > 29: * @library /testlibrary > 30: * @run main/othervm -Xint SuspendWithInterruptLock Doesn't it make sense to add a testcase without -Xint also? Just to give stress testing with compilation. test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock/SuspendWithInterruptLock.java line 36: > 34: > 35: public class SuspendWithInterruptLock { > 36: static boolean done; done is accessed from different threads, should be volatile. test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock/SuspendWithInterruptLock.java line 54: > 52: Thread.yield(); > 53: } > 54: done = true; I think it is better to use done to stop all threads and set it to true in the main thread after some time. So you could be sure that the yielder hadn't been completed before the suspender started. But it is just proposal. ------------- PR Review: https://git.openjdk.org/jdk/pull/17011#pullrequestreview-1778571090 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1424694672 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1424697179 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1424687810 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1424662055 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1424663078 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1424683585 From amenkov at openjdk.org Wed Dec 13 01:40:45 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 13 Dec 2023 01:40:45 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> <7slUFI0uNgKfMUIsekZHL-sjqHXdvoTwpRVddEC1f-I=.9ad81ada-4865-45c0-98e7-a6c775abaeff@github.com> Message-ID: On Tue, 12 Dec 2023 23:11:41 GMT, David Holmes wrote: > So is the main thread operating in_native whilst doing the merge? I suspect the admonition of not doing the merge at a safepoint actually meant "not a safepoint by the VMThread" as that would cause the whole VM to pause. Even doing it in the VMThread at all can delay the next safepoint, which does not seem good. I think the thread is in_vm. Main java thread call Runtime.gc(), GC itself calls heap dumper before/after full gc. I don't know if there is a performance difference in the case between executing merge on the current thread and VMThread. HeapDumpBeforeFullGC/HeapDumpAfterFullGC/HeapDumpOnOutOfMemoryError are diagnostic flags, so I think performance is not so important here. There are plans to improve their performance by turning on parallel dump (using GC worker) - often parallel heap dump + merge takes less time than single-threaded dump (we have now). > I'm not familiar with this code in general (and only looked at this because of the previous issue in the CI) but I'm unclear what including virtual thread stack referencves has to do with the merging logic? I would not expect merging to be affected by the current change. This is caused by a nature of unmounted virtual threads. We need to dump stack traces before we dump heap objects, but we can find unmounted virtual threads only by iterating over heap. So we need a way to add data to the beginning of the dump file while dumping heap objects. Segmented dump helps here - we have a designated segment to write stack traces. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1424747414 From amenkov at openjdk.org Wed Dec 13 05:01:48 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 13 Dec 2023 05:01:48 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> <7slUFI0uNgKfMUIsekZHL-sjqHXdvoTwpRVddEC1f-I=.9ad81ada-4865-45c0-98e7-a6c775abaeff@github.com> Message-ID: <8g_qYNlXu7mzCiIRDyb9HEAStKcDw4GL_QBxKtsTSFM=.070c31f5-3ce4-40b9-be92-91ad7ed9eca2@github.com> On Wed, 13 Dec 2023 01:38:03 GMT, Alex Menkov wrote: >>> I.e. merge is performed on main java thread, but VM is at safepoint >> >> So is the main thread operating in_native whilst doing the merge? I suspect the admonition of not doing the merge at a safepoint actually meant "not a safepoint by the VMThread" as that would cause the whole VM to pause. Even doing it in the VMThread at all can delay the next safepoint, which does not seem good. >> >> I'm not familiar with this code in general (and only looked at this because of the previous issue in the CI) but I'm unclear what including virtual thread stack referencves has to do with the merging logic? I would not expect merging to be affected by the current change. > >> So is the main thread operating in_native whilst doing the merge? I suspect the admonition of not doing the merge at a safepoint actually meant "not a safepoint by the VMThread" as that would cause the whole VM to pause. Even doing it in the VMThread at all can delay the next safepoint, which does not seem good. > > I think the thread is in_vm. > Main java thread call Runtime.gc(), GC itself calls heap dumper before/after full gc. > I don't know if there is a performance difference in the case between executing merge on the current thread and VMThread. > HeapDumpBeforeFullGC/HeapDumpAfterFullGC/HeapDumpOnOutOfMemoryError are diagnostic flags, so I think performance is not so important here. > There are plans to improve their performance by turning on parallel dump (using GC worker) - often parallel heap dump + merge takes less time than single-threaded dump (we have now). > >> I'm not familiar with this code in general (and only looked at this because of the previous issue in the CI) but I'm unclear what including virtual thread stack referencves has to do with the merging logic? I would not expect merging to be affected by the current change. > > This is caused by a nature of unmounted virtual threads. > We need to dump stack traces before we dump heap objects, but we can find unmounted virtual threads only by iterating over heap. So we need a way to add data to the beginning of the dump file while dumping heap objects. Segmented dump helps here - we have a designated segment to write stack traces. I tested different scenarios (all I know) of heap dump: - via attach (jcmd); - JMX (HotSpotDiagnosticMXBean); - HeapDumpBeforeFullGC/HeapDumpAfterFullGC; - HeapDumpOnOutOfMemoryError. I see no problem with merge on the current thread. Only HeapDumpBeforeFullGC/HeapDumpAfterFullGC causes heap dump (and merge) at safepoint. But HeapDumpBeforeFullGC/HeapDumpAfterFullGC/HeapDumpOnOutOfMemoryError depend on GC and require much more testing, I think it's better to integrate this fix as it is and update the way merger is executed by a separate follow-up change (with cleanup of related stuff) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1424848688 From dholmes at openjdk.org Wed Dec 13 07:24:45 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 13 Dec 2023 07:24:45 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: <8g_qYNlXu7mzCiIRDyb9HEAStKcDw4GL_QBxKtsTSFM=.070c31f5-3ce4-40b9-be92-91ad7ed9eca2@github.com> References: <9JfEtFZ7b-9NlcNyIQTS4CMUXTUPXHSFY_VN3OLQ28s=.e470285f-b4da-4ced-8981-5d91e415e136@github.com> <7slUFI0uNgKfMUIsekZHL-sjqHXdvoTwpRVddEC1f-I=.9ad81ada-4865-45c0-98e7-a6c775abaeff@github.com> <8g_qYNlXu7mzCiIRDyb9HEAStKcDw4GL_QBxKtsTSFM=.070c31f5-3ce4-40b9-be92-91ad7ed9eca2@github.com> Message-ID: <0l3DaiQ3yLEZlQBwYqz0VJY-xg-NKfEotrYc09b5i9M=.95163d85-032e-4a87-b94f-79c263695d99@github.com> On Wed, 13 Dec 2023 04:59:16 GMT, Alex Menkov wrote: >>> So is the main thread operating in_native whilst doing the merge? I suspect the admonition of not doing the merge at a safepoint actually meant "not a safepoint by the VMThread" as that would cause the whole VM to pause. Even doing it in the VMThread at all can delay the next safepoint, which does not seem good. >> >> I think the thread is in_vm. >> Main java thread call Runtime.gc(), GC itself calls heap dumper before/after full gc. >> I don't know if there is a performance difference in the case between executing merge on the current thread and VMThread. >> HeapDumpBeforeFullGC/HeapDumpAfterFullGC/HeapDumpOnOutOfMemoryError are diagnostic flags, so I think performance is not so important here. >> There are plans to improve their performance by turning on parallel dump (using GC worker) - often parallel heap dump + merge takes less time than single-threaded dump (we have now). >> >>> I'm not familiar with this code in general (and only looked at this because of the previous issue in the CI) but I'm unclear what including virtual thread stack referencves has to do with the merging logic? I would not expect merging to be affected by the current change. >> >> This is caused by a nature of unmounted virtual threads. >> We need to dump stack traces before we dump heap objects, but we can find unmounted virtual threads only by iterating over heap. So we need a way to add data to the beginning of the dump file while dumping heap objects. Segmented dump helps here - we have a designated segment to write stack traces. > > I tested different scenarios (all I know) of heap dump: > - via attach (jcmd); > - JMX (HotSpotDiagnosticMXBean); > - HeapDumpBeforeFullGC/HeapDumpAfterFullGC; > - HeapDumpOnOutOfMemoryError. > I see no problem with merge on the current thread. > Only HeapDumpBeforeFullGC/HeapDumpAfterFullGC causes heap dump (and merge) at safepoint. > But HeapDumpBeforeFullGC/HeapDumpAfterFullGC/HeapDumpOnOutOfMemoryError depend on GC and require much more testing, I think it's better to integrate this fix as it is and update the way merger is executed by a separate follow-up change (with cleanup of related stuff) Okay. I'm concerned that the merge, whether by the current thread whilst in_vm, or by the VMThread, will either prevent the VM from going to a safepoint in a timely manner, or cause the safepoint to be excessively long. I'm not sure how problems in this area will become visible. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17040#discussion_r1424940134 From dholmes at openjdk.org Wed Dec 13 07:24:44 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 13 Dec 2023 07:24:44 GMT Subject: RFR: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 22:29:12 GMT, Alex Menkov wrote: > Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): > test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. > > Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). > The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). > > Run tier1-3 and heapdump-related tests. Approving the removal of the problematic assert from the first version. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17040#pullrequestreview-1778987777 From amenkov at openjdk.org Wed Dec 13 18:49:50 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 13 Dec 2023 18:49:50 GMT Subject: Integrated: JDK-8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Fri, 8 Dec 2023 22:29:12 GMT, Alex Menkov wrote: > Original fix for JDK-8299426 (Heap dump does not contain virtual Thread stack references, #16665) caused failures of new test (added while #16665 was under review): > test/hotspot/jtreg/compiler/c2/TestReduceAllocationAndHeapDump.java in many tears and was reverted. > > Segmented heap dump assumes "merge" stage is executed outside of safepoint (to not block the VM), but heap dump may happen during safepoint (and TestReduceAllocationAndHeapDump.java test provoke the case). > The change contains original fix for JDK-8299426 ("[original fix](https://github.com/openjdk/jdk/commit/bdbf768eafa86e0007aca4188e0567693afe9071)") and removes asserts from HeapMerger ([allow heapdump in safepoints](https://github.com/openjdk/jdk/commit/44670ca4bf55dd2a5f1f44686758844aed68937e)). > > Run tier1-3 and heapdump-related tests. This pull request has now been integrated. Changeset: cf948548 Author: Alex Menkov URL: https://git.openjdk.org/jdk/commit/cf948548c390c42ca63525d41a9d63ff31349c3a Stats: 319 lines in 3 files changed: 152 ins; 81 del; 86 mod 8321565: [REDO] Heap dump does not contain virtual Thread stack references Reviewed-by: sspitsyn, yyang, dholmes ------------- PR: https://git.openjdk.org/jdk/pull/17040 From dlong at openjdk.org Wed Dec 13 20:54:45 2023 From: dlong at openjdk.org (Dean Long) Date: Wed, 13 Dec 2023 20:54:45 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 22:41:56 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) >> >> >> Compiler.perfmap [arguments] (Linux only) >> Write map file for Linux perf tool. >> >> Impact: Low >> >> arguments: >> >> ? filename: (Optional) Name of the map file (STRING, no default value) >> >> If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then >> the default filename will be /tmp/perf-12345.map. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright of PerfMapTest The man page says "no default value" but then right below describes the default value, which is confusing. I would remove "no default value". The code already deals with patterns, so why not allow a pattern like /dir/perf-%x.map and document that the platform-specific process id will be passed to String.format() to expand any formatting tokens in the string? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1854683037 From cjplummer at openjdk.org Wed Dec 13 21:20:45 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 13 Dec 2023 21:20:45 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: On Wed, 13 Dec 2023 20:51:36 GMT, Dean Long wrote: > The man page says "no default value" but then right below describes the default value, which is confusing. I would remove "no default value". This was copied from VM.cds, which pretty much does the same thing (says "no default value" and then explains the default value below). Since the default is based on the pid, and we probably don't want a long description here, maybe just say "/tmp/perf-.map". Or it can say "(STRING, system-generated default name)" as I see in one other jcmd. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1854714496 From amenkov at openjdk.org Wed Dec 13 21:38:54 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 13 Dec 2023 21:38:54 GMT Subject: RFR: JDK-8318563: GetClassFields should not use random access to field Message-ID: FieldStream/FilteredFieldStream classes from reflectionUtils.hpp iterate class fields in the reverse order and use field indexes to access instead of forward iteration. This is performance ineffective (see [JDK-8317692](https://bugs.openjdk.org/browse/JDK-8317692) for details). The change introduces new class FilteredJavaFieldStream as a replacement for FilteredFieldStream. It uses the same FilteredField/FilteredFieldsMap stuff as FilteredJavaFieldStream does. FieldStream/FilteredFieldStream are still used by heap walking API, will be cleaned by [JDK-8317636](https://bugs.openjdk.org/browse/JDK-8317636) Testing: - tier1..3 - all tests which calls GetClassFields: open/test/hotspot/jtreg/serviceability/jvmti,open/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields,open/test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateThroughHeap,open/test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/IsSynthetic including - test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields/getclfld007.java - tests that GetClassFields returns fields in correct order; - test/hotspot/jtreg/serviceability/jvmti/GetClassFields/FilteredFields/FilteredFieldsTest.java - test that GetClassFields filters out field like reflection. ------------- Commit messages: - GetClassFields Changes: https://git.openjdk.org/jdk/pull/17094/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17094&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8318563 Stats: 51 lines in 2 files changed: 36 ins; 8 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/17094.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17094/head:pull/17094 PR: https://git.openjdk.org/jdk/pull/17094 From ddong at openjdk.org Thu Dec 14 02:08:42 2023 From: ddong at openjdk.org (Denghui Dong) Date: Thu, 14 Dec 2023 02:08:42 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC In-Reply-To: References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> Message-ID: On Wed, 6 Dec 2023 04:00:51 GMT, David Holmes wrote: >> Hi, >> >> Could I have a review of this patch? >> >> In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. >> >> This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. >> >> Best, >> Denghui > > @D-D-H adding a new manageable flag requires a CSR request to be approved. @dholmes-ora Thanks for the review. Could anyone in the serviceability area help review CSR and this patch? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16976#issuecomment-1854994491 From sspitsyn at openjdk.org Thu Dec 14 02:52:36 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 02:52:36 GMT Subject: RFR: JDK-8318563: GetClassFields should not use random access to field In-Reply-To: References: Message-ID: <73S5mXiweVRQDjHUz3dHncaRgYfQb3gzSiwqVshk_Ak=.d15ac98f-a569-40b6-9023-01d7f01ba27c@github.com> On Wed, 13 Dec 2023 21:32:50 GMT, Alex Menkov wrote: > FieldStream/FilteredFieldStream classes from reflectionUtils.hpp iterate class fields in the reverse order and use field indexes to access instead of forward iteration. This is performance ineffective (see [JDK-8317692](https://bugs.openjdk.org/browse/JDK-8317692) for details). > The change introduces new class FilteredJavaFieldStream as a replacement for FilteredFieldStream. > It uses the same FilteredField/FilteredFieldsMap stuff as FilteredJavaFieldStream does. > > FieldStream/FilteredFieldStream are still used by heap walking API, will be cleaned by [JDK-8317636](https://bugs.openjdk.org/browse/JDK-8317636) > > Testing: > - tier1..3 > - all tests which calls GetClassFields: open/test/hotspot/jtreg/serviceability/jvmti,open/test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields,open/test/hotspot/jtreg/vmTestbase/nsk/jvmti/IterateThroughHeap,open/test/hotspot/jtreg/vmTestbase/nsk/jvmti/unit/IsSynthetic > including > - test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields/getclfld007.java - tests that GetClassFields returns fields in correct order; > - test/hotspot/jtreg/serviceability/jvmti/GetClassFields/FilteredFields/FilteredFieldsTest.java - test that GetClassFields filters out field like reflection. This looks good in general. Will make one more pass tomorrow. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17094#issuecomment-1855032678 From dholmes at openjdk.org Thu Dec 14 05:51:45 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 14 Dec 2023 05:51:45 GMT Subject: RFR: 8322065: Initial nroff manpage generation for JDK 23 Message-ID: Updated the version to 23-ea and year to 2024. This initial generation also picks up the unpublished changes from: - [JDK-8302233](https://bugs.openjdk.org/browse/JDK-8302233) (keytool & jarsigner) - [JDK-8290702](https://bugs.openjdk.org/browse/JDK-8290702) (javadoc) (JDK 23 backport) - [JDK-8321384](https://bugs.openjdk.org/browse/JDK-8321384) (javadoc) In addition this includes the updates for - [JDK-8309981](https://bugs.openjdk.org/browse/8309981) Remove expired flags in JDK 23 Thanks ------------- Commit messages: - 8322065: Initial nroff manpage generation for JDK 23 - 8309981: Remove expired flags in JDK 23 Changes: https://git.openjdk.org/jdk/pull/17101/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17101&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8322065 Stats: 216 lines in 29 files changed: 47 ins; 61 del; 108 mod Patch: https://git.openjdk.org/jdk/pull/17101.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17101/head:pull/17101 PR: https://git.openjdk.org/jdk/pull/17101 From alanb at openjdk.org Thu Dec 14 09:03:38 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 14 Dec 2023 09:03:38 GMT Subject: RFR: 8322065: Initial nroff manpage generation for JDK 23 In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 05:46:01 GMT, David Holmes wrote: > Updated the version to 23-ea and year to 2024. > > This initial generation also picks up the unpublished changes from: > > - [JDK-8302233](https://bugs.openjdk.org/browse/JDK-8302233) (keytool & jarsigner) > - [JDK-8290702](https://bugs.openjdk.org/browse/JDK-8290702) (javadoc) (JDK 23 backport) > - [JDK-8321384](https://bugs.openjdk.org/browse/JDK-8321384) (javadoc) > > > In addition this includes the updates for > > - [JDK-8309981](https://bugs.openjdk.org/browse/8309981) Remove expired flags in JDK 23 > > Thanks Initially I wondered if JDK-8309981 should be separated but include keeps things in sync so I think okay. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17101#pullrequestreview-1781343785 From prappo at openjdk.org Thu Dec 14 09:19:37 2023 From: prappo at openjdk.org (Pavel Rappo) Date: Thu, 14 Dec 2023 09:19:37 GMT Subject: RFR: 8322065: Initial nroff manpage generation for JDK 23 In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 05:46:01 GMT, David Holmes wrote: > Updated the version to 23-ea and year to 2024. > > This initial generation also picks up the unpublished changes from: > > - [JDK-8302233](https://bugs.openjdk.org/browse/JDK-8302233) (keytool & jarsigner) > - [JDK-8290702](https://bugs.openjdk.org/browse/JDK-8290702) (javadoc) (JDK 23 backport) > - [JDK-8321384](https://bugs.openjdk.org/browse/JDK-8321384) (javadoc) > > > In addition this includes the updates for > > - [JDK-8309981](https://bugs.openjdk.org/browse/8309981) Remove expired flags in JDK 23 > > Thanks > Updated the version to 23-ea and year to 2024. > > This initial generation also picks up the unpublished changes from: > > * [JDK-8321384](https://bugs.openjdk.org/browse/JDK-8321384) (javadoc) Thanks for doing this, David. I only note that the changes for JDK-8321384 were published in [JDK-8308715](https://bugs.openjdk.org/browse/JDK-8308715), which was integrated in the mainline before JDK 22 RDP 1. So they are already present in the mainline. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17101#issuecomment-1855467435 From sspitsyn at openjdk.org Thu Dec 14 12:14:40 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 12:14:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Fri, 8 Dec 2023 11:54:40 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods @AlanBateman Thank you for reviewing an the comment. > It shouldn't be necessary to touch mount/unmount as the thread identity is the carrier, not the virtual thread, when executing the "critical code". Carrier thread also can be suspended when executing the "critical code". Why do you think it can't be a problem? Do you think the deadlocking scenario described in the bug report is not possible? > toggle_is_in_critical_section needs to detect reentrancy, it is otherwise too easy to refactor the Java code, e.g. call threadState while holding the interrupt lock. Is your concern a recursive `interruptLock` enter? I was also thinking if this scenario is possible, so a counter can be used instead of boolean. > All the use-sides will need to use try-finally to more reliably revert the critical section flag when rewinding. Right, thanks. It is already done. > The naming is very problematic, we'll need to replace with methods that are clearly named enter and exit critical section. Ongoing work in this area to support monitors has to introduce some temporary pinning so there will be enter/exitCriticalSection methods, that's a better place for the JVMTI hooks. Okay. What about the Leonid's suggestion to name it `notifyJvmtiDisableSuspend()` ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1855730274 From sspitsyn at openjdk.org Thu Dec 14 12:14:45 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 12:14:45 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Tue, 12 Dec 2023 23:42:07 GMT, Leonid Mesnik wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods > > src/java.base/share/classes/java/lang/VirtualThread.java line 1164: > >> 1162: >> 1163: @IntrinsicCandidate >> 1164: private native void notifyJvmtiCriticalLock(boolean enter); > > The name is confusing to me, the CriticalLock looks like it is the section is critical and might be taken by a single thread only. Or it's just unclear what is critical here. > However, the purpose is to disable suspend > Wouldn't be 'notifyJvmtiSuspendLock notifyJvmtiDisableSuspend' better name here? > or comment what critical means here. Okay, thanks. I like your name suggestion but let's check with Alan first. > test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock/SuspendWithInterruptLock.java line 30: > >> 28: * @requires vm.continuations >> 29: * @library /testlibrary >> 30: * @run main/othervm -Xint SuspendWithInterruptLock > > Doesn't it make sense to add a testcase without -Xint also? Just to give stress testing with compilation. Thanks. I was also thinking about this. Will add a sub-test without -Xint. > test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock/SuspendWithInterruptLock.java line 36: > >> 34: >> 35: public class SuspendWithInterruptLock { >> 36: static boolean done; > > done is accessed from different threads, should be volatile. Good suggestion, thanks. > test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock/SuspendWithInterruptLock.java line 54: > >> 52: Thread.yield(); >> 53: } >> 54: done = true; > > I think it is better to use done to stop all threads and set it to true in the main thread after some time. So you could be sure that the yielder hadn't been completed before the suspender started. But it is just proposal. Thank you. Will consider this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1426638981 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1426635613 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1426636196 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1426637200 From sspitsyn at openjdk.org Thu Dec 14 12:19:43 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 12:19:43 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Tue, 12 Dec 2023 23:54:43 GMT, Leonid Mesnik wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods > > src/hotspot/share/prims/jvm.cpp line 4013: > >> 4011: // Notification from VirtualThread about entering/exiting sync critical section. >> 4012: // Needed to avoid deadlocks with JVMTI suspend mechanism. >> 4013: JVM_ENTRY(void, JVM_VirtualThreadCriticalLock(JNIEnv* env, jobject vthread, jboolean enter)) > > the jobject vthread is not used. Can't be the method made static to reduce the number of arguments? > It is the performance-critical code, I don't know if it is optimized by C2. Good question. In general, I'd like to keep this unified with the other `notiftJvmti` methods. Let me double check how it fits together. Also, I'm not sure how is going to impact the intrinsification. > src/hotspot/share/runtime/javaThread.hpp line 320: > >> 318: bool _is_in_VTMS_transition; // thread is in virtual thread mount state transition >> 319: bool _is_in_tmp_VTMS_transition; // thread is in temporary virtual thread mount state transition >> 320: bool _is_in_critical_section; // thread is in a locking critical section > > might make sense to add a comment, that his variable Is changed/read only by current thread and no sync is needed. Good suggestion, thanks. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1426643218 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1426643663 From alanb at openjdk.org Thu Dec 14 12:22:40 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 14 Dec 2023 12:22:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Thu, 14 Dec 2023 12:06:41 GMT, Serguei Spitsyn wrote: > Carrier thread also can be suspended when executing the "critical code". Why do you think it can't be a problem? Do you think the deadlocking scenario described in the bug report is not possible? It's a different scenario. When mounting, the coordination of the interrupt status is done before the thread identity is changed. Similarly, when unmounting, the coordination is done after reverting the thread identity to the carrier. So if there is an agent randomly suspending threads when it shouldn't be an issue here. > > toggle_is_in_critical_section needs to detect reentrancy, it is otherwise too easy to refactor the Java code, e.g. call threadState while holding the interrupt lock. > > Is your concern a recursive `interruptLock` enter? I was also thinking if this scenario is possible, so a counter can be used instead of boolean. Minimally an assert. A counter might be needed later. > Okay. What about the Leonid's suggestion to name it `notifyJvmtiDisableSuspend()` ? We have changes in the works that require pinning during some critical sections so I think I prefer to use that terminology. We can move the notification to JVMTI to the enter/leave methods. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1855748841 From dholmes at openjdk.org Thu Dec 14 12:26:40 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 14 Dec 2023 12:26:40 GMT Subject: RFR: 8322065: Initial nroff manpage generation for JDK 23 In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 09:17:05 GMT, Pavel Rappo wrote: > Thanks for doing this, David. I only note that the changes for JDK-8321384 were published in [JDK-8308715](https://bugs.openjdk.org/browse/JDK-8308715), which was integrated in the mainline before JDK 22 RDP 1. So they are already present in the mainline. Ah I see. Thanks for correcting that, I will update the PR and JBS issue. And thanks for looking at this @pavelrappo . ------------- PR Comment: https://git.openjdk.org/jdk/pull/17101#issuecomment-1855755042 From dholmes at openjdk.org Thu Dec 14 12:30:38 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 14 Dec 2023 12:30:38 GMT Subject: RFR: 8322065: Initial nroff manpage generation for JDK 23 In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 09:01:17 GMT, Alan Bateman wrote: > Initially I wondered if JDK-8309981 should be separated but include keeps things in sync so I think okay. Thanks for the review @AlanBateman . Yeah I was in two minds there myself. I started fixing [JDK-8309981](https://bugs.openjdk.org/browse/JDK-8309981) only to discover that the start of release updates had not been done as part of the start of release, so I figured I may as well fix it all together given I'd generated all the updated files anyway. But I'm still a little unsure ... in fact I think I will remove it in the morning. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17101#issuecomment-1855761906 From dholmes at openjdk.org Thu Dec 14 12:35:52 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 14 Dec 2023 12:35:52 GMT Subject: RFR: 8322065: Initial nroff manpage generation for JDK 23 [v2] In-Reply-To: References: Message-ID: > Updated the version to 23-ea and year to 2024. > > This initial generation also picks up the unpublished changes from: > > - [JDK-8302233](https://bugs.openjdk.org/browse/JDK-8302233) (keytool & jarsigner) > - [JDK-8290702](https://bugs.openjdk.org/browse/JDK-8290702) (javadoc) (JDK 23 backport) > > Thanks David Holmes has updated the pull request incrementally with one additional commit since the last revision: Revert "8309981: Remove expired flags in JDK 23" This reverts commit 0324a90e936ae01e42ae099e7235156326cc318a. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17101/files - new: https://git.openjdk.org/jdk/pull/17101/files/65a8c9ed..8b052141 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17101&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17101&range=00-01 Stats: 23 lines in 2 files changed: 10 ins; 11 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/17101.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17101/head:pull/17101 PR: https://git.openjdk.org/jdk/pull/17101 From dchuyko at openjdk.org Thu Dec 14 15:29:06 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Thu, 14 Dec 2023 15:29:06 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v15] In-Reply-To: References: Message-ID: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - ... and 23 more: https://git.openjdk.org/jdk/compare/fde5b168...44d680cd ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=14 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From sroy at openjdk.org Thu Dec 14 16:21:40 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Thu, 14 Dec 2023 16:21:40 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v3] In-Reply-To: <7V0zHrWeOjnDyHJuq3DFsb-BvaQvZbwE5zIGyxWvGNE=.48a0fc72-c70d-4e21-891a-5f4714bac830@github.com> References: <7V0zHrWeOjnDyHJuq3DFsb-BvaQvZbwE5zIGyxWvGNE=.48a0fc72-c70d-4e21-891a-5f4714bac830@github.com> Message-ID: On Tue, 5 Dec 2023 13:48:11 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> encapsulate everything in os::Aix::dlopen > > Excellent, this is how I have pictured a good solution. Very nice. > > A number of remarks, but nothing fundamental. @tstuefe Sorry to tag you. Can you review the code. Once this code goes in I can push in my changes. We are targeting the fix for January. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1856145815 From sspitsyn at openjdk.org Thu Dec 14 16:59:40 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 16:59:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Thu, 14 Dec 2023 12:11:42 GMT, Serguei Spitsyn wrote: >> src/java.base/share/classes/java/lang/VirtualThread.java line 1164: >> >>> 1162: >>> 1163: @IntrinsicCandidate >>> 1164: private native void notifyJvmtiCriticalLock(boolean enter); >> >> The name is confusing to me, the CriticalLock looks like it is the section is critical and might be taken by a single thread only. Or it's just unclear what is critical here. >> However, the purpose is to disable suspend >> Wouldn't be 'notifyJvmtiSuspendLock notifyJvmtiDisableSuspend' better name here? >> or comment what critical means here. > > Okay, thanks. I like your name suggestion but let's check with Alan first. Implemented this renaming suggestion. Let's wait if Alan ia okay with it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1426990736 From alanb at openjdk.org Thu Dec 14 17:08:40 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 14 Dec 2023 17:08:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Thu, 14 Dec 2023 16:57:25 GMT, Serguei Spitsyn wrote: > Implemented this renaming suggestion. Let's wait if Alan ia okay with it. Are you planning to drop the changes to mount/unmount too? They shouldn't be needed. notifyJvmtiCriticalLock(boolean) is okay for now but needs to be called before the try, not in the block. We have changes coming that will require moving these hooks to critical section enter/exit methods, so the naming will be less important then. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427000950 From sspitsyn at openjdk.org Thu Dec 14 17:30:54 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 17:30:54 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v4] In-Reply-To: References: Message-ID: > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: 1) replace CriticalLock with DisableSuspend; 2) minor tweaks ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17011/files - new: https://git.openjdk.org/jdk/pull/17011/files/18f1752e..4e5f6447 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=02-03 Stats: 68 lines in 14 files changed: 9 ins; 10 del; 49 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From sspitsyn at openjdk.org Thu Dec 14 17:37:39 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 17:37:39 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Thu, 14 Dec 2023 17:06:05 GMT, Alan Bateman wrote: >> Implemented this renaming suggestion. Let's wait if Alan ia okay with it. > >> Implemented this renaming suggestion. Let's wait if Alan ia okay with it. > > Are you planning to drop the changes to mount/unmount too? They shouldn't be needed. > > notifyJvmtiCriticalLock(boolean) is okay for now but needs to be called before the try, not in the block. We have changes coming that will require moving these hooks to critical section enter/exit methods, so the naming will be less important then. Yes, I've dropped changes in the mount/unmount methods. I've already done renaming to `notifyJvmtiDisableSuspend(boolean)`. Let's see if it is okay with you. It is not a problem to rename it back to `notifyJvmtiCriticalLock(boolean)`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427032721 From alanb at openjdk.org Thu Dec 14 18:06:40 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 14 Dec 2023 18:06:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: <0IfR_dHcE18gNj1SXT6J0RppMZ2bNU7U51iqi-25bh0=.728701de-31f3-4a6e-a454-a4395f4effe5@github.com> On Thu, 14 Dec 2023 12:19:43 GMT, Alan Bateman wrote: > Okay. What about the Leonid's suggestion to name it `notifyJvmtiDisableSuspend()` ? Okay with me. We'll need to move the notifyJvmtiDisableSuspend(true) to before the try in all cases, I've pointed out the cases that we missed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1856339508 From alanb at openjdk.org Thu Dec 14 18:06:47 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 14 Dec 2023 18:06:47 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v4] In-Reply-To: References: Message-ID: <7LcxdLPaxGRqXIER2MYIoCk6yk0CCnzLk-CtSn7A800=.1de3bfc7-4dc0-443f-bc48-983c22f24766@github.com> On Thu, 14 Dec 2023 17:30:54 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: 1) replace CriticalLock with DisableSuspend; 2) minor tweaks src/java.base/share/classes/java/lang/VirtualThread.java line 746: > 744: } else if ((s == PINNED) || (s == TIMED_PINNED)) { > 745: try { > 746: notifyJvmtiDisableSuspend(true); Move to before the try. src/java.base/share/classes/java/lang/VirtualThread.java line 853: > 851: checkAccess(); > 852: try { > 853: notifyJvmtiDisableSuspend(true); This one also needs to be before try. src/java.base/share/classes/java/lang/VirtualThread.java line 886: > 884: if (oldValue) { > 885: try { > 886: notifyJvmtiDisableSuspend(true); This one also needs to be before try. src/java.base/share/classes/java/lang/VirtualThread.java line 917: > 915: case RUNNING: > 916: try { > 917: notifyJvmtiDisableSuspend(true); This one also needs to be before try. src/java.base/share/classes/java/lang/VirtualThread.java line 1042: > 1040: if (carrier != null) { > 1041: try { > 1042: notifyJvmtiDisableSuspend(true); this one too. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427080057 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427080394 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427080484 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427080704 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427080811 From sspitsyn at openjdk.org Thu Dec 14 18:26:55 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 18:26:55 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v5] In-Reply-To: References: Message-ID: > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: moved notifyJvmtiDisableSuspend(true) out of try-block ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17011/files - new: https://git.openjdk.org/jdk/pull/17011/files/4e5f6447..ad990422 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=03-04 Stats: 10 lines in 1 file changed: 5 ins; 5 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From sspitsyn at openjdk.org Thu Dec 14 18:26:56 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 18:26:56 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: <0IfR_dHcE18gNj1SXT6J0RppMZ2bNU7U51iqi-25bh0=.728701de-31f3-4a6e-a454-a4395f4effe5@github.com> References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> <0IfR_dHcE18gNj1SXT6J0RppMZ2bNU7U51iqi-25bh0=.728701de-31f3-4a6e-a454-a4395f4effe5@github.com> Message-ID: On Thu, 14 Dec 2023 18:04:02 GMT, Alan Bateman wrote: > Okay with me. We'll need to move the notifyJvmtiDisableSuspend(true) to before the try in all cases, I've pointed out the cases that we missed. Thank you, Alan. Fixed now. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1856366484 From sspitsyn at openjdk.org Thu Dec 14 18:26:57 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 18:26:57 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> Message-ID: On Thu, 14 Dec 2023 12:16:34 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/runtime/javaThread.hpp line 320: >> >>> 318: bool _is_in_VTMS_transition; // thread is in virtual thread mount state transition >>> 319: bool _is_in_tmp_VTMS_transition; // thread is in temporary virtual thread mount state transition >>> 320: bool _is_in_critical_section; // thread is in a locking critical section >> >> might make sense to add a comment, that his variable Is changed/read only by current thread and no sync is needed. > > Good suggestion, thanks. Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427099325 From sspitsyn at openjdk.org Thu Dec 14 18:27:00 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 18:27:00 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v4] In-Reply-To: <7LcxdLPaxGRqXIER2MYIoCk6yk0CCnzLk-CtSn7A800=.1de3bfc7-4dc0-443f-bc48-983c22f24766@github.com> References: <7LcxdLPaxGRqXIER2MYIoCk6yk0CCnzLk-CtSn7A800=.1de3bfc7-4dc0-443f-bc48-983c22f24766@github.com> Message-ID: On Thu, 14 Dec 2023 18:03:00 GMT, Alan Bateman wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: 1) replace CriticalLock with DisableSuspend; 2) minor tweaks > > src/java.base/share/classes/java/lang/VirtualThread.java line 1042: > >> 1040: if (carrier != null) { >> 1041: try { >> 1042: notifyJvmtiDisableSuspend(true); > > this one too. Thanks. All cases fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427095561 From alanb at openjdk.org Thu Dec 14 19:53:40 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 14 Dec 2023 19:53:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v3] In-Reply-To: References: <-3HkgdDQk9480AnHQeIxaNsqK0nVzjaf6UZJ5E9LGHo=.bdff2ed6-db98-4a77-832f-6accad54245f@github.com> <0IfR_dHcE18gNj1SXT6J0RppMZ2bNU7U51iqi-25bh0=.728701de-31f3-4a6e-a454-a4395f4effe5@github.com> Message-ID: On Thu, 14 Dec 2023 18:24:16 GMT, Serguei Spitsyn wrote: > Thank you, Alan. Fixed now. I believe, all your suggestions have been addressed now. Thanks, it looks much better now. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1856485757 From alanb at openjdk.org Thu Dec 14 19:53:44 2023 From: alanb at openjdk.org (Alan Bateman) Date: Thu, 14 Dec 2023 19:53:44 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v5] In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 18:26:55 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: moved notifyJvmtiDisableSuspend(true) out of try-block src/java.base/share/classes/java/lang/VirtualThread.java line 918: > 916: notifyJvmtiDisableSuspend(true); > 917: try { > 918: // if mounted then return state of carrier thread Can you move this comment line to before the notifyJvmtiDisableSuspend(true)? src/java.base/share/classes/java/lang/VirtualThread.java line 1043: > 1041: notifyJvmtiDisableSuspend(true); > 1042: try { > 1043: // include the carrier thread state and name when mounted This one too, can you move the comment to before the notifyJvmtiDisableSuspend. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427198296 PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427198673 From dholmes at openjdk.org Thu Dec 14 21:28:47 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 14 Dec 2023 21:28:47 GMT Subject: Integrated: 8322065: Initial nroff manpage generation for JDK 23 In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 05:46:01 GMT, David Holmes wrote: > Updated the version to 23-ea and year to 2024. > > This initial generation also picks up the unpublished changes from: > > - [JDK-8302233](https://bugs.openjdk.org/browse/JDK-8302233) (keytool & jarsigner) > - [JDK-8290702](https://bugs.openjdk.org/browse/JDK-8290702) (javadoc) (JDK 23 backport) > > Thanks This pull request has now been integrated. Changeset: 692be577 Author: David Holmes URL: https://git.openjdk.org/jdk/commit/692be577385844bf00a01ff10e390e014191569f Stats: 193 lines in 27 files changed: 36 ins; 51 del; 106 mod 8322065: Initial nroff manpage generation for JDK 23 Reviewed-by: alanb ------------- PR: https://git.openjdk.org/jdk/pull/17101 From sspitsyn at openjdk.org Thu Dec 14 22:37:42 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 22:37:42 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v5] In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 19:50:00 GMT, Alan Bateman wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: moved notifyJvmtiDisableSuspend(true) out of try-block > > src/java.base/share/classes/java/lang/VirtualThread.java line 1043: > >> 1041: notifyJvmtiDisableSuspend(true); >> 1042: try { >> 1043: // include the carrier thread state and name when mounted > > This one too, can you move the comment to before the notifyJvmtiDisableSuspend. Moved both comments out of try blocks. What about this one (it seems we would wont to do the same) ? : notifyJvmtiDisableSuspend(true); try { // unpark carrier thread when pinned synchronized (carrierThreadAccessLock()) { Thread carrier = carrierThread; if (carrier != null && ((s = state()) == PINNED || s == TIMED_PINNED)) { U.unpark(carrier); } } } finally { notifyJvmtiDisableSuspend(false); } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427373522 From sspitsyn at openjdk.org Thu Dec 14 22:57:53 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 22:57:53 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v6] In-Reply-To: References: Message-ID: > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: moved a couple of comments out of try blocks ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17011/files - new: https://git.openjdk.org/jdk/pull/17011/files/ad990422..917dc724 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=04-05 Stats: 6 lines in 1 file changed: 3 ins; 3 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From sspitsyn at openjdk.org Thu Dec 14 22:57:55 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 14 Dec 2023 22:57:55 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v5] In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 22:35:18 GMT, Serguei Spitsyn wrote: >> src/java.base/share/classes/java/lang/VirtualThread.java line 1043: >> >>> 1041: notifyJvmtiDisableSuspend(true); >>> 1042: try { >>> 1043: // include the carrier thread state and name when mounted >> >> This one too, can you move the comment to before the notifyJvmtiDisableSuspend. > > Moved both comments out of try blocks. > What about this one (it seems we would wont to do the same) ? : > > notifyJvmtiDisableSuspend(true); > try { > // unpark carrier thread when pinned > synchronized (carrierThreadAccessLock()) { > Thread carrier = carrierThread; > if (carrier != null && ((s = state()) == PINNED || s == TIMED_PINNED)) { > U.unpark(carrier); > } > } > } finally { > notifyJvmtiDisableSuspend(false); > } Moved 3 comments out of try blocks. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427386103 From dholmes at openjdk.org Fri Dec 15 06:13:38 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 15 Dec 2023 06:13:38 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 10:13:51 GMT, Jaikiran Pai wrote: > Can I please get a review of this change which proposes to improve the code in `get_user_name_slow` function, which is used to identify the target JVM owner's user name? This addresses https://bugs.openjdk.org/browse/JDK-8321971. > > As noted in that JBS issue, in its current form, the nested loop ends up iterating over the directory contents of `hsperfdata_xxx` directory and then for each iteration it checks if the name of the entry matches the pid. This iteration shouldn't be needed and instead one could look for a file named `` within that directory. > > No new test has been added, given the nature of this change. Existing tier1, tier2, tier3 and svc_tools tests pass with this change on Linux, Windows and macosx. Just some passing comments. I'm not familiar enough with the existing logic. src/hotspot/os/posix/perfMemory_posix.cpp line 434: > 432: // shared memory region for the given user name and vmid. > 433: // > 434: // the caller is expected to free the allocated memory. Nit: comments with punctuation (e.g. terminating period) should be written as sentences and start with a capital letter e.g. // either do this // Or this. Thanks src/hotspot/os/posix/perfMemory_posix.cpp line 606: > 604: } > 605: // skip over files that are not regular files. > 606: if (!S_ISREG(statbuf.st_mode)) { These appear to be able to be combined into a single if block. src/hotspot/os/posix/perfMemory_posix.cpp line 612: > 610: continue; > 611: } > 612: FREE_C_HEAP_ARRAY(char, filename); If you move this to immediately after lstat then you don't need it in the if-block ------------- PR Review: https://git.openjdk.org/jdk/pull/17104#pullrequestreview-1783152470 PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1427568288 PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1427589543 PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1427590464 From stuefe at openjdk.org Fri Dec 15 07:38:58 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 15 Dec 2023 07:38:58 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Tue, 12 Dec 2023 14:05:48 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > followed the proposals Is this libpath parsing code copied from the R3 kernel? If yes, pls make sure there are no licensing issues. src/hotspot/os/aix/os_aix.cpp line 206: > 204: constexpr int max_handletable = 1024; > 205: static int g_handletable_used = 0; > 206: static struct handletableentry g_handletable[max_handletable] = {{0, 0, 0, 0}}; I would move all that new and clearly delineated dlopen stuff into an own file, e.g. dlopen_aix.cpp or porting_aix.cpp (in porting_aix.cpp, we already have wrappers for other functions). os_aix.cpp is already massive. src/hotspot/os/aix/os_aix.cpp line 1129: > 1127: > 1128: // get the library search path burned in to the executable file during linking > 1129: // If the libpath cannot be retrieved return an empty path This is new. Is this complexity needed, if yes, why? Don't see a comment, may have missed it. src/hotspot/os/aix/os_aix.cpp line 1131: > 1129: // If the libpath cannot be retrieved return an empty path > 1130: static const char* rtv_linkedin_libpath() { > 1131: static char buffer[4096]; This coding has some issues: - a generic char buffer is not a good idea. Forces you to do casts all over the place, and introduces alignment issues with unaligned char buffer. Which I assume is the reason for all the separate memcpy-into-structures below. I would just read into the structures directly. - you need to check the return codes for fread to make sure you read the number of bytes expected, lest you work with uninitialized memory and maybe to handle sporadic EINTR. - I don't get all the separate "SZ" macros. They must be equal to sizeof(structure), right, otherwise you get buffer overruns or work with uninitialized memory? Proposal: add a local wrapper function like this: template static bool my_checked_fread(FILE* f, T* out) { // read sizeof(T) from f. // Check return code. // Return bool if sizeof(T) bytes were read. e.g. in a very trivial form: int bytesread = fread(out, sizeof(T), 1, f); return bytesread == sizeof(T); } and use it in your code like this: struct xcoff64 the_xcoff64; struct scn64 the_scn64; struct ldr64 the_ldr64; if (!my_checked_fread(f, &the_xcoff64)) { assert? } ... if (!my_checked_fread(f, &the_ldr64) { .. handle error } src/hotspot/os/aix/os_aix.cpp line 1132: > 1130: static const char* rtv_linkedin_libpath() { > 1131: static char buffer[4096]; > 1132: static const char* libpath = 0; If your intent is to return an empty buffer if there is no contained libpath, I would just: static const char* libpath = ""; then you can always just return libpath. src/hotspot/os/aix/os_aix.cpp line 1135: > 1133: > 1134: if (libpath) > 1135: return libpath; { } src/hotspot/os/aix/os_aix.cpp line 1137: > 1135: return libpath; > 1136: > 1137: char pgmpath[32+1]; Will overflow if pid_t is 64bit. Give it a larger size; after all, you are giving buffer 4K above, so you are not overly concerned with saving stack space. src/hotspot/os/aix/os_aix.cpp line 1146: > 1144: fread(buffer, 1, FILHSZ_64 + _AOUTHSZ_EXEC_64, f); > 1145: > 1146: if (((struct filehdr*)buffer)->f_magic == U802TOCMAGIC ) { as stated above, I don't think this section is needed. src/hotspot/os/aix/os_aix.cpp line 1170: > 1168: else if (((struct filehdr*)buffer)->f_magic == U64_TOCMAGIC ) { > 1169: // __XCOFF64__ > 1170: struct _S_(xcoffhdr) xcoff64; whats with the `_S_`? src/hotspot/os/aix/os_aix.cpp line 1174: > 1172: struct _S_(ldhdr) ldr64; > 1173: memcpy((char*)&xcoff64, buffer, FILHSZ_64 + _AOUTHSZ_EXEC_64); > 1174: int ldroffset = FILHSZ_64 + xcoff64.filehdr.f_opthdr + (xcoff64.aouthdr.o_snloader -1)*SCNHSZ_64; why the -1? I assume thats the section number? is it 1 based? how odd.. src/hotspot/os/aix/os_aix.cpp line 1187: > 1185: fread(buffer, 1, LDHDRSZ_64, f); > 1186: memcpy((char*)&ldr64, buffer, LDHDRSZ_64); > 1187: fseek (f, scn64.s_scnptr + ldr64.l_impoff, SEEK_SET); nit: please use consistent spacing according to hotspot rules. here, remove space. src/hotspot/os/aix/os_aix.cpp line 1191: > 1189: } > 1190: else > 1191: buffer[0] = 0; {} src/hotspot/os/aix/os_aix.cpp line 1234: > 1232: > 1233: stringStream Libpath; > 1234: if (env == nullptr) { Proposal for shorter version not needing string assembly: const char* paths [2] = { env, rtv_linkedin_libpath() }: for (int i = 0; i < 2; i ++) { const char* this_libpath = paths[i]; if (this_libpath == nullptr) { continue; } ... do the token thing... } } ------------- Changes requested by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16920#pullrequestreview-1783187856 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427593949 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427597791 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427606275 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427632243 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427594255 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427604761 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427610156 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427610650 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427622550 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427635888 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427633296 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427640624 From stuefe at openjdk.org Fri Dec 15 07:38:58 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 15 Dec 2023 07:38:58 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 06:22:39 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> followed the proposals > > src/hotspot/os/aix/os_aix.cpp line 1129: > >> 1127: >> 1128: // get the library search path burned in to the executable file during linking >> 1129: // If the libpath cannot be retrieved return an empty path > > This is new. Is this complexity needed, if yes, why? Don't see a comment, may have missed it. Also, why are we parsing xcoff32 headers in there? AIX OpenJDK will always be 64-bit. So, you can replace the whole xcoff32 section with assert( f_magic == U802TOCMAGIC, ..). The function becomes a lot simpler then. > src/hotspot/os/aix/os_aix.cpp line 1132: > >> 1130: static const char* rtv_linkedin_libpath() { >> 1131: static char buffer[4096]; >> 1132: static const char* libpath = 0; > > If your intent is to return an empty buffer if there is no contained libpath, I would just: > > > static const char* libpath = ""; > > then you can always just return libpath. But looking at the using code, returning NULL in case there is no contained libpath would be actually easier, see below. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427609926 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427639138 From alanb at openjdk.org Fri Dec 15 09:00:43 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 15 Dec 2023 09:00:43 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v6] In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 22:57:53 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: moved a couple of comments out of try blocks src/hotspot/share/prims/jvm.cpp line 4019: > 4017: return; > 4018: } > 4019: assert(thread->is_disable_suspend() != (bool)enter, "recursive disable suspend is not allowed"); This is an important assert, the message should probably say nested or unbalanced enter/exit not allowed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427719197 From alanb at openjdk.org Fri Dec 15 09:29:41 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 15 Dec 2023 09:29:41 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v6] In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 22:57:53 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: moved a couple of comments out of try blocks I think okay, I don't have any other comments. ------------- Marked as reviewed by alanb (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17011#pullrequestreview-1783445183 From jkern at openjdk.org Fri Dec 15 09:59:40 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 09:59:40 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 06:44:03 GMT, Thomas Stuefe wrote: >> src/hotspot/os/aix/os_aix.cpp line 1129: >> >>> 1127: >>> 1128: // get the library search path burned in to the executable file during linking >>> 1129: // If the libpath cannot be retrieved return an empty path >> >> This is new. Is this complexity needed, if yes, why? Don't see a comment, may have missed it. > > Also, why are we parsing xcoff32 headers in there? AIX OpenJDK will always be 64-bit. So, you can replace the whole xcoff32 section with assert( f_magic == U802TOCMAGIC, ..). The function becomes a lot simpler then. I found a leak in my previous implementation. It is more or less academical, but this solution is the complete one. I would prefer this complete solution even it is complex, because if dlopen follows a slightly different algorithm in resolving the library we surely get into trouble. If we omit the xcoff32 we have to ensure that no xcoff32 executable file comes into play. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427782107 From jkern at openjdk.org Fri Dec 15 10:21:44 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 10:21:44 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 06:15:15 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> followed the proposals > > src/hotspot/os/aix/os_aix.cpp line 206: > >> 204: constexpr int max_handletable = 1024; >> 205: static int g_handletable_used = 0; >> 206: static struct handletableentry g_handletable[max_handletable] = {{0, 0, 0, 0}}; > > I would move all that new and clearly delineated dlopen stuff into an own file, e.g. dlopen_aix.cpp or porting_aix.cpp (in porting_aix.cpp, we already have wrappers for other functions). os_aix.cpp is already massive. I moved the static variable declarations and the functions `Aix_dlopen(), search_file_in_LIBPATH(), rtv_linkedin_libpath()` and `os::pd_dll_unload()` to porting_aix.cpp. This links, but in my opinion `os::pd_dll_unload()` should reside in os_aix.cpp, because it is member of the os class. But there it will not compile anymore if the static variables are moved away. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427803856 From stuefe at openjdk.org Fri Dec 15 10:21:46 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 15 Dec 2023 10:21:46 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 09:57:19 GMT, Joachim Kern wrote: > If we omit the xcoff32 we have to ensure that no xcoff32 executable file comes into play. xcoff32 is for 32-bit binaries. The AIX port only exists for 64-bit, and there will never be a 32-bit AIX port, so there is no reason for handling 32-bit xcoff headers. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427803763 From stuefe at openjdk.org Fri Dec 15 10:29:40 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 15 Dec 2023 10:29:40 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 10:18:53 GMT, Joachim Kern wrote: >> src/hotspot/os/aix/os_aix.cpp line 206: >> >>> 204: constexpr int max_handletable = 1024; >>> 205: static int g_handletable_used = 0; >>> 206: static struct handletableentry g_handletable[max_handletable] = {{0, 0, 0, 0}}; >> >> I would move all that new and clearly delineated dlopen stuff into an own file, e.g. dlopen_aix.cpp or porting_aix.cpp (in porting_aix.cpp, we already have wrappers for other functions). os_aix.cpp is already massive. > > I moved the static variable declarations and the functions `Aix_dlopen(), search_file_in_LIBPATH(), rtv_linkedin_libpath()` and `os::pd_dll_unload()` to porting_aix.cpp. This links, but in my opinion `os::pd_dll_unload()` should reside in os_aix.cpp, because it is member of the os class. But there it will not compile anymore if the static variables are moved away. No, what I meant was to provide a "libc-like" equivalent for dlopen, similar to what we do with dladdr (see https://github.com/openjdk/jdk/blob/b7676822886eac21f61ff361a32928a966d8fe31/src/hotspot/os/aix/porting_aix.cpp#L306). But never mind; I am also fine with moving os::pd_dlopen into a different cpp file, e.g. "dlopen_aix.cpp". Just move it out of os_aix.cpp, since that is already massive and you add >300 lines of more code and more dependencies. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427812795 From sspitsyn at openjdk.org Fri Dec 15 10:49:56 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 15 Dec 2023 10:49:56 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v7] In-Reply-To: References: Message-ID: <4iSULgKTef_C2q4AJpEKB64tZh_QDIB77Ov2rwZ78nY=.39d3c708-4175-42b5-8eb9-58684e131ccf@github.com> > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: improve an assert message ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17011/files - new: https://git.openjdk.org/jdk/pull/17011/files/917dc724..6f8cdf06 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=05-06 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From sspitsyn at openjdk.org Fri Dec 15 10:50:01 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 15 Dec 2023 10:50:01 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v6] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 08:57:45 GMT, Alan Bateman wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: moved a couple of comments out of try blocks > > src/hotspot/share/prims/jvm.cpp line 4019: > >> 4017: return; >> 4018: } >> 4019: assert(thread->is_disable_suspend() != (bool)enter, "recursive disable suspend is not allowed"); > > This is an important assert, the message should probably say nested or unbalanced enter/exit not allowed. Thanks. Updated the assert message as suggested. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1427830694 From jkern at openjdk.org Fri Dec 15 11:24:44 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 11:24:44 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 06:15:56 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> followed the proposals > > src/hotspot/os/aix/os_aix.cpp line 1135: > >> 1133: >> 1134: if (libpath) >> 1135: return libpath; > > { } done > src/hotspot/os/aix/os_aix.cpp line 1137: > >> 1135: return libpath; >> 1136: >> 1137: char pgmpath[32+1]; > > Will overflow if pid_t is 64bit. Give it a larger size; after all, you are giving buffer 4K above, so you are not overly concerned with saving stack space. adopted. use buffer instead of pgmpath > src/hotspot/os/aix/os_aix.cpp line 1146: > >> 1144: fread(buffer, 1, FILHSZ_64 + _AOUTHSZ_EXEC_64, f); >> 1145: >> 1146: if (((struct filehdr*)buffer)->f_magic == U802TOCMAGIC ) { > > as stated above, I don't think this section is needed. Completely rewritten; Only xcoff64 handled > src/hotspot/os/aix/os_aix.cpp line 1170: > >> 1168: else if (((struct filehdr*)buffer)->f_magic == U64_TOCMAGIC ) { >> 1169: // __XCOFF64__ >> 1170: struct _S_(xcoffhdr) xcoff64; > > whats with the `_S_`? Not needed any more, because only xcoff64 handled ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427862523 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427862370 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427863562 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427864005 From jkern at openjdk.org Fri Dec 15 11:27:47 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 11:27:47 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 07:01:06 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> followed the proposals > > src/hotspot/os/aix/os_aix.cpp line 1174: > >> 1172: struct _S_(ldhdr) ldr64; >> 1173: memcpy((char*)&xcoff64, buffer, FILHSZ_64 + _AOUTHSZ_EXEC_64); >> 1174: int ldroffset = FILHSZ_64 + xcoff64.filehdr.f_opthdr + (xcoff64.aouthdr.o_snloader -1)*SCNHSZ_64; > > why the -1? I assume thats the section number? is it 1 based? how odd.. Yes, the section numbers are 1 based. e.g. Beginning of section 4 has an offset of 3 section sizes. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427866203 From jkern at openjdk.org Fri Dec 15 11:31:41 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 11:31:41 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 07:20:47 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> followed the proposals > > src/hotspot/os/aix/os_aix.cpp line 1187: > >> 1185: fread(buffer, 1, LDHDRSZ_64, f); >> 1186: memcpy((char*)&ldr64, buffer, LDHDRSZ_64); >> 1187: fseek (f, scn64.s_scnptr + ldr64.l_impoff, SEEK_SET); > > nit: please use consistent spacing according to hotspot rules. here, remove space. Do you mean the space `fseek (` ? Done. > src/hotspot/os/aix/os_aix.cpp line 1191: > >> 1189: } >> 1190: else >> 1191: buffer[0] = 0; > > {} Done, due to complete rewriting. s.o. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427869786 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427870433 From jkern at openjdk.org Fri Dec 15 11:39:41 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 11:39:41 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 07:27:14 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> followed the proposals > > src/hotspot/os/aix/os_aix.cpp line 1234: > >> 1232: >> 1233: stringStream Libpath; >> 1234: if (env == nullptr) { > > Proposal for shorter version not needing string assembly: > > const char* paths [2] = { env, rtv_linkedin_libpath() }: > for (int i = 0; i < 2; i ++) { > const char* this_libpath = paths[i]; > if (this_libpath == nullptr) { > continue; > } > ... do the token thing... > } > } Sorry, I did not clearly understand how this should work. The mystery must be in _... do the token thing ..._ ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1427877337 From jkern at openjdk.org Fri Dec 15 11:57:51 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 11:57:51 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: - trailing whitespace - Following most of Thomas proposals ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/b7676822..18d9d2b0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=03-04 Stats: 562 lines in 3 files changed: 272 ins; 290 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From jkern at openjdk.org Fri Dec 15 11:57:53 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 15 Dec 2023 11:57:53 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Tue, 12 Dec 2023 14:05:48 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > followed the proposals The libpath parsing code is from me, so no license problems. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1857762912 From amenkov at openjdk.org Fri Dec 15 19:48:44 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 15 Dec 2023 19:48:44 GMT Subject: RFR: JDK-8322062: com/sun/jdi/JdwpAllowTest.java does not performs negative testing with prefix length Message-ID: The fix updated JdwpAllowTest to test negative scenarios (debugger tries to attach from not allowed address). MaskTest ctor incorrectly calculates prefix length for negative tests and test main() calls `positiveTest` instead of `negativeTest` for `prefixLengthBad`. Also fixes test output: LingeredApp logs to System.out and the test logs to System.err, this makes harder log analysis. ------------- Commit messages: - Update JdwpAllowTest.java Changes: https://git.openjdk.org/jdk/pull/17131/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17131&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8322062 Stats: 26 lines in 1 file changed: 8 ins; 0 del; 18 mod Patch: https://git.openjdk.org/jdk/pull/17131.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17131/head:pull/17131 PR: https://git.openjdk.org/jdk/pull/17131 From cjplummer at openjdk.org Sat Dec 16 01:05:39 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 16 Dec 2023 01:05:39 GMT Subject: RFR: JDK-8322062: com/sun/jdi/JdwpAllowTest.java does not performs negative testing with prefix length In-Reply-To: References: Message-ID: <9_qg3Mwu03QGFPE0wT1BVS7mzj4BYqEtbpxDgtEPCiM=.a1149e62-5ec9-4581-89b4-b4e0c36cfcc1@github.com> On Fri, 15 Dec 2023 19:43:36 GMT, Alex Menkov wrote: > The fix updated JdwpAllowTest to test negative scenarios (debugger tries to attach from not allowed address). > > MaskTest ctor incorrectly calculates prefix length for negative tests and test main() calls `positiveTest` instead of `negativeTest` for `prefixLengthBad`. > > Also fixes test output: > LingeredApp logs to System.out and the test logs to System.err, this makes harder log analysis. Changes look good. I'd support a PR to change all uses of System.err in our tests to System.out. I'm not so sure there is ever a case where we don't want to see the errors in the same stream as regular log message. I'm tired of seeing System.err dumps at the end of the test output, and not knowing where it happened relative to the System.out output. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17131#pullrequestreview-1785091559 From cjplummer at openjdk.org Sat Dec 16 01:20:41 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 16 Dec 2023 01:20:41 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v2] In-Reply-To: <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> Message-ID: <8LPvhH3PtUVqRGTTOj05yDoPMATSyJLTL_h6tkk_lXs=.5f4e9703-9126-458b-ac10-f0b61984304b@github.com> On Wed, 6 Dec 2023 05:50:02 GMT, Denghui Dong wrote: >> Hi, >> >> Could I have a review of this patch? >> >> In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. >> >> This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. >> >> Best, >> Denghui > > Denghui Dong has updated the pull request incrementally with one additional commit since the last revision: > > change the location of test src/hotspot/share/runtime/globals.hpp line 557: > 555: "Limit the number of heap dumps triggered by " \ > 556: "HeapDumpBeforeFullGC or HeapDumpAfterFullGC " \ > 557: "(0 means no limit)" ) \ nit: should be no space before the last paren. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16976#discussion_r1428625595 From cjplummer at openjdk.org Sat Dec 16 01:20:42 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Sat, 16 Dec 2023 01:20:42 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v2] In-Reply-To: <8LPvhH3PtUVqRGTTOj05yDoPMATSyJLTL_h6tkk_lXs=.5f4e9703-9126-458b-ac10-f0b61984304b@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> <8LPvhH3PtUVqRGTTOj05yDoPMATSyJLTL_h6tkk_lXs=.5f4e9703-9126-458b-ac10-f0b61984304b@github.com> Message-ID: On Sat, 16 Dec 2023 01:09:16 GMT, Chris Plummer wrote: >> Denghui Dong has updated the pull request incrementally with one additional commit since the last revision: >> >> change the location of test > > src/hotspot/share/runtime/globals.hpp line 557: > >> 555: "Limit the number of heap dumps triggered by " \ >> 556: "HeapDumpBeforeFullGC or HeapDumpAfterFullGC " \ >> 557: "(0 means no limit)" ) \ > > nit: should be no space before the last paren. I was wondering if it is worth having HeapDumpBeforeFullGC and HeapDumpAfterFullGC also mention FullGCHeapDumpLimit. Just something simple like "Also see FullGCHeapDumpLimit." Not necessary, but if you think it would be useful then please add. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16976#discussion_r1428626556 From ddong at openjdk.org Sat Dec 16 04:36:57 2023 From: ddong at openjdk.org (Denghui Dong) Date: Sat, 16 Dec 2023 04:36:57 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v3] In-Reply-To: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> Message-ID: <8GGPQMjfU6YWa1i0yjk7SvrJ-lnZu6TxG8zPcbWN3jE=.1a4bb16e-dfc6-46ed-84e1-f2ed3d911699@github.com> > Hi, > > Could I have a review of this patch? > > In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. > > This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. > > Best, > Denghui Denghui Dong has updated the pull request incrementally with one additional commit since the last revision: refine description ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16976/files - new: https://git.openjdk.org/jdk/pull/16976/files/442b7f47..a0cae5c2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16976&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16976&range=01-02 Stats: 5 lines in 1 file changed: 2 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16976.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16976/head:pull/16976 PR: https://git.openjdk.org/jdk/pull/16976 From ddong at openjdk.org Sat Dec 16 04:36:58 2023 From: ddong at openjdk.org (Denghui Dong) Date: Sat, 16 Dec 2023 04:36:58 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v2] In-Reply-To: References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> <8LPvhH3PtUVqRGTTOj05yDoPMATSyJLTL_h6tkk_lXs=.5f4e9703-9126-458b-ac10-f0b61984304b@github.com> Message-ID: On Sat, 16 Dec 2023 01:11:44 GMT, Chris Plummer wrote: >> src/hotspot/share/runtime/globals.hpp line 557: >> >>> 555: "Limit the number of heap dumps triggered by " \ >>> 556: "HeapDumpBeforeFullGC or HeapDumpAfterFullGC " \ >>> 557: "(0 means no limit)" ) \ >> >> nit: should be no space before the last paren. > > I was wondering if it is worth having HeapDumpBeforeFullGC and HeapDumpAfterFullGC also mention FullGCHeapDumpLimit. Just something simple like "Also see FullGCHeapDumpLimit." Not necessary, but if you think it would be useful then please add. > nit: should be no space before the last paren. fixed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16976#discussion_r1428685031 From ddong at openjdk.org Sat Dec 16 04:36:59 2023 From: ddong at openjdk.org (Denghui Dong) Date: Sat, 16 Dec 2023 04:36:59 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v2] In-Reply-To: References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> <11q5GvzTPts9R6r7B1-KNh0me5AJwnSuqPP-J-LTuRc=.66380e47-6f34-4803-b522-9b22395466cb@github.com> <8LPvhH3PtUVqRGTTOj05yDoPMATSyJLTL_h6tkk_lXs=.5f4e9703-9126-458b-ac10-f0b61984304b@github.com> Message-ID: On Sat, 16 Dec 2023 04:33:48 GMT, Denghui Dong wrote: >> I was wondering if it is worth having HeapDumpBeforeFullGC and HeapDumpAfterFullGC also mention FullGCHeapDumpLimit. Just something simple like "Also see FullGCHeapDumpLimit." Not necessary, but if you think it would be useful then please add. > >> nit: should be no space before the last paren. > > fixed. > I was wondering if it is worth having HeapDumpBeforeFullGC and HeapDumpAfterFullGC also mention FullGCHeapDumpLimit. Just something simple like "Also see FullGCHeapDumpLimit." Not necessary, but if you think it would be useful then please add. Make sense. Added. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16976#discussion_r1428685126 From sspitsyn at openjdk.org Sat Dec 16 07:29:36 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 16 Dec 2023 07:29:36 GMT Subject: RFR: JDK-8322062: com/sun/jdi/JdwpAllowTest.java does not performs negative testing with prefix length In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 19:43:36 GMT, Alex Menkov wrote: > The fix updated JdwpAllowTest to test negative scenarios (debugger tries to attach from not allowed address). > > MaskTest ctor incorrectly calculates prefix length for negative tests and test main() calls `positiveTest` instead of `negativeTest` for `prefixLengthBad`. > > Also fixes test output: > LingeredApp logs to System.out and the test logs to System.err, this makes harder log analysis. Looks good. Nice comment about `bitToChange` values. ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17131#pullrequestreview-1785143379 From alanb at openjdk.org Sun Dec 17 07:38:05 2023 From: alanb at openjdk.org (Alan Bateman) Date: Sun, 17 Dec 2023 07:38:05 GMT Subject: RFR: 8320707: Virtual thread test updates Message-ID: A lot of test changes have accumulated in the loom repo, this includes both new tests and updates to existing tests. Some of these updates can be brought to the main line. This update brings over: - The existing tests for pinning use synchronized blocks. In preparation for changes to allow carrier thread be released when a virtual thread parks holding a monitor or blocks on monitorenter, these tests are changed to pin by having a native frame on the stack. This part includes test infrastructure to make it easy to add more tests that do operations while pinned. The tests still test what they were originally created to test of course. - The test for the JFR jdk.VirtualThreadPinned event is refactored to allow for additional cases where the event may be reported. - ThreadAPI is expanded to cover test for uncaught exception handling. - GetStackTraceWhenRunnable is refactored to not use a Selector, otherwise this test will be invalidated when blocking selection operations release the carrier. - StressStackOverflow is dialed down to run for 1m instead of 2mins. - The use of CountDownLatch in a number of tests that poll thread state has been dropped to keep the tests as simple as possible. ------------- Commit messages: - Initial commit Changes: https://git.openjdk.org/jdk/pull/17136/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17136&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320707 Stats: 572 lines in 12 files changed: 381 ins; 48 del; 143 mod Patch: https://git.openjdk.org/jdk/pull/17136.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17136/head:pull/17136 PR: https://git.openjdk.org/jdk/pull/17136 From liach at openjdk.org Sun Dec 17 19:22:06 2023 From: liach at openjdk.org (Chen Liang) Date: Sun, 17 Dec 2023 19:22:06 GMT Subject: RFR: 8294977: Convert test/jdk/java tests from ASM library to Classfile API [v10] In-Reply-To: References: Message-ID: > Summaries: > 1. A few recommendations about updating the constant API is made at https://mail.openjdk.org/pipermail/classfile-api-dev/2023-March/000233.html and I may update this patch shall the API changes be integrated before > 2. One ASM library-specific test, `LambdaAsm` is removed. Others have their code generation infrastructure upgraded from ASM to Classfile API. > 3. Most tests are included in tier1, but some are not: > In `:jdk_io`: (tier2, part 2) > > test/jdk/java/io/Serializable/records/SerialPersistentFieldsTest.java > test/jdk/java/io/Serializable/records/ProhibitedMethods.java > test/jdk/java/io/Serializable/records/BadCanonicalCtrTest.java > > In `:jdk_instrument`: (tier 3) > > test/jdk/java/lang/instrument/RetransformAgent.java > test/jdk/java/lang/instrument/NativeMethodPrefixAgent.java > test/jdk/java/lang/instrument/asmlib/Instrumentor.java > > > @asotona Would you mind reviewing? Chen Liang has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 17 commits: - Some tests don't work well with previews - Update tests to Class-File API - Merge branch 'master' into invoke-test-classfile - Classfile object update - Merge branch 'master' into invoke-test-classfile - Merge branch 'master' into invoke-test-classfile - Switch to ConstantDescs for and void constants - Merge AnnotationsTest, remove ModuleTargetAttribute call - Merge branch 'invoke-test-classfile' of https://github.com/liachmodded/jdk into invoke-test-classfile - Update test/jdk/java/lang/invoke/8022701/MHIllegalAccess.java Co-authored-by: Andrey Turbanov - ... and 7 more: https://git.openjdk.org/jdk/compare/34351b7a...7f15d2d7 ------------- Changes: https://git.openjdk.org/jdk/pull/13009/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=13009&range=09 Stats: 1832 lines in 31 files changed: 361 ins; 713 del; 758 mod Patch: https://git.openjdk.org/jdk/pull/13009.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13009/head:pull/13009 PR: https://git.openjdk.org/jdk/pull/13009 From lmesnik at openjdk.org Sun Dec 17 20:30:39 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Sun, 17 Dec 2023 20:30:39 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v7] In-Reply-To: <4iSULgKTef_C2q4AJpEKB64tZh_QDIB77Ov2rwZ78nY=.39d3c708-4175-42b5-8eb9-58684e131ccf@github.com> References: <4iSULgKTef_C2q4AJpEKB64tZh_QDIB77Ov2rwZ78nY=.39d3c708-4175-42b5-8eb9-58684e131ccf@github.com> Message-ID: On Fri, 15 Dec 2023 10:49:56 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: improve an assert message Marked as reviewed by lmesnik (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/17011#pullrequestreview-1785514646 From liach at openjdk.org Sun Dec 17 23:11:10 2023 From: liach at openjdk.org (Chen Liang) Date: Sun, 17 Dec 2023 23:11:10 GMT Subject: RFR: 8294977: Convert test/jdk/java tests from ASM library to Classfile API [v11] In-Reply-To: References: Message-ID: > Summaries: > 1. A few recommendations about updating the constant API is made at https://mail.openjdk.org/pipermail/classfile-api-dev/2023-March/000233.html and I may update this patch shall the API changes be integrated before > 2. One ASM library-specific test, `LambdaAsm` is removed. Others have their code generation infrastructure upgraded from ASM to Classfile API. > 3. Most tests are included in tier1, but some are not: > In `:jdk_io`: (tier2, part 2) > > test/jdk/java/io/Serializable/records/SerialPersistentFieldsTest.java > test/jdk/java/io/Serializable/records/ProhibitedMethods.java > test/jdk/java/io/Serializable/records/BadCanonicalCtrTest.java > > In `:jdk_instrument`: (tier 3) > > test/jdk/java/lang/instrument/RetransformAgent.java > test/jdk/java/lang/instrument/NativeMethodPrefixAgent.java > test/jdk/java/lang/instrument/asmlib/Instrumentor.java > > > @asotona Would you mind reviewing? Chen Liang has updated the pull request incrementally with one additional commit since the last revision: Fix the 2 failing tests and add notes ------------- Changes: - all: https://git.openjdk.org/jdk/pull/13009/files - new: https://git.openjdk.org/jdk/pull/13009/files/7f15d2d7..eb4851fd Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=13009&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=13009&range=09-10 Stats: 30 lines in 2 files changed: 4 ins; 19 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/13009.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/13009/head:pull/13009 PR: https://git.openjdk.org/jdk/pull/13009 From dholmes at openjdk.org Mon Dec 18 00:38:39 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 18 Dec 2023 00:38:39 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v3] In-Reply-To: <8GGPQMjfU6YWa1i0yjk7SvrJ-lnZu6TxG8zPcbWN3jE=.1a4bb16e-dfc6-46ed-84e1-f2ed3d911699@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> <8GGPQMjfU6YWa1i0yjk7SvrJ-lnZu6TxG8zPcbWN3jE=.1a4bb16e-dfc6-46ed-84e1-f2ed3d911699@github.com> Message-ID: On Sat, 16 Dec 2023 04:36:57 GMT, Denghui Dong wrote: >> Hi, >> >> Could I have a review of this patch? >> >> In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. >> >> This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. >> >> Best, >> Denghui > > Denghui Dong has updated the pull request incrementally with one additional commit since the last revision: > > refine description Update looks good. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16976#pullrequestreview-1785576985 From liach at openjdk.org Mon Dec 18 00:39:39 2023 From: liach at openjdk.org (Chen Liang) Date: Mon, 18 Dec 2023 00:39:39 GMT Subject: RFR: 8294977: Convert test/jdk/java tests from ASM library to Classfile API [v11] In-Reply-To: References: Message-ID: On Sun, 17 Dec 2023 23:11:10 GMT, Chen Liang wrote: >> Summaries: >> 1. A few recommendations about updating the constant API is made at https://mail.openjdk.org/pipermail/classfile-api-dev/2023-March/000233.html and I may update this patch shall the API changes be integrated before >> 2. One ASM library-specific test, `LambdaAsm` is removed. Others have their code generation infrastructure upgraded from ASM to Classfile API. >> 3. Most tests are included in tier1, but some are not: >> In `:jdk_io`: (tier2, part 2) >> >> test/jdk/java/io/Serializable/records/SerialPersistentFieldsTest.java >> test/jdk/java/io/Serializable/records/ProhibitedMethods.java >> test/jdk/java/io/Serializable/records/BadCanonicalCtrTest.java >> >> In `:jdk_instrument`: (tier 3) >> >> test/jdk/java/lang/instrument/RetransformAgent.java >> test/jdk/java/lang/instrument/NativeMethodPrefixAgent.java >> test/jdk/java/lang/instrument/asmlib/Instrumentor.java >> >> >> @asotona Would you mind reviewing? > > Chen Liang has updated the pull request incrementally with one additional commit since the last revision: > > Fix the 2 failing tests and add notes I have migrated the tests to Preview Class-File API and fixed a few issues related to preview status that broke the tests. All tests now pass. ------------- PR Comment: https://git.openjdk.org/jdk/pull/13009#issuecomment-1859351576 From epeter at openjdk.org Mon Dec 18 07:20:05 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Mon, 18 Dec 2023 07:20:05 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity Message-ID: Before this patch, we always initialized the GrowableArray up to its `capacity`, and not just up to `length`. This is problematic for a few reasons: - It is not expected. `std::vector` also only initializes the elements up to its size, and not to capacity. - It requires a default-constructor for the element type. And the default-constructor is then used to fill in the elements between length and capacity. If the elements do any allocation themselves, then this is a waste of resources. - The implementation also required the copy-assignment-operator for the element type. This is a lesser restriction. But the copy-assignment-operator was used in cases like `append` (where placement copy-construct would be expected), and not just in true assignment kinds of cases like `at_put`. For this reason, I reworked a lot of the methods to ensure that only the "slots" up to `length` are ever initialized, and the space between `length` and `capacity` is always garbage. ----- Also, before this patch, one can CHeap allocate both with `GrowableArray` and `GrowableArrayCHeap`. This is unnecessary. It required more complex verification in `GrowableArray` to deal with all cases. And `GrowableArrayCHeap` is already explicitly a smaller object, and should hence be preferred. Hence I changed all CHeap allocating cases of `GrowableArray` to `GrowableArrayCHeap`. This also allows for a clear separation: - `GrowableArray` only deals with arena / resource area allocation. These are arrays that are regularly abandoned at the end of their use, rather than deleted or even cleared. - `GrowableArrayCHeap` only deals with CHeap allocated memory. We expect that the destructor for it is called eventually, either when it goes out of scope or when `delete` is explicitly called. We expect that the elements could be allocating resources internally, and hence rely on the destructors for the elements being called, which may free up those internally allocated resources. Therefore, we now only allow `GrowableArrayCHeap` to have element types with non-trivial destructors, but `GrowableArray` checks that element types do not have non-trivial destructors (since it is common practice to just abandon arena / resource area allocated arrays, rather than calling the destructor or clearing the array, which also destructs all elements). This more clearly separates the two worlds: clean-up your own mess (CHeap) vs abandon your mess (arena / resource area). ----- I also completely refactored and improved the tests for `GrowableArray(CHeap)`: https://github.com/openjdk/jdk/blob/e5eb36010355b444a719da6bdcd8c5de3145b961/test/hotspot/gtest/utilities/test_growableArray.cpp#L29-L60 The main improvement is that now **all** `GrowableArray` methods are tested, and that we test it with many different element types (including such without default-constructor or copy-assign-constructor). And we also check that the number of live elements is correct, which we can compute as `live = constructred - destructed`. This is especially valuable because I refactored the use of constructors/destructors heavily, to do the change from initializing up to `length` instead of `capacity`. ---- **Note on move-semantics** Since move semantics is currently not allowed by the style guide, we have to "simulate" a move by placement new with copy-constructor, and then destruct the old element. See this example when we need to grow an array, and move the elements from the old data to the new data: https://github.com/openjdk/jdk/blob/e5eb36010355b444a719da6bdcd8c5de3145b961/src/hotspot/share/utilities/growableArray.hpp#L530-L563 Of course this is nothing new with my change here. I just want to record that we are doing it this way, and in fact have to do so without any move-semantics. The problem with this: If you use nested `GrowableArray>`, then the inner arrays get copy-constructed around when we re-allocate the outer array. We now have two choices for how `GrowableArray` could copy (currently it is a shallow-copy): - shallow-copy: works well for reallocating outer arrays, since the inner array is now just shallow-copied to the new data, and the destructor for the old inner arrays does nothing. Shallow-copy of course would not work for `GrowableArrayCHeap`, since it would deallocate the data of the old inner arrays, and the new inner array would still have a pointer to that deallocated data (bad!). But shallow-copy is generally dangerous, since the copy-constructor may be used in non-obvious cases: ResourceMark rm; GrowableArray> outer; outer.at_grow(100); // default argument calls default constructor, and (shallow) copy-constructs it so all elements outer.at(0).at_put_grow(0, 42); outer.at(1).at_put_grow(0, 666); // inner array at position 1 has reference to same data as inner array 0 ASSERT_EQ(outer.at(0).at(0), 42); // fails, we see 666 instead of 42 ASSERT_EQ(outer.at(1).at(0), 666); - deep-copy: This ensures correctness, we never have two arrays with the same underlying data. But that also means that when we re-allocate an outer array, we now (deep) copy-construct all new elements from the old elements. And that seems quite wasteful, both for the memory and the time needed to deep-copy everything over. Neither of these options is good. This is exactly why the move-semantics were introduced in `C++11`. We should therefore discuss the introduction of move-semantics, and weigh it against the additional complexity that it introduces. ----- Testing: tier1-3 and stress testing. Running. ------------- Commit messages: - whitespaces - fix issue with clang, need explicit copy constructor if assignment operator deleted - Merge branch 'master' into JDK-8319115 - fix small comment - fix some comments - test shallow assign / copy - 2 negative tests: nested ra, insert_before to itself - refactor and test swap - find_sorted and insert_sorted - test 2 versions of sort - ... and 44 more: https://git.openjdk.org/jdk/compare/b31454e3...1ed037fd Changes: https://git.openjdk.org/jdk/pull/16918/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16918&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319115 Stats: 3439 lines in 127 files changed: 2150 ins; 493 del; 796 mod Patch: https://git.openjdk.org/jdk/pull/16918.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16918/head:pull/16918 PR: https://git.openjdk.org/jdk/pull/16918 From dholmes at openjdk.org Mon Dec 18 07:51:43 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 18 Dec 2023 07:51:43 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 07:56:04 GMT, Emanuel Peter wrote: > Before this patch, we always initialized the GrowableArray up to its `capacity`, and not just up to `length`. This is problematic for a few reasons: > > - It is not expected. `std::vector` also only initializes the elements up to its size, and not to capacity. > - It requires a default-constructor for the element type. And the default-constructor is then used to fill in the elements between length and capacity. If the elements do any allocation themselves, then this is a waste of resources. > - The implementation also required the copy-assignment-operator for the element type. This is a lesser restriction. But the copy-assignment-operator was used in cases like `append` (where placement copy-construct would be expected), and not just in true assignment kinds of cases like `at_put`. > > For this reason, I reworked a lot of the methods to ensure that only the "slots" up to `length` are ever initialized, and the space between `length` and `capacity` is always garbage. > > ----- > > Also, before this patch, one can CHeap allocate both with `GrowableArray` and `GrowableArrayCHeap`. This is unnecessary. It required more complex verification in `GrowableArray` to deal with all cases. And `GrowableArrayCHeap` is already explicitly a smaller object, and should hence be preferred. Hence I changed all CHeap allocating cases of `GrowableArray` to `GrowableArrayCHeap`. This also allows for a clear separation: > - `GrowableArray` only deals with arena / resource area allocation. These are arrays that are regularly abandoned at the end of their use, rather than deleted or even cleared. > - `GrowableArrayCHeap` only deals with CHeap allocated memory. We expect that the destructor for it is called eventually, either when it goes out of scope or when `delete` is explicitly called. We expect that the elements could be allocating resources internally, and hence rely on the destructors for the elements being called, which may free up those internally allocated resources. > > Therefore, we now only allow `GrowableArrayCHeap` to have element types with non-trivial destructors, but `GrowableArray` checks that element types do not have non-trivial destructors (since it is common practice to just abandon arena / resource area allocated arrays, rather than calling the destructor or clearing the array, which also destructs all elements). This more clearly separates the two worlds: clean-up your own mess (CHeap) vs abandon your mess (arena / resource area). > > ----- > > I also completely refactored and improved ... @eme64 Is it feasible to split this up to solve each of the problems you identify in stages? There is also overlap here with JDK-8319709 IIUC. Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16918#issuecomment-1859711325 From epeter at openjdk.org Mon Dec 18 09:14:45 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Mon, 18 Dec 2023 09:14:45 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 07:49:04 GMT, David Holmes wrote: >> Before this patch, we always initialized the GrowableArray up to its `capacity`, and not just up to `length`. This is problematic for a few reasons: >> >> - It is not expected. `std::vector` also only initializes the elements up to its size, and not to capacity. >> - It requires a default-constructor for the element type. And the default-constructor is then used to fill in the elements between length and capacity. If the elements do any allocation themselves, then this is a waste of resources. >> - The implementation also required the copy-assignment-operator for the element type. This is a lesser restriction. But the copy-assignment-operator was used in cases like `append` (where placement copy-construct would be expected), and not just in true assignment kinds of cases like `at_put`. >> >> For this reason, I reworked a lot of the methods to ensure that only the "slots" up to `length` are ever initialized, and the space between `length` and `capacity` is always garbage. >> >> ----- >> >> Also, before this patch, one can CHeap allocate both with `GrowableArray` and `GrowableArrayCHeap`. This is unnecessary. It required more complex verification in `GrowableArray` to deal with all cases. And `GrowableArrayCHeap` is already explicitly a smaller object, and should hence be preferred. Hence I changed all CHeap allocating cases of `GrowableArray` to `GrowableArrayCHeap`. This also allows for a clear separation: >> - `GrowableArray` only deals with arena / resource area allocation. These are arrays that are regularly abandoned at the end of their use, rather than deleted or even cleared. >> - `GrowableArrayCHeap` only deals with CHeap allocated memory. We expect that the destructor for it is called eventually, either when it goes out of scope or when `delete` is explicitly called. We expect that the elements could be allocating resources internally, and hence rely on the destructors for the elements being called, which may free up those internally allocated resources. >> >> Therefore, we now only allow `GrowableArrayCHeap` to have element types with non-trivial destructors, but `GrowableArray` checks that element types do not have non-trivial destructors (since it is common practice to just abandon arena / resource area allocated arrays, rather than calling the destructor or clearing the array, which also destructs all elements). This more clearly separates the two worlds: clean-up your own mess (CHeap) vs abandon your mess (arena / resource area). >> >> ----- >> >> I al... > > @eme64 Is it feasible to split this up to solve each of the problems you identify in stages? There is also overlap here with JDK-8319709 IIUC. Thanks. @dholmes-ora These are the "parts": 1. initialize up to capacity vs length 2. update the test to verify this (complete refactoring) 3. remove cheap use of GrowableArray -> use GrowableArrayCHeap instead The first 2 items are inseparable, I cannot make substantial changes to many GrowableArray methods without there even being tests for them. And the tests would not pass before the changes for item 1, since the tests also verify what elements of the array are initialized. So adding the tests first would not be very feasible. The 3rd item could maybe be split, and be done before the rest. Though it would also require lots of changes to the test, which then I would have to completely refactor with items 1+2 anyway. And the items are related conceptually, that is why I would felt ok pushing them together. It is all about when (item 1) and what kinds of (item 3) constructors / destructors are called for the elements of the arrays, and verifying that thoroughly (item 2). Hence: feasible probably, but lots of work overhead. Do you think it is worth it? I am aware of JDK-8319709, and in conversation with @jdksjolen - I basically took this item over for him ;) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16918#issuecomment-1859859939 From jsjolen at openjdk.org Mon Dec 18 10:13:40 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Mon, 18 Dec 2023 10:13:40 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 10:13:51 GMT, Jaikiran Pai wrote: > Can I please get a review of this change which proposes to improve the code in `get_user_name_slow` function, which is used to identify the target JVM owner's user name? This addresses https://bugs.openjdk.org/browse/JDK-8321971. > > As noted in that JBS issue, in its current form, the nested loop ends up iterating over the directory contents of `hsperfdata_xxx` directory and then for each iteration it checks if the name of the entry matches the pid. This iteration shouldn't be needed and instead one could look for a file named `` within that directory. > > No new test has been added, given the nature of this change. Existing tier1, tier2, tier3 and svc_tools tests pass with this change on Linux, Windows and macosx. Hi, A small review on the usage of the string functions. Have you considered using `stringStream` and passing references to an instance of it instead? src/hotspot/os/posix/perfMemory_posix.cpp line 447: > 445: > 446: return name; > 447: } This drops the `snprintf` return value which indicates if an error has occurred, can this be remediated? src/hotspot/os/posix/perfMemory_posix.cpp line 617: > 615: > 616: if (statbuf.st_ctime > oldest_ctime) { > 617: char* user = strchr(dentry->d_name, '_') + 1; Invalid pointer if `strchr` returns null. ------------- PR Review: https://git.openjdk.org/jdk/pull/17104#pullrequestreview-1786373167 PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1429831911 PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1429840030 From jpai at openjdk.org Mon Dec 18 10:24:06 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Mon, 18 Dec 2023 10:24:06 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow [v2] In-Reply-To: References: Message-ID: > Can I please get a review of this change which proposes to improve the code in `get_user_name_slow` function, which is used to identify the target JVM owner's user name? This addresses https://bugs.openjdk.org/browse/JDK-8321971. > > As noted in that JBS issue, in its current form, the nested loop ends up iterating over the directory contents of `hsperfdata_xxx` directory and then for each iteration it checks if the name of the entry matches the pid. This iteration shouldn't be needed and instead one could look for a file named `` within that directory. > > No new test has been added, given the nature of this change. Existing tier1, tier2, tier3 and svc_tools tests pass with this change on Linux, Windows and macosx. Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: - David's review comments - reduce if blocks and release the array outside if block - David's review comment - punctuation ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17104/files - new: https://git.openjdk.org/jdk/pull/17104/files/4d428363..cfd50d79 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17104&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17104&range=00-01 Stats: 25 lines in 2 files changed: 0 ins; 14 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/17104.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17104/head:pull/17104 PR: https://git.openjdk.org/jdk/pull/17104 From jpai at openjdk.org Mon Dec 18 10:24:08 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Mon, 18 Dec 2023 10:24:08 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow [v2] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 06:08:33 GMT, David Holmes wrote: >> Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: >> >> - David's review comments - reduce if blocks and release the array outside if block >> - David's review comment - punctuation > > src/hotspot/os/posix/perfMemory_posix.cpp line 612: > >> 610: continue; >> 611: } >> 612: FREE_C_HEAP_ARRAY(char, filename); > > If you move this to immediately after lstat then you don't need it in the if-block Thank you David for these inputs; they were useful. I've updated the PR to use those suggestions. Locally tests continue to pass. I've triggered CI testing with these new changes. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1429865689 From jpai at openjdk.org Mon Dec 18 10:26:39 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Mon, 18 Dec 2023 10:26:39 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow [v2] In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 09:55:30 GMT, Johan Sj?len wrote: >> Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: >> >> - David's review comments - reduce if blocks and release the array outside if block >> - David's review comment - punctuation > > src/hotspot/os/posix/perfMemory_posix.cpp line 447: > >> 445: >> 446: return name; >> 447: } > > This drops the `snprintf` return value which indicates if an error has occurred, can this be remediated? Hello Johan, I'll take a look at the code if there are other places where this is happening. Since this is existing code (not added by this PR), if there are too many other unrelated places where this is happening then I'll create a separate issue to address it. Otherwise, I'll do it in this PR itself. Thank you for catching this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1429868521 From stuefe at openjdk.org Mon Dec 18 10:29:50 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 18 Dec 2023 10:29:50 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: References: Message-ID: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> On Fri, 15 Dec 2023 11:57:51 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: > > - trailing whitespace > - Following most of Thomas proposals I like this, this is good. Small nits remain. src/hotspot/os/aix/os_aix.cpp line 30: > 28: #pragma alloca > 29: > 30: please remove whitespace change src/hotspot/os/aix/os_aix.cpp line 193: > 191: // local variables > 192: > 193: please remove whitespace change src/hotspot/os/aix/os_aix.cpp line 1113: > 1111: } > 1112: > 1113: please remove whitespace change src/hotspot/os/aix/porting_aix.cpp line 934: > 932: struct scnhdr the_scn; > 933: struct ldhdr the_ldr; > 934: size_t sz = FILHSZ + _AOUTHSZ_EXEC; please rename to xcoffsz, and make constexpr: `constexpr size_t xcoffsz = ...` src/hotspot/os/aix/porting_aix.cpp line 990: > 988: if (env == nullptr) { > 989: // no LIBPATH, try with LD_LIBRARY_PATH > 990: env = getenv("LD_LIBRARY_PATH"); Is LD_LIBRARY_PATH a thing on AIX? I thought it is only used on non-AIX. src/hotspot/os/aix/porting_aix.cpp line 1005: > 1003: // LIBPATH or LD_LIBRARY_PATH and second with burned in libpath. > 1004: // No check against current working directory > 1005: Libpath.print("%s:%s", env, rtv_linkedin_libpath()); Are you sure libpath env var has precedence over the baked-in libpath? src/hotspot/os/aix/porting_aix.cpp line 1097: > 1095: } > 1096: > 1097: pthread_mutex_lock(&g_handletable_mutex); You can make your life a lot easier by defining an RAII object at the start of the file: struct TableLocker { TableLocker() { pthread_mutex_lock(&g_handletable_mutex); } ~TableLocker() { pthread_mutex_unlock(&g_handletable_mutex); } }; and just place this at the beginning of your two functions TableLocker lock: ... no need to manually unlock then, with the danger of missing a return. src/hotspot/os/aix/porting_aix.cpp line 1101: > 1099: for (i = 0; i < g_handletable_used; i++) { > 1100: if (g_handletable[i].handle == libhandle) { > 1101: // handle found, decrease refcount `assert(refcount > 0, "Sanity"))` src/hotspot/os/aix/porting_aix.cpp line 1143: > 1141: // entry of the array to the place of the entry we want to remove and overwrite it > 1142: if (i < g_handletable_used) { > 1143: g_handletable[i] = g_handletable[g_handletable_used]; To be super careful, I would zero out at least the handle of the moved item like this: `g_handletable[g_handletable_used].handle = nullptr` ------------- Changes requested by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16920#pullrequestreview-1786400492 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429870755 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429870833 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429870885 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429849403 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429858465 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429859923 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429868182 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429863665 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429870057 From stuefe at openjdk.org Mon Dec 18 10:29:52 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 18 Dec 2023 10:29:52 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:06:34 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: >> >> - trailing whitespace >> - Following most of Thomas proposals > > src/hotspot/os/aix/porting_aix.cpp line 934: > >> 932: struct scnhdr the_scn; >> 933: struct ldhdr the_ldr; >> 934: size_t sz = FILHSZ + _AOUTHSZ_EXEC; > > please rename to xcoffsz, and make constexpr: `constexpr size_t xcoffsz = ...` Also, can you please add STATIC_ASSERT(sizeof(the_xcoff) == xcoffsz); STATIC_ASSERT(sizeof(the_scn) == SCNHSZ); STATIC_ASSERT(sizeof(the_ldr) == LDHDRSZ); ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429853292 From jkern at openjdk.org Mon Dec 18 10:38:42 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 18 Dec 2023 10:38:42 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:14:42 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: >> >> - trailing whitespace >> - Following most of Thomas proposals > > src/hotspot/os/aix/porting_aix.cpp line 990: > >> 988: if (env == nullptr) { >> 989: // no LIBPATH, try with LD_LIBRARY_PATH >> 990: env = getenv("LD_LIBRARY_PATH"); > > Is LD_LIBRARY_PATH a thing on AIX? I thought it is only used on non-AIX. Yes it is, It's the fallback if LIBPATH is not defined ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429891049 From jkern at openjdk.org Mon Dec 18 11:02:45 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 18 Dec 2023 11:02:45 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:19:24 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: >> >> - trailing whitespace >> - Following most of Thomas proposals > > src/hotspot/os/aix/porting_aix.cpp line 1101: > >> 1099: for (i = 0; i < g_handletable_used; i++) { >> 1100: if (g_handletable[i].handle == libhandle) { >> 1101: // handle found, decrease refcount > > `assert(refcount > 0, "Sanity"))` Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429931831 From stuefe at openjdk.org Mon Dec 18 10:57:46 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 18 Dec 2023 10:57:46 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:35:48 GMT, Joachim Kern wrote: >> src/hotspot/os/aix/porting_aix.cpp line 990: >> >>> 988: if (env == nullptr) { >>> 989: // no LIBPATH, try with LD_LIBRARY_PATH >>> 990: env = getenv("LD_LIBRARY_PATH"); >> >> Is LD_LIBRARY_PATH a thing on AIX? I thought it is only used on non-AIX. > > Yes it is, It's the fallback if LIBPATH is not defined In that case there may be errors in other places, since so far we assumed its either one or the other, but not both. Example: https://github.com/openjdk/jdk/blob/a247d0c74bea50f11d24fb5f3576947c6901e567/src/java.base/unix/native/libjli/java_md.c#L43C1-L47 Maybe you need to take a look here, in case LD_LIBRARYPATH needs to be handled in addition to LIBPATH? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429917901 From jkern at openjdk.org Mon Dec 18 10:57:48 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 18 Dec 2023 10:57:48 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:16:07 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: >> >> - trailing whitespace >> - Following most of Thomas proposals > > src/hotspot/os/aix/porting_aix.cpp line 1005: > >> 1003: // LIBPATH or LD_LIBRARY_PATH and second with burned in libpath. >> 1004: // No check against current working directory >> 1005: Libpath.print("%s:%s", env, rtv_linkedin_libpath()); > > Are you sure libpath env var has precedence over the baked-in libpath? Yes, that was the outcome of my experiments, although the IBM docu says the oposite: _"Specifies that the library path used at process exec time should be prepended to any library path specified in the load call (either as an argument or environment variable). It is recommended that this flag be specified in all calls to the load subroutine."_ My experiment showed: LIBPATH=libpath; baked-in-libpath=baked-in-libpath; mylib.so is in both paths. After dlopen(mylib.so) a map call shows the library was loaded from libpath. Then I remove the LIBPATH envvar and repeat. Now after dlopen(mylib.so) a map call shows the library was loaded from baked-in-libpath. So the LIBPATH envvar has precedence over the baked-in-libpath. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429919510 From jkern at openjdk.org Mon Dec 18 11:19:47 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 18 Dec 2023 11:19:47 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:25:50 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: >> >> - trailing whitespace >> - Following most of Thomas proposals > > src/hotspot/os/aix/os_aix.cpp line 30: > >> 28: #pragma alloca >> 29: >> 30: > > please remove whitespace change Done > src/hotspot/os/aix/os_aix.cpp line 193: > >> 191: // local variables >> 192: >> 193: > > please remove whitespace change Done > src/hotspot/os/aix/porting_aix.cpp line 1097: > >> 1095: } >> 1096: >> 1097: pthread_mutex_lock(&g_handletable_mutex); > > You can make your life a lot easier by defining an RAII object at the start of the file: > > struct TableLocker { > TableLocker() { pthread_mutex_lock(&g_handletable_mutex); } > ~TableLocker() { pthread_mutex_unlock(&g_handletable_mutex); } > }; > > and just place this at the beginning of your two functions > > TableLocker lock: > ... > > > no need to manually unlock then, with the danger of missing a return. Great, thank you. This was one of the things I thought about, but was not sure, because I did not fully understood the MutexLocker class and the difference between Monitor and Mutex. > src/hotspot/os/aix/porting_aix.cpp line 1143: > >> 1141: // entry of the array to the place of the entry we want to remove and overwrite it >> 1142: if (i < g_handletable_used) { >> 1143: g_handletable[i] = g_handletable[g_handletable_used]; > > To be super careful, I would zero out at least the handle of the moved item like this: > `g_handletable[g_handletable_used].handle = nullptr` Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429950832 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429951237 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429946043 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1429947950 From jkern at openjdk.org Mon Dec 18 11:30:59 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 18 Dec 2023 11:30:59 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v6] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: Followed Thomas proposals ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/18d9d2b0..978ed33c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=04-05 Stats: 79 lines in 2 files changed: 19 ins; 21 del; 39 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From jkern at openjdk.org Mon Dec 18 12:53:43 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 18 Dec 2023 12:53:43 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:54:31 GMT, Thomas Stuefe wrote: >> Yes it is, It's the fallback if LIBPATH is not defined > > In that case there may be errors in other places, since so far we assumed its either one or the other, but not both. Example: > > https://github.com/openjdk/jdk/blob/a247d0c74bea50f11d24fb5f3576947c6901e567/src/java.base/unix/native/libjli/java_md.c#L43C1-L47 > > Maybe you need to take a look here, in case LD_LIBRARYPATH needs to be handled in addition to LIBPATH? Yes, it's one or the other. If LIBPATH envvar exists (even empty string), LD_LIBRARY_PATH is ignored. So, no problems. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1430106335 From stuefe at openjdk.org Mon Dec 18 13:36:44 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 18 Dec 2023 13:36:44 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v6] In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 11:30:59 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > Followed Thomas proposals Well done. Releasing the mutex before asserting is not necessary; we don't pull the handle table lock as part of error reporting. ------------- Marked as reviewed by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16920#pullrequestreview-1786905733 From duke at openjdk.org Mon Dec 18 15:23:54 2023 From: duke at openjdk.org (Yi-Fan Tsai) Date: Mon, 18 Dec 2023 15:23:54 GMT Subject: Integrated: 8314029: Add file name parameter to Compiler.perfmap In-Reply-To: References: Message-ID: On Thu, 21 Sep 2023 20:43:56 GMT, Yi-Fan Tsai wrote: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. > > `jcmd PID help Compiler.perfmap` shows the following usage. > > > Compiler.perfmap > Write map file for Linux perf tool. > > Impact: Low > > Syntax : Compiler.perfmap [] > > Arguments: > filename : [optional] Name of the map file (STRING, no default value) > > > The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) > > > Compiler.perfmap [arguments] (Linux only) > Write map file for Linux perf tool. > > Impact: Low > > arguments: > > ? filename: (Optional) Name of the map file (STRING, no default value) > > If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then > the default filename will be /tmp/perf-12345.map. This pull request has now been integrated. Changeset: a5122d7f Author: Yi-Fan Tsai Committer: Paul Hohensee URL: https://git.openjdk.org/jdk/commit/a5122d7f6c36a4c98ea4bea7a7c8081e2a4dadca Stats: 56 lines in 6 files changed: 37 ins; 3 del; 16 mod 8314029: Add file name parameter to Compiler.perfmap Reviewed-by: cjplummer, eastigeevich ------------- PR: https://git.openjdk.org/jdk/pull/15871 From mdoerr at openjdk.org Mon Dec 18 16:22:46 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 18 Dec 2023 16:22:46 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v6] In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 11:30:59 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > Followed Thomas proposals I like getting rid of `#ifdef AIX` in shared code. The change is not simple, but looks basically good to me. I'll take a closer look when I find more time. I have some coding style requests. Please also see https://wiki.openjdk.org/display/HotSpot/StyleGuide (especially section Whitespace). src/hotspot/os/aix/porting_aix.cpp line 964: > 962: > 963: return libpath; > 964: Empty line could get removed. src/hotspot/os/aix/porting_aix.cpp line 985: > 983: if (strchr(path2, '/')) { > 984: stringStream combined; > 985: if (*path2 == '/' || *path2 == '.') We usually use `{` and `}` unless for extremely simple substatements on the same line src/hotspot/share/runtime/os.hpp line 1068: > 1066: static bool pd_dll_unload(void* libhandle, char* ebuf, int ebuflen); > 1067: > 1068: Please remove empty lines. ------------- PR Review: https://git.openjdk.org/jdk/pull/16920#pullrequestreview-1787236876 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1430362306 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1430366154 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1430367434 From mdoerr at openjdk.org Mon Dec 18 16:22:50 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 18 Dec 2023 16:22:50 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 10:25:57 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with two additional commits since the last revision: >> >> - trailing whitespace >> - Following most of Thomas proposals > > src/hotspot/os/aix/os_aix.cpp line 1113: > >> 1111: } >> 1112: >> 1113: > > please remove whitespace change +1 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1430352685 From stuefe at openjdk.org Mon Dec 18 16:31:43 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 18 Dec 2023 16:31:43 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v5] In-Reply-To: References: <6kfjEICoOee2rRHe1OsqY2xHvvd_Cab0ZQCpp41VfNk=.0fb32b43-56ed-4d81-ab1d-8d40dfb58e4f@github.com> Message-ID: On Mon, 18 Dec 2023 11:12:23 GMT, Joachim Kern wrote: >> src/hotspot/os/aix/porting_aix.cpp line 1097: >> >>> 1095: } >>> 1096: >>> 1097: pthread_mutex_lock(&g_handletable_mutex); >> >> You can make your life a lot easier by defining an RAII object at the start of the file: >> >> struct TableLocker { >> TableLocker() { pthread_mutex_lock(&g_handletable_mutex); } >> ~TableLocker() { pthread_mutex_unlock(&g_handletable_mutex); } >> }; >> >> and just place this at the beginning of your two functions >> >> TableLocker lock: >> ... >> >> >> no need to manually unlock then, with the danger of missing a return. > > Great, thank you. This was one of the things I thought about, but was not sure, because I did not fully understood the MutexLocker class and the difference between Monitor and Mutex. In hindsight, pthread mutex is the better choice anyway: it avoids dependencies to the VM lifecycle (VM mutexes are only available after VM initialization, so we could not call dlopen() before that). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1430380082 From sspitsyn at openjdk.org Mon Dec 18 16:35:40 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 18 Dec 2023 16:35:40 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v7] In-Reply-To: <4iSULgKTef_C2q4AJpEKB64tZh_QDIB77Ov2rwZ78nY=.39d3c708-4175-42b5-8eb9-58684e131ccf@github.com> References: <4iSULgKTef_C2q4AJpEKB64tZh_QDIB77Ov2rwZ78nY=.39d3c708-4175-42b5-8eb9-58684e131ccf@github.com> Message-ID: On Fri, 15 Dec 2023 10:49:56 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: improve an assert message Alan and Leonid, thank you for review! Will push after the final mach5 testing is completed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17011#issuecomment-1860980377 From jkern at openjdk.org Mon Dec 18 16:57:55 2023 From: jkern at openjdk.org (Joachim Kern) Date: Mon, 18 Dec 2023 16:57:55 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v7] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: cosmetic changes ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/978ed33c..f79c89da Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=05-06 Stats: 7 lines in 3 files changed: 1 ins; 4 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From cjplummer at openjdk.org Mon Dec 18 17:07:59 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 18 Dec 2023 17:07:59 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 22:41:56 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) >> >> >> Compiler.perfmap [arguments] (Linux only) >> Write map file for Linux perf tool. >> >> Impact: Low >> >> arguments: >> >> ? filename: (Optional) Name of the map file (STRING, no default value) >> >> If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then >> the default filename will be /tmp/perf-12345.map. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright of PerfMapTest @phohensee Although this PR had 2 reviews, there was still some unresolved discussion. It should not have been pushed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1861074850 From sspitsyn at openjdk.org Mon Dec 18 17:09:59 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 18 Dec 2023 17:09:59 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v8] In-Reply-To: References: Message-ID: > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits: - Merge - review: improve an assert message - review: moved a couple of comments out of try blocks - review: moved notifyJvmtiDisableSuspend(true) out of try-block - review: 1) replace CriticalLock with DisableSuspend; 2) minor tweaks - review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods - Resolved merge conflict in VirtualThread.java - added @summary to new test SuspendWithInterruptLock.java - add new test SuspendWithInterruptLock.java - 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable ------------- Changes: https://git.openjdk.org/jdk/pull/17011/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17011&range=07 Stats: 229 lines in 15 files changed: 196 ins; 0 del; 33 mod Patch: https://git.openjdk.org/jdk/pull/17011.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17011/head:pull/17011 PR: https://git.openjdk.org/jdk/pull/17011 From cjplummer at openjdk.org Mon Dec 18 17:49:39 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 18 Dec 2023 17:49:39 GMT Subject: RFR: 8321404: Limit the number of heap dumps triggered by HeapDumpBeforeFullGC/AfterFullGC [v3] In-Reply-To: <8GGPQMjfU6YWa1i0yjk7SvrJ-lnZu6TxG8zPcbWN3jE=.1a4bb16e-dfc6-46ed-84e1-f2ed3d911699@github.com> References: <0q_yL6Q90R3L0R2-m94w1cCdbkOwapo0hLn_x_QAIVc=.f7a3b73f-9015-485f-9e9c-b4585ca84dd9@github.com> <8GGPQMjfU6YWa1i0yjk7SvrJ-lnZu6TxG8zPcbWN3jE=.1a4bb16e-dfc6-46ed-84e1-f2ed3d911699@github.com> Message-ID: On Sat, 16 Dec 2023 04:36:57 GMT, Denghui Dong wrote: >> Hi, >> >> Could I have a review of this patch? >> >> In the current implementation, HeapDumpBeforeFullGC/AfterFullGC will generate dumps for every FGC, increasing the risk of disk full. >> >> This patch introduces a new option 'FullGCHeapDumpLimit' to limit the number of dumps triggered by HeapDumpBeforeFullGC/AfterFullGC to enhance production-friendliness. >> >> Best, >> Denghui > > Denghui Dong has updated the pull request incrementally with one additional commit since the last revision: > > refine description Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16976#pullrequestreview-1787426650 From amenkov at openjdk.org Mon Dec 18 18:13:47 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Mon, 18 Dec 2023 18:13:47 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads Message-ID: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> HeapDumper dumps virtual threads in 2 places: - dumping platform threads (mounted virtual threads are dumped as separate thread object); - dumping heap objects when the object is `java.lang.VirtualThread`. In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb ------------- Commit messages: - jcheck - JDK-8322237 Changes: https://git.openjdk.org/jdk/pull/17134/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17134&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8322237 Stats: 25 lines in 2 files changed: 23 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/17134.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17134/head:pull/17134 PR: https://git.openjdk.org/jdk/pull/17134 From phh at openjdk.org Mon Dec 18 18:56:54 2023 From: phh at openjdk.org (Paul Hohensee) Date: Mon, 18 Dec 2023 18:56:54 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: <5jNzyVdKTTdgaMGoKhHaqoRBnLHkXRys3gCIRvjHIdE=.bf6c28df-ebfe-4486-a146-dbfa55cffd33@github.com> On Mon, 11 Dec 2023 22:41:56 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) >> >> >> Compiler.perfmap [arguments] (Linux only) >> Write map file for Linux perf tool. >> >> Impact: Low >> >> arguments: >> >> ? filename: (Optional) Name of the map file (STRING, no default value) >> >> If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then >> the default filename will be /tmp/perf-12345.map. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright of PerfMapTest Shall I revert it? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1861334362 From cjplummer at openjdk.org Mon Dec 18 20:33:54 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 18 Dec 2023 20:33:54 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 22:41:56 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) >> >> >> Compiler.perfmap [arguments] (Linux only) >> Write map file for Linux perf tool. >> >> Impact: Low >> >> arguments: >> >> ? filename: (Optional) Name of the map file (STRING, no default value) >> >> If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then >> the default filename will be /tmp/perf-12345.map. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright of PerfMapTest If we do settle on some additional changes, I think probably a follow-up CR would be cleaner than a BACKOUT and REDO. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1861563261 From cjplummer at openjdk.org Mon Dec 18 20:40:39 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Mon, 18 Dec 2023 20:40:39 GMT Subject: RFR: 8320707: Virtual thread test updates In-Reply-To: References: Message-ID: On Sat, 16 Dec 2023 17:25:20 GMT, Alan Bateman wrote: > A lot of test changes have accumulated in the loom repo, this includes both new tests and updates to existing tests. Some of these updates can be brought to the main line. This update brings over: > > - The existing tests for pinning use synchronized blocks. In preparation for changes to allow carrier thread be released when a virtual thread parks holding a monitor or blocks on monitorenter, these tests are changed to pin by having a native frame on the stack. This part includes test infrastructure to make it easy to add more tests that do operations while pinned. The tests still test what they were originally created to test of course. > > - The test for the JFR jdk.VirtualThreadPinned event is refactored to allow for additional cases where the event may be reported. > > - ThreadAPI is expanded to cover test for uncaught exception handling. > > - GetStackTraceWhenRunnable is refactored to not use a Selector, otherwise this test will be invalidated when blocking selection operations release the carrier. > > - StressStackOverflow is dialed down to run for 1m instead of 2mins. > > - The use of CountDownLatch in a number of tests that poll thread state has been dropped to keep the tests as simple as possible. Can you explain your motivation for using AtomicBoolean with a polling loop rather than CountDownLatch(1)? I'm working on a test where I just added a CountDownLatch(1) and am wondering if I should do the same, but I'm not sure if there is something about these tests that is motivating the change. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17136#issuecomment-1861578806 From amenkov at openjdk.org Mon Dec 18 21:16:42 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Mon, 18 Dec 2023 21:16:42 GMT Subject: Integrated: JDK-8322062: com/sun/jdi/JdwpAllowTest.java does not performs negative testing with prefix length In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 19:43:36 GMT, Alex Menkov wrote: > The fix updated JdwpAllowTest to test negative scenarios (debugger tries to attach from not allowed address). > > MaskTest ctor incorrectly calculates prefix length for negative tests and test main() calls `positiveTest` instead of `negativeTest` for `prefixLengthBad`. > > Also fixes test output: > LingeredApp logs to System.out and the test logs to System.err, this makes harder log analysis. This pull request has now been integrated. Changeset: 459957f3 Author: Alex Menkov URL: https://git.openjdk.org/jdk/commit/459957f30a6e0fe40636dd72faa3f0d86151c94f Stats: 26 lines in 1 file changed: 8 ins; 0 del; 18 mod 8322062: com/sun/jdi/JdwpAllowTest.java does not performs negative testing with prefix length Reviewed-by: cjplummer, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/17131 From phh at openjdk.org Mon Dec 18 21:27:50 2023 From: phh at openjdk.org (Paul Hohensee) Date: Mon, 18 Dec 2023 21:27:50 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v8] In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 22:41:56 GMT, Yi-Fan Tsai wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an optional argument for specifying a file name. >> >> `jcmd PID help Compiler.perfmap` shows the following usage. >> >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [] >> >> Arguments: >> filename : [optional] Name of the map file (STRING, no default value) >> >> >> The following section of man page is also updated. (`man -l src/jdk.jcmd/share/man/jcmd.1`) >> >> >> Compiler.perfmap [arguments] (Linux only) >> Write map file for Linux perf tool. >> >> Impact: Low >> >> arguments: >> >> ? filename: (Optional) Name of the map file (STRING, no default value) >> >> If filename is not specified, a default file name is chosen using the pid of the target JVM process. For example, if the pid is 12345, then >> the default filename will be /tmp/perf-12345.map. > > Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright of PerfMapTest Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1861689670 From kbarrett at openjdk.org Mon Dec 18 22:50:44 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Mon, 18 Dec 2023 22:50:44 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 07:49:04 GMT, David Holmes wrote: >> Before this patch, we always initialized the GrowableArray up to its `capacity`, and not just up to `length`. This is problematic for a few reasons: >> >> - It is not expected. `std::vector` also only initializes the elements up to its size, and not to capacity. >> - It requires a default-constructor for the element type. And the default-constructor is then used to fill in the elements between length and capacity. If the elements do any allocation themselves, then this is a waste of resources. >> - The implementation also required the copy-assignment-operator for the element type. This is a lesser restriction. But the copy-assignment-operator was used in cases like `append` (where placement copy-construct would be expected), and not just in true assignment kinds of cases like `at_put`. >> >> For this reason, I reworked a lot of the methods to ensure that only the "slots" up to `length` are ever initialized, and the space between `length` and `capacity` is always garbage. >> >> ----- >> >> Also, before this patch, one can CHeap allocate both with `GrowableArray` and `GrowableArrayCHeap`. This is unnecessary. It required more complex verification in `GrowableArray` to deal with all cases. And `GrowableArrayCHeap` is already explicitly a smaller object, and should hence be preferred. Hence I changed all CHeap allocating cases of `GrowableArray` to `GrowableArrayCHeap`. This also allows for a clear separation: >> - `GrowableArray` only deals with arena / resource area allocation. These are arrays that are regularly abandoned at the end of their use, rather than deleted or even cleared. >> - `GrowableArrayCHeap` only deals with CHeap allocated memory. We expect that the destructor for it is called eventually, either when it goes out of scope or when `delete` is explicitly called. We expect that the elements could be allocating resources internally, and hence rely on the destructors for the elements being called, which may free up those internally allocated resources. >> >> Therefore, we now only allow `GrowableArrayCHeap` to have element types with non-trivial destructors, but `GrowableArray` checks that element types do not have non-trivial destructors (since it is common practice to just abandon arena / resource area allocated arrays, rather than calling the destructor or clearing the array, which also destructs all elements). This more clearly separates the two worlds: clean-up your own mess (CHeap) vs abandon your mess (arena / resource area). >> >> ----- >> >> I al... > > @eme64 Is it feasible to split this up to solve each of the problems you identify in stages? There is also overlap here with JDK-8319709 IIUC. Thanks. > @dholmes-ora These are the "parts": > > 1. initialize up to capacity vs length > > 2. update the test to verify this (complete refactoring) > > 3. remove cheap use of GrowableArray -> use GrowableArrayCHeap instead > > > The first 2 items are inseparable, I cannot make substantial changes to many GrowableArray methods without there even being tests for them. And the tests would not pass before the changes for item 1, since the tests also verify what elements of the array are initialized. So adding the tests first would not be very feasible. > > The 3rd item could maybe be split, and be done before the rest. Though it would also require lots of changes to the test, which then I would have to completely refactor with items 1+2 anyway. > > And the items are related conceptually, that is why I would felt ok pushing them together. It is all about when (item 1) and what kinds of (item 3) constructors / destructors are called for the elements of the arrays, and verifying that thoroughly (item 2). > > Hence: feasible probably, but lots of work overhead. Do you think it is worth it? I too would prefer that it be split up. It's very easy to miss important details in amongst all the mostly relatively simple renamings. That is, I think 3 should be separate from the other changes. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16918#issuecomment-1861812983 From dholmes at openjdk.org Tue Dec 19 01:19:42 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 19 Dec 2023 01:19:42 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads In-Reply-To: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Sat, 16 Dec 2023 02:15:16 GMT, Alex Menkov wrote: > HeapDumper dumps virtual threads in 2 places: > - dumping platform threads (mounted virtual threads are dumped as separate thread object); > - dumping heap objects when the object is `java.lang.VirtualThread`. > > In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) > Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. > > Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb src/hotspot/share/services/heapDumper.cpp line 1647: > 1645: static bool is_vthread_mounted(oop vt) { > 1646: return JvmtiEnvBase::get_JavaThread_or_null(vt) != nullptr; > 1647: } It doesn't seem appropriate to couple this to the JVMTI code (can this code be present if JVMTI is not part of the build?). Doesn't the VT state give you a good enough approximation of whether it is mounted i.e. RUNNABLE? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1430801839 From kbarrett at openjdk.org Tue Dec 19 02:08:54 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Tue, 19 Dec 2023 02:08:54 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 07:56:04 GMT, Emanuel Peter wrote: > Before this patch, we always initialized the GrowableArray up to its `capacity`, and not just up to `length`. This is problematic for a few reasons: > > - It is not expected. `std::vector` also only initializes the elements up to its size, and not to capacity. > - It requires a default-constructor for the element type. And the default-constructor is then used to fill in the elements between length and capacity. If the elements do any allocation themselves, then this is a waste of resources. > - The implementation also required the copy-assignment-operator for the element type. This is a lesser restriction. But the copy-assignment-operator was used in cases like `append` (where placement copy-construct would be expected), and not just in true assignment kinds of cases like `at_put`. > > For this reason, I reworked a lot of the methods to ensure that only the "slots" up to `length` are ever initialized, and the space between `length` and `capacity` is always garbage. > > ----- > > Also, before this patch, one can CHeap allocate both with `GrowableArray` and `GrowableArrayCHeap`. This is unnecessary. It required more complex verification in `GrowableArray` to deal with all cases. And `GrowableArrayCHeap` is already explicitly a smaller object, and should hence be preferred. Hence I changed all CHeap allocating cases of `GrowableArray` to `GrowableArrayCHeap`. This also allows for a clear separation: > - `GrowableArray` only deals with arena / resource area allocation. These are arrays that are regularly abandoned at the end of their use, rather than deleted or even cleared. > - `GrowableArrayCHeap` only deals with CHeap allocated memory. We expect that the destructor for it is called eventually, either when it goes out of scope or when `delete` is explicitly called. We expect that the elements could be allocating resources internally, and hence rely on the destructors for the elements being called, which may free up those internally allocated resources. > > Therefore, we now only allow `GrowableArrayCHeap` to have element types with non-trivial destructors, but `GrowableArray` checks that element types do not have non-trivial destructors (since it is common practice to just abandon arena / resource area allocated arrays, rather than calling the destructor or clearing the array, which also destructs all elements). This more clearly separates the two worlds: clean-up your own mess (CHeap) vs abandon your mess (arena / resource area). > > ----- > > I also completely refactored and improved ... That's it for today. I'll continue looking at this tomorrow. src/hotspot/share/utilities/bitMap.hpp line 191: > 189: verify_size(size_in_bits); > 190: } > 191: ~BitMap() {} This change is incorrect. This destructor is intentionally declared protected, to prevent slicing through it. It would be reasonable to change it to have a `= default` definition though, rather than the empty body definition it currently has. Note that BitMap copying has the same shallow-copying problems as GrowableArray. src/hotspot/share/utilities/growableArray.hpp line 87: > 85: } > 86: > 87: ~GrowableArrayBase() {} Another incorrect removal of an intentionally protected destructor. src/hotspot/share/utilities/growableArray.hpp line 124: > 122: GrowableArrayBase(capacity, initial_len), _data(data) {} > 123: > 124: ~GrowableArrayView() {} Another incorrect removal of an intentionally protected destructor. src/hotspot/share/utilities/growableArray.hpp line 294: > 292: void remove_range(int start, int end) { > 293: assert(0 <= start, "illegal start index %d", start); > 294: assert(start < end && end <= _len, "erase called with invalid range (%d, %d) for length %d", start, end, _len); pre-existing: I think start == end should be permitted. There's no reason to forbid an empty range, and there are algorithms that are simpler if empty ranges are permitted. src/hotspot/share/utilities/growableArray.hpp line 319: > 317: ::new ((void*)&this->_data[index]) E(_data[_len]); > 318: // Destruct last element > 319: this->_data[_len].~E(); Must not do the copy/destruct if index designated the last element. src/hotspot/share/utilities/growableArray.hpp line 327: > 325: // sort by fixed-stride sub arrays: > 326: void sort(int f(E*, E*), int stride) { > 327: qsort(_data, length() / stride, sizeof(E) * stride, (_sort_Fn)f); pre-existing: Use of qsort presumes E is trivially copyable/assignable. Use QuickSort::sort instead. src/hotspot/share/utilities/growableArray.hpp line 398: > 396: } > 397: > 398: ~GrowableArrayWithAllocator() {} Another incorrect removal of an intentionally protected destructor. src/hotspot/share/utilities/growableArray.hpp line 414: > 412: // Assignment would be wrong, as it assumes the destination > 413: // was already initialized. > 414: ::new ((void*)&this->_data[idx]) E(elem); I don't think the cast to void* is needed, and just adds clutter. There are many more of these. ------------- Changes requested by kbarrett (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16918#pullrequestreview-1787825840 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430721171 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430728218 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430728372 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430760537 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430759359 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430762966 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430734644 PR Review Comment: https://git.openjdk.org/jdk/pull/16918#discussion_r1430747894 From amenkov at openjdk.org Tue Dec 19 04:12:37 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 19 Dec 2023 04:12:37 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Tue, 19 Dec 2023 01:17:10 GMT, David Holmes wrote: >> HeapDumper dumps virtual threads in 2 places: >> - dumping platform threads (mounted virtual threads are dumped as separate thread object); >> - dumping heap objects when the object is `java.lang.VirtualThread`. >> >> In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) >> Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. >> >> Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb > > src/hotspot/share/services/heapDumper.cpp line 1647: > >> 1645: static bool is_vthread_mounted(oop vt) { >> 1646: return JvmtiEnvBase::get_JavaThread_or_null(vt) != nullptr; >> 1647: } > > It doesn't seem appropriate to couple this to the JVMTI code (can this code be present if JVMTI is not part of the build?). Doesn't the VT state give you a good enough approximation of whether it is mounted i.e. RUNNABLE? Good point. I'll remove dependency on JVMTI. I don't think approximation would be good here (comparing state to RUNNABLE/PINNED/TIMED_PINNED or comparing carrierThread with null). It's racy and we have a chance to not dump unmounted vthread or dump mounted vthread twice. Maybe `is_vthread_mounted` should check if the virtual thread continuation has non-empty chunk. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1430893424 From dholmes at openjdk.org Tue Dec 19 05:19:38 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 19 Dec 2023 05:19:38 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Tue, 19 Dec 2023 04:09:47 GMT, Alex Menkov wrote: >> src/hotspot/share/services/heapDumper.cpp line 1647: >> >>> 1645: static bool is_vthread_mounted(oop vt) { >>> 1646: return JvmtiEnvBase::get_JavaThread_or_null(vt) != nullptr; >>> 1647: } >> >> It doesn't seem appropriate to couple this to the JVMTI code (can this code be present if JVMTI is not part of the build?). Doesn't the VT state give you a good enough approximation of whether it is mounted i.e. RUNNABLE? > > Good point. I'll remove dependency on JVMTI. > I don't think approximation would be good here (comparing state to RUNNABLE/PINNED/TIMED_PINNED or comparing carrierThread with null). > It's racy and we have a chance to not dump unmounted vthread or dump mounted vthread twice. > Maybe `is_vthread_mounted` should check if the virtual thread continuation has non-empty chunk. If that is racy then any solution is going to be racy. I assumed this was all happening at a global safepoint, otherwise threads could be mounting/unmounting at any time. I said "approximation" only because I'm unsure exactly when the thread state gets updated in the mounting/unmounting process. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1430924428 From dholmes at openjdk.org Tue Dec 19 05:35:43 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 19 Dec 2023 05:35:43 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 07:56:04 GMT, Emanuel Peter wrote: > Before this patch, we always initialized the GrowableArray up to its `capacity`, and not just up to `length`. This is problematic for a few reasons: > > - It is not expected. `std::vector` also only initializes the elements up to its size, and not to capacity. > - It requires a default-constructor for the element type. And the default-constructor is then used to fill in the elements between length and capacity. If the elements do any allocation themselves, then this is a waste of resources. > - The implementation also required the copy-assignment-operator for the element type. This is a lesser restriction. But the copy-assignment-operator was used in cases like `append` (where placement copy-construct would be expected), and not just in true assignment kinds of cases like `at_put`. > > For this reason, I reworked a lot of the methods to ensure that only the "slots" up to `length` are ever initialized, and the space between `length` and `capacity` is always garbage. > > ----- > > Also, before this patch, one can CHeap allocate both with `GrowableArray` and `GrowableArrayCHeap`. This is unnecessary. It required more complex verification in `GrowableArray` to deal with all cases. And `GrowableArrayCHeap` is already explicitly a smaller object, and should hence be preferred. Hence I changed all CHeap allocating cases of `GrowableArray` to `GrowableArrayCHeap`. This also allows for a clear separation: > - `GrowableArray` only deals with arena / resource area allocation. These are arrays that are regularly abandoned at the end of their use, rather than deleted or even cleared. > - `GrowableArrayCHeap` only deals with CHeap allocated memory. We expect that the destructor for it is called eventually, either when it goes out of scope or when `delete` is explicitly called. We expect that the elements could be allocating resources internally, and hence rely on the destructors for the elements being called, which may free up those internally allocated resources. > > Therefore, we now only allow `GrowableArrayCHeap` to have element types with non-trivial destructors, but `GrowableArray` checks that element types do not have non-trivial destructors (since it is common practice to just abandon arena / resource area allocated arrays, rather than calling the destructor or clearing the array, which also destructs all elements). This more clearly separates the two worlds: clean-up your own mess (CHeap) vs abandon your mess (arena / resource area). > > ----- > > I also completely refactored and improved ... Splitting out part 3 would have been preferable IMO. The CHeap changes are unrelated to the capacity issue and should have their own JBS issue. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16918#issuecomment-1862149245 From epeter at openjdk.org Tue Dec 19 06:46:47 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Tue, 19 Dec 2023 06:46:47 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 22:48:18 GMT, Kim Barrett wrote: >> @eme64 Is it feasible to split this up to solve each of the problems you identify in stages? There is also overlap here with JDK-8319709 IIUC. Thanks. > >> @dholmes-ora These are the "parts": >> >> 1. initialize up to capacity vs length >> >> 2. update the test to verify this (complete refactoring) >> >> 3. remove cheap use of GrowableArray -> use GrowableArrayCHeap instead >> >> >> The first 2 items are inseparable, I cannot make substantial changes to many GrowableArray methods without there even being tests for them. And the tests would not pass before the changes for item 1, since the tests also verify what elements of the array are initialized. So adding the tests first would not be very feasible. >> >> The 3rd item could maybe be split, and be done before the rest. Though it would also require lots of changes to the test, which then I would have to completely refactor with items 1+2 anyway. >> >> And the items are related conceptually, that is why I would felt ok pushing them together. It is all about when (item 1) and what kinds of (item 3) constructors / destructors are called for the elements of the arrays, and verifying that thoroughly (item 2). >> >> Hence: feasible probably, but lots of work overhead. Do you think it is worth it? > > I too would prefer that it be split up. It's very easy to miss important details in amongst all the mostly relatively > simple renamings. That is, I think 3 should be separate from the other changes. @kimbarrett @dholmes-ora I will try to split out the GrowableArray cheap -> GrowableArrayCHeap changes. And thanks for the feedback you already gave, Kim! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16918#issuecomment-1862209037 From alanb at openjdk.org Tue Dec 19 09:18:39 2023 From: alanb at openjdk.org (Alan Bateman) Date: Tue, 19 Dec 2023 09:18:39 GMT Subject: RFR: 8320707: Virtual thread test updates In-Reply-To: References: Message-ID: <-J_DKfk6g-qoYrPZcs-mdUUAG_wnsMzle2aZQhdXps4=.53d1ae20-503b-4b01-94c8-a10421e5405d@github.com> On Mon, 18 Dec 2023 20:37:29 GMT, Chris Plummer wrote: > I'm working on a test where I just added a CountDownLatch(1) and am wondering if I should do the same, but I'm not sure if there is something about these tests that is motivating the change. CountDownLatch is great for many tests. It's not as powerful as a Phaser of course but good enough and usually easy to understand quickly what is going on. However, for tests that are testing thread state then you often want to have as few dependencies as possible. In the case of CountDownlatch, the await method may park. The countDown method may have to unpark waiters for for virtual threads it means potentially parking (to queue a task) when unparking, so temporary transitions that JVMTI has to be concerned with. The other thing is keeping tests simple/consistent, it can be hard to maintain tests where one test method coordinates with one approach, another test method does something different. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17136#issuecomment-1862390382 From sroy at openjdk.org Tue Dec 19 12:40:51 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 12:40:51 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Fri, 15 Dec 2023 11:51:43 GMT, Joachim Kern wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> followed the proposals > > The libpath parsing code is from me, so no license problems. Hi @JoKern65 Is this good to integrate now ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1862684040 From jkern at openjdk.org Tue Dec 19 12:44:52 2023 From: jkern at openjdk.org (Joachim Kern) Date: Tue, 19 Dec 2023 12:44:52 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Tue, 19 Dec 2023 12:37:33 GMT, Suchismith Roy wrote: >> The libpath parsing code is from me, so no license problems. > > Hi @JoKern65 Is this good to integrate now ? Hi @suchismith1993, I'm waiting for a second review. Complex hotspot changes should be reviewed twice. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1862690708 From stuefe at openjdk.org Tue Dec 19 12:49:42 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 19 Dec 2023 12:49:42 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v6] In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 13:33:46 GMT, Thomas Stuefe wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> Followed Thomas proposals > > Well done. > > Releasing the mutex before asserting is not necessary; we don't pull the handle table lock as part of error reporting. > @tstuefe Sorry to tag you. Can you review the code. Once this code goes in I can push in my changes. We are targeting the fix for January. > Hi @JoKern65 Is this good to integrate now ? @suchismith1993 Please don't put pressure on patch authors and developers. There is zero reason why this patch should be rushed. > Hi @suchismith1993, I'm waiting for a second review. Complex hotspot changes should be reviewed twice. Not only that, hotspot changes *need* to be reviewed by at least two reviewers. That is not optional. See OpenJDK bylaws. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1862695052 From stuefe at openjdk.org Tue Dec 19 12:54:43 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 19 Dec 2023 12:54:43 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Tue, 19 Dec 2023 12:37:33 GMT, Suchismith Roy wrote: >> The libpath parsing code is from me, so no license problems. > > Hi @JoKern65 Is this good to integrate now ? @suchismith1993 > Once this code goes in I can push in my changes. We are targeting the fix for January. If you talk about https://github.com/openjdk/jdk/pull/16604, no, you cannot push that even if Joachim finishes his work. Your patch has not even a single review, is quite controversial, and none of the issues the reviewers have mentioned are addressed. This needs a lot more discussion time. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1862704694 From sroy at openjdk.org Tue Dec 19 13:36:51 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 13:36:51 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Tue, 19 Dec 2023 12:37:33 GMT, Suchismith Roy wrote: >> The libpath parsing code is from me, so no license problems. > > Hi @JoKern65 Is this good to integrate now ? > @suchismith1993 > > > Once this code goes in I can push in my changes. We are targeting the fix for January. > > If you talk about #16604, no, you cannot push that even if Joachim finishes his work. > > Your patch has not even a single review, is quite controversial, and none of the issues the reviewers have mentioned are addressed. This needs a lot more discussion time. I have the patch ready based on the changes in this patch, as I take the diff and apply. But I cannot push since it will end up adding the entire file. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1862768974 From sroy at openjdk.org Tue Dec 19 13:43:51 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 13:43:51 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v4] In-Reply-To: References: Message-ID: On Tue, 19 Dec 2023 12:52:23 GMT, Thomas Stuefe wrote: >> Hi @JoKern65 Is this good to integrate now ? > > @suchismith1993 > >> Once this code goes in I can push in my changes. We are targeting the fix for January. > > If you talk about https://github.com/openjdk/jdk/pull/16604, no, you cannot push that even if Joachim finishes his work. > > Your patch has not even a single review, is quite controversial, and none of the issues the reviewers have mentioned are addressed. This needs a lot more discussion time. > > @tstuefe Sorry to tag you. Can you review the code. Once this code goes in I can push in my changes. > > We are targeting the fix for January. > > > Hi @JoKern65 Is this good to integrate now ? > > @suchismith1993 Please don't put pressure on patch authors and developers. There is zero reason why this patch should be rushed. > > > Hi @suchismith1993, I'm waiting for a second review. Complex hotspot changes should be reviewed twice. > > Not only that, hotspot changes _need_ to be reviewed by at least two reviewers. That is not optional. See OpenJDK bylaws. Sorry about that. The fix was critical for the adoptium builds and hence was looking to fix this soon. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16920#issuecomment-1862776678 From goetz at openjdk.org Tue Dec 19 13:50:53 2023 From: goetz at openjdk.org (Goetz Lindenmaier) Date: Tue, 19 Dec 2023 13:50:53 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> <7FfEZmI1lotj-z6P6mJtk-jH7vfiq_mO0EYtW2iHuGI=.033a826a-7083-48dc-882a-2ded7b8b0da1@github.com> Message-ID: On Tue, 28 Nov 2023 12:59:01 GMT, Suchismith Roy wrote: >>> > >>> >>> @tstuefe 3rd parameter to pass the either of 2 things: >>> >>> 1. The JvmTiAgent object "agent", so that after shifting the save_library_signature to os_aix,we can still access the agent object.-> For this i tried importing jvm/prims header file, but i get segmentation faults during build . Not sure if i am doing it the right way. >>> >>> 2. Pass a character buffer(and not const char*) where we copy the modified filename back to it and then use it in jvmAgent. code. >> >> Does not sound really appealing tbh. We pile one hack atop of another. >> >> Please synchronize with @JoKern65 at SAP. He will rewrite the JVMTI handler code, which will make this point moot. See https://bugs.openjdk.org/browse/JDK-8320890. > >> > > >> > >> > >> > @tstuefe 3rd parameter to pass the either of 2 things: >> > ``` >> > 1. The JvmTiAgent object "agent", so that after shifting the save_library_signature to os_aix,we can still access the agent object.-> For this i tried importing jvm/prims header file, but i get segmentation faults during build . Not sure if i am doing it the right way. >> > >> > 2. Pass a character buffer(and not const char*) where we copy the modified filename back to it and then use it in jvmAgent. code. >> > ``` >> >> Does not sound really appealing tbh. We pile one hack atop of another. >> >> Please synchronize with @JoKern65 at SAP. He will rewrite the JVMTI handler code, which will make this point moot. See https://bugs.openjdk.org/browse/JDK-8320890. > > Hi @tstuefe Should i then wait for this code to be integrated and then rewrite the .a handling ? > I mean this PR shall remain open then right ? > @JoKern65 Are you even handling the .a handling case ? i would like this PR to stay open. Maybe i can wait for the design change that you are working on. Hi @suchismith1993 , you can place this change on top of #16920 by comparing it with branch origin/pr/16920 instead of master. This way you might be able to proceed with your change. But as Thomas says you can only push if you have appropriate reviews. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1862791022 From sroy at openjdk.org Tue Dec 19 13:57:49 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 13:57:49 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> <7FfEZmI1lotj-z6P6mJtk-jH7vfiq_mO0EYtW2iHuGI=.033a826a-7083-48dc-882a-2ded7b8b0da1@github.com> Message-ID: <4Owrf3BOOYfX2TRr_umNoiMCTAblzSc4Es44GUnC5Vc=.9fc5e71c-2b50-4617-b82f-fc3afeb9db21@github.com> On Tue, 28 Nov 2023 12:59:01 GMT, Suchismith Roy wrote: >>> > >>> >>> @tstuefe 3rd parameter to pass the either of 2 things: >>> >>> 1. The JvmTiAgent object "agent", so that after shifting the save_library_signature to os_aix,we can still access the agent object.-> For this i tried importing jvm/prims header file, but i get segmentation faults during build . Not sure if i am doing it the right way. >>> >>> 2. Pass a character buffer(and not const char*) where we copy the modified filename back to it and then use it in jvmAgent. code. >> >> Does not sound really appealing tbh. We pile one hack atop of another. >> >> Please synchronize with @JoKern65 at SAP. He will rewrite the JVMTI handler code, which will make this point moot. See https://bugs.openjdk.org/browse/JDK-8320890. > >> > > >> > >> > >> > @tstuefe 3rd parameter to pass the either of 2 things: >> > ``` >> > 1. The JvmTiAgent object "agent", so that after shifting the save_library_signature to os_aix,we can still access the agent object.-> For this i tried importing jvm/prims header file, but i get segmentation faults during build . Not sure if i am doing it the right way. >> > >> > 2. Pass a character buffer(and not const char*) where we copy the modified filename back to it and then use it in jvmAgent. code. >> > ``` >> >> Does not sound really appealing tbh. We pile one hack atop of another. >> >> Please synchronize with @JoKern65 at SAP. He will rewrite the JVMTI handler code, which will make this point moot. See https://bugs.openjdk.org/browse/JDK-8320890. > > Hi @tstuefe Should i then wait for this code to be integrated and then rewrite the .a handling ? > I mean this PR shall remain open then right ? > @JoKern65 Are you even handling the .a handling case ? i would like this PR to stay open. Maybe i can wait for the design change that you are working on. > Hi @suchismith1993 , you can place this change on top of #16920 by comparing it with branch origin/pr/16920 instead of master. This way you might be able to proceed with your change. But as Thomas says you can only push if you have appropriate reviews. Hi @GoeLin I am not sure how to do that . Could you tell me in brief ? Do I run the checkout command on the other PR and then place my change of top of it ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1862802768 From goetz at openjdk.org Tue Dec 19 16:08:51 2023 From: goetz at openjdk.org (Goetz Lindenmaier) Date: Tue, 19 Dec 2023 16:08:51 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <4Owrf3BOOYfX2TRr_umNoiMCTAblzSc4Es44GUnC5Vc=.9fc5e71c-2b50-4617-b82f-fc3afeb9db21@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> <7FfEZmI1lotj-z6P6mJtk-jH7vfiq_mO0EYtW2iHuGI=.033a826a-7083-48dc-882a-2ded7b8b0da1@github.com> <4Owrf3BOOYfX2TRr_umNoiMCTAblzSc4Es44GUnC5Vc=.9fc5e71c-2b50-4617-b82f-fc3afeb9db21@github.com> Message-ID: On Tue, 19 Dec 2023 13:55:09 GMT, Suchismith Roy wrote: > > Hi @suchismith1993 , you can place this change on top of #16920 by comparing it with branch origin/pr/16920 instead of master. This way you might be able to proceed with your change. But as Thomas says you can only push if you have appropriate reviews. > > Hi @GoeLin I am not sure how to do that . Could you tell me in brief ? Do I run the checkout command on the other PR and then place my change of top of it ? Yes, you can do that. You can also change the branch in this pr. Click edit on the top right. Choose an alternative for "openjdk:master" ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1863051404 From epeter at openjdk.org Tue Dec 19 16:13:53 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Tue, 19 Dec 2023 16:13:53 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Fri, 1 Dec 2023 07:56:04 GMT, Emanuel Peter wrote: > Before this patch, we always initialized the GrowableArray up to its `capacity`, and not just up to `length`. This is problematic for a few reasons: > > - It is not expected. `std::vector` also only initializes the elements up to its size, and not to capacity. > - It requires a default-constructor for the element type. And the default-constructor is then used to fill in the elements between length and capacity. If the elements do any allocation themselves, then this is a waste of resources. > - The implementation also required the copy-assignment-operator for the element type. This is a lesser restriction. But the copy-assignment-operator was used in cases like `append` (where placement copy-construct would be expected), and not just in true assignment kinds of cases like `at_put`. > > For this reason, I reworked a lot of the methods to ensure that only the "slots" up to `length` are ever initialized, and the space between `length` and `capacity` is always garbage. > > ----- > > Also, before this patch, one can CHeap allocate both with `GrowableArray` and `GrowableArrayCHeap`. This is unnecessary. It required more complex verification in `GrowableArray` to deal with all cases. And `GrowableArrayCHeap` is already explicitly a smaller object, and should hence be preferred. Hence I changed all CHeap allocating cases of `GrowableArray` to `GrowableArrayCHeap`. This also allows for a clear separation: > - `GrowableArray` only deals with arena / resource area allocation. These are arrays that are regularly abandoned at the end of their use, rather than deleted or even cleared. > - `GrowableArrayCHeap` only deals with CHeap allocated memory. We expect that the destructor for it is called eventually, either when it goes out of scope or when `delete` is explicitly called. We expect that the elements could be allocating resources internally, and hence rely on the destructors for the elements being called, which may free up those internally allocated resources. > > Therefore, we now only allow `GrowableArrayCHeap` to have element types with non-trivial destructors, but `GrowableArray` checks that element types do not have non-trivial destructors (since it is common practice to just abandon arena / resource area allocated arrays, rather than calling the destructor or clearing the array, which also destructs all elements). This more clearly separates the two worlds: clean-up your own mess (CHeap) vs abandon your mess (arena / resource area). > > ----- > > I also completely refactored and improved ... Filed: JDK-8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap Will work on that first, and then come back here later. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16918#issuecomment-1863061669 From sroy at openjdk.org Tue Dec 19 16:28:53 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 16:28:53 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <4Owrf3BOOYfX2TRr_umNoiMCTAblzSc4Es44GUnC5Vc=.9fc5e71c-2b50-4617-b82f-fc3afeb9db21@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> <7FfEZmI1lotj-z6P6mJtk-jH7vfiq_mO0EYtW2iHuGI=.033a826a-7083-48dc-882a-2ded7b8b0da1@github.com> <4Owrf3BOOYfX2TRr_umNoiMCTAblzSc4Es44GUnC5Vc=.9fc5e71c-2b50-4617-b82f-fc3afeb9db21@github.com> Message-ID: <8KgUJ7tjDJSJthb_9T1pAdJuRr88z_C9B714r5ZqJIE=.2443ed76-72b6-4e14-9821-6eea8f26c24d@github.com> On Tue, 19 Dec 2023 13:55:09 GMT, Suchismith Roy wrote: >>> > > >>> > >>> > >>> > @tstuefe 3rd parameter to pass the either of 2 things: >>> > ``` >>> > 1. The JvmTiAgent object "agent", so that after shifting the save_library_signature to os_aix,we can still access the agent object.-> For this i tried importing jvm/prims header file, but i get segmentation faults during build . Not sure if i am doing it the right way. >>> > >>> > 2. Pass a character buffer(and not const char*) where we copy the modified filename back to it and then use it in jvmAgent. code. >>> > ``` >>> >>> Does not sound really appealing tbh. We pile one hack atop of another. >>> >>> Please synchronize with @JoKern65 at SAP. He will rewrite the JVMTI handler code, which will make this point moot. See https://bugs.openjdk.org/browse/JDK-8320890. >> >> Hi @tstuefe Should i then wait for this code to be integrated and then rewrite the .a handling ? >> I mean this PR shall remain open then right ? >> @JoKern65 Are you even handling the .a handling case ? i would like this PR to stay open. Maybe i can wait for the design change that you are working on. > >> Hi @suchismith1993 , you can place this change on top of #16920 by comparing it with branch origin/pr/16920 instead of master. This way you might be able to proceed with your change. But as Thomas says you can only push if you have appropriate reviews. > > Hi @GoeLin I am not sure how to do that . Could you tell me in brief ? > Do I run the checkout command on the other PR and then place my change of top of it ? > > > Hi @suchismith1993 , you can place this change on top of #16920 by comparing it with branch origin/pr/16920 instead of master. This way you might be able to proceed with your change. But as Thomas says you can only push if you have appropriate reviews. > > > > > > Hi @GoeLin I am not sure how to do that . Could you tell me in brief ? Do I run the checkout command on the other PR and then place my change of top of it ? > > Yes, you can do that. You can also change the branch in this pr. Click edit on the top right. Choose an alternative for "openjdk:master" I see. So If I do that, will it reflect on my command line in local machine ? I mean I need run a rebase command with origin as pr/16920 ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1863086746 From goetz at openjdk.org Tue Dec 19 16:50:52 2023 From: goetz at openjdk.org (Goetz Lindenmaier) Date: Tue, 19 Dec 2023 16:50:52 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Wed, 22 Nov 2023 16:24:24 GMT, Suchismith Roy wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > Suchismith Roy has updated the pull request incrementally with one additional commit since the last revision: > > change macro position try it! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1863123599 From sroy at openjdk.org Tue Dec 19 17:06:02 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 17:06:02 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v3] In-Reply-To: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: > J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. > After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. > Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. Suchismith Roy has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: - Merge branch 'pr/16920' into jvmagent - change macro position - Adapt hotspot coding style - Improve comments and coding style. - Remove macro for file extension. - Move mapping function to aix specific file. - Introduce new macro for AIX archives. - Add support for .a extension in jvm agent. 1. Add support to load archive files and shared objects in jvm agent for AIX. ------------- Changes: https://git.openjdk.org/jdk/pull/16604/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=02 Stats: 13 lines in 3 files changed: 13 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16604.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16604/head:pull/16604 PR: https://git.openjdk.org/jdk/pull/16604 From sspitsyn at openjdk.org Tue Dec 19 17:29:56 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 19 Dec 2023 17:29:56 GMT Subject: Integrated: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable In-Reply-To: References: Message-ID: On Thu, 7 Dec 2023 06:28:43 GMT, Serguei Spitsyn wrote: > This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. > It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. > The deadlocking scenario is well described by Patricio in a bug report comment. > In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. > > The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. > This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. > > Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. > > New test was developed by Patricio: > `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > The test is very nice as it reliably in 100% reproduces the deadlock without the fix. > The test is never failing with this fix. > > Testing: > - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` > - tested with mach5 tiers 1-6 This pull request has now been integrated. Changeset: 0f8e4e0a Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/0f8e4e0a81257c678e948c341a241dc0b810494f Stats: 229 lines in 15 files changed: 196 ins; 0 del; 33 mod 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable Reviewed-by: lmesnik, alanb ------------- PR: https://git.openjdk.org/jdk/pull/17011 From sroy at openjdk.org Tue Dec 19 17:38:52 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 17:38:52 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Tue, 19 Dec 2023 16:47:33 GMT, Goetz Lindenmaier wrote: > try it! error: failed to push some refs to 'github.com:suchismith1993/jdk.git' I tried the fork instructions . However I think I need to do a force push. Will that be fine since the PR is not in draft state ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1863210891 From sroy at openjdk.org Tue Dec 19 17:59:17 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 17:59:17 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v4] In-Reply-To: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: > J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. > After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. > Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. Suchismith Roy has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains eight new commits since the last revision: - merge pr/16920 - change macro position - Adapt hotspot coding style - Improve comments and coding style. - Remove macro for file extension. - Move mapping function to aix specific file. - Introduce new macro for AIX archives. - Add support for .a extension in jvm agent. 1. Add support to load archive files and shared objects in jvm agent for AIX. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16604/files - new: https://git.openjdk.org/jdk/pull/16604/files/151f6c20..eb09224d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=02-03 Stats: 3 lines in 2 files changed: 0 ins; 3 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16604.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16604/head:pull/16604 PR: https://git.openjdk.org/jdk/pull/16604 From sroy at openjdk.org Tue Dec 19 19:17:03 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Tue, 19 Dec 2023 19:17:03 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v5] In-Reply-To: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: > J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. > After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. > Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. Suchismith Roy has updated the pull request incrementally with four additional commits since the last revision: - Change return type - Change dll load function signature that does dlopen - Remove AIX macros - Add wrapper function to check extension before dlopen ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16604/files - new: https://git.openjdk.org/jdk/pull/16604/files/eb09224d..cd7e0e64 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=03-04 Stats: 34 lines in 2 files changed: 22 ins; 11 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16604.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16604/head:pull/16604 PR: https://git.openjdk.org/jdk/pull/16604 From amenkov at openjdk.org Tue Dec 19 21:36:47 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 19 Dec 2023 21:36:47 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Tue, 19 Dec 2023 05:16:38 GMT, David Holmes wrote: >> Good point. I'll remove dependency on JVMTI. >> I don't think approximation would be good here (comparing state to RUNNABLE/PINNED/TIMED_PINNED or comparing carrierThread with null). >> It's racy and we have a chance to not dump unmounted vthread or dump mounted vthread twice. >> Maybe `is_vthread_mounted` should check if the virtual thread continuation has non-empty chunk. > > If that is racy then any solution is going to be racy. I assumed this was all happening at a global safepoint, otherwise threads could be mounting/unmounting at any time. > > I said "approximation" only because I'm unsure exactly when the thread state gets updated in the mounting/unmounting process. I mean race between virtual thread state change and the thread stack switch (to/from carrier). Correct condition here is "dump the virtual thread if it was not dumped as mounted virtual thread". Most likely for RUNNABLE/PINNED/TIMED_PINNED we can be sure the thread is mounted, but we can get vthread in transition (PARKING, YIELDING, etc). Virtual threads may be in transition at safepoints (but they can't change mounted/unmounted state). So I think `is_vthread_mounted` can be implemented in 2 ways: 1) copy logic of JvmtiEnvBase::get_JavaThread_or_null: get JavaThread for java_lang_VirtualThread::carrier_thread(vt); if not null, check Continuation::is_continuation_mounted(java_thread, java_lang_VirtualThread::continuation(vt)) - this is to handle transition, when vthread is already unmounted, but carrierThread is not yet set to null; 2) check that java_lang_VirtualThread::continuation(vt) doesn't have non-empty chunk. AFAIU this is true for mounted vthreads. If we get it for unmounted vt, its stack trace of the thread is empty anyway, so it's ok to skip it (most likely it can happen only if thread state is NEW or TERMINATED, we already skip such vthreads). @AlanBateman could you please comment if my understanding is correct ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1431959463 From cjplummer at openjdk.org Tue Dec 19 21:53:48 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 19 Dec 2023 21:53:48 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow [v2] In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 10:24:06 GMT, Jaikiran Pai wrote: >> Can I please get a review of this change which proposes to improve the code in `get_user_name_slow` function, which is used to identify the target JVM owner's user name? This addresses https://bugs.openjdk.org/browse/JDK-8321971. >> >> As noted in that JBS issue, in its current form, the nested loop ends up iterating over the directory contents of `hsperfdata_xxx` directory and then for each iteration it checks if the name of the entry matches the pid. This iteration shouldn't be needed and instead one could look for a file named `` within that directory. >> >> No new test has been added, given the nature of this change. Existing tier1, tier2, tier3 and svc_tools tests pass with this change on Linux, Windows and macosx. > > Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: > > - David's review comments - reduce if blocks and release the array outside if block > - David's review comment - punctuation src/hotspot/os/posix/perfMemory_posix.cpp line 609: > 607: if (statbuf.st_size > 0 && statbuf.st_ctime > oldest_ctime) { > 608: > 609: if (statbuf.st_ctime > oldest_ctime) { This `if` expression is repeats what we already know to be true from the previous `if` expression. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1431895566 From amenkov at openjdk.org Tue Dec 19 22:55:01 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 19 Dec 2023 22:55:01 GMT Subject: [jdk22] RFR: 8321565: [REDO] Heap dump does not contain virtual Thread stack references Message-ID: Hi all, This pull request contains a backport of commit [cf948548](https://github.com/openjdk/jdk/commit/cf948548c390c42ca63525d41a9d63ff31349c3a) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository. The commit being backported was authored by Alex Menkov on 13 Dec 2023 and was reviewed by Serguei Spitsyn, Yi Yang and David Holmes. Thanks! ------------- Commit messages: - Backport cf948548c390c42ca63525d41a9d63ff31349c3a Changes: https://git.openjdk.org/jdk22/pull/21/files Webrev: https://webrevs.openjdk.org/?repo=jdk22&pr=21&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8321565 Stats: 319 lines in 3 files changed: 152 ins; 81 del; 86 mod Patch: https://git.openjdk.org/jdk22/pull/21.diff Fetch: git fetch https://git.openjdk.org/jdk22.git pull/21/head:pull/21 PR: https://git.openjdk.org/jdk22/pull/21 From apangin at openjdk.org Wed Dec 20 03:39:43 2023 From: apangin at openjdk.org (Andrei Pangin) Date: Wed, 20 Dec 2023 03:39:43 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v15] In-Reply-To: References: Message-ID: On Thu, 14 Dec 2023 15:29:06 GMT, Dmitry Chuyko wrote: >> Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. >> >> A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. >> >> It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). >> >> Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. >> >> A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. >> >> In addition, a new diagnostic command `Compiler.replace_directives... > > Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits: > > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - Merge branch 'openjdk:master' into compiler-directives-force-update > - ... and 23 more: https://git.openjdk.org/jdk/compare/fde5b168...44d680cd src/hotspot/share/code/codeCache.cpp line 1409: > 1407: while(iter.next()) { > 1408: CompiledMethod* nm = iter.method(); > 1409: methodHandle mh(thread, nm->method()); If there are two CompiledMethods for the same Java method, will it be scheduled for recompilation twice? Related question: if `nm` is an OSR method, does it make sense to go directly for deoptimization rather than compiling a non-OSR version? src/hotspot/share/code/codeCache.cpp line 1413: > 1411: ResourceMark rm; > 1412: // Try the max level and let the directives be applied during the compilation. > 1413: int complevel = CompLevel::CompLevel_full_optimization; Should the highest level depend on the configuration instead of the hard-coded constant? Perhaps, needs to be `highest_compile_level()` src/hotspot/share/compiler/compilerDirectives.cpp line 750: > 748: if (!dir->is_default_directive() && dir->match(method)) { > 749: match_found = true; > 750: break; `match_found` is redundant: for better readability, you may just return true. Curly braces around MutexLocker won't be needed either. src/hotspot/share/oops/method.hpp line 820: > 818: // Clear the flags related to compiler directives that were set by the compilerBroker, > 819: // because the directives can be updated. > 820: void clear_method_flags() { The function name is a bit misleading - it clears only flags related to directives. src/hotspot/share/oops/methodFlags.hpp line 61: > 59: status(has_loops_flag_init , 1 << 14) /* The loop flag has been initialized */ \ > 60: status(on_stack_flag , 1 << 15) /* RedefineClasses support to keep Metadata from being cleaned */ \ > 61: status(has_matching_directives , 1 << 16) /* The method has matching directives */ \ It's worth noting that the flag is temporary and is valid only during DCmd execution. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1432195677 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1432187571 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1432200716 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1432210229 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1432212171 From dholmes at openjdk.org Wed Dec 20 04:47:58 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 20 Dec 2023 04:47:58 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v8] In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 17:09:59 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits: > > - Merge > - review: improve an assert message > - review: moved a couple of comments out of try blocks > - review: moved notifyJvmtiDisableSuspend(true) out of try-block > - review: 1) replace CriticalLock with DisableSuspend; 2) minor tweaks > - review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods > - Resolved merge conflict in VirtualThread.java > - added @summary to new test SuspendWithInterruptLock.java > - add new test SuspendWithInterruptLock.java > - 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable src/hotspot/share/prims/jvm.cpp line 4024: > 4022: #else > 4023: fatal("Should only be called with JVMTI enabled"); > 4024: #endif You can't do this! The Java code knows nothing about JVM TI being enabled/disabled and will call this function unconditionally. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1432241016 From jpai at openjdk.org Wed Dec 20 07:29:07 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Wed, 20 Dec 2023 07:29:07 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow [v3] In-Reply-To: References: Message-ID: > Can I please get a review of this change which proposes to improve the code in `get_user_name_slow` function, which is used to identify the target JVM owner's user name? This addresses https://bugs.openjdk.org/browse/JDK-8321971. > > As noted in that JBS issue, in its current form, the nested loop ends up iterating over the directory contents of `hsperfdata_xxx` directory and then for each iteration it checks if the name of the entry matches the pid. This iteration shouldn't be needed and instead one could look for a file named `` within that directory. > > No new test has been added, given the nature of this change. Existing tier1, tier2, tier3 and svc_tools tests pass with this change on Linux, Windows and macosx. Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - remove redundant if block - merge latest from master branch - David's review comments - reduce if blocks and release the array outside if block - David's review comment - punctuation - 8321971: Improve the user name detection logic in perfMemory get_user_name_slow ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17104/files - new: https://git.openjdk.org/jdk/pull/17104/files/cfd50d79..243d9817 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17104&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17104&range=01-02 Stats: 3660 lines in 265 files changed: 2177 ins; 586 del; 897 mod Patch: https://git.openjdk.org/jdk/pull/17104.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17104/head:pull/17104 PR: https://git.openjdk.org/jdk/pull/17104 From alanb at openjdk.org Wed Dec 20 08:05:06 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 20 Dec 2023 08:05:06 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v8] In-Reply-To: References: Message-ID: On Wed, 20 Dec 2023 04:44:35 GMT, David Holmes wrote: > You can't do this! The Java code knows nothing about JVM TI being enabled/disabled and will call this function unconditionally. Indeed. I wonder if anyone is testing minimal builds to catch issues like this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1432377494 From mbaesken at openjdk.org Wed Dec 20 08:05:45 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Wed, 20 Dec 2023 08:05:45 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: References: Message-ID: On Mon, 11 Dec 2023 14:38:08 GMT, Matthias Baesken wrote: >> A colleague contacted IBM about the different behavior of getaddrinfo on AIX (compared to Linux/macOS); maybe we have to adjust the result of the getaddrinfo call on AIX. > >> A colleague contacted IBM about the different behavior of getaddrinfo on AIX (compared to Linux/macOS); maybe we have to adjust the result of the getaddrinfo call on AIX. > > Haven't heard from them so far, hopefully we get an update soon about the behavior of getaddrinfo on AIX . > @MBaesken Just a thought: parseAllowedAddr() needs to parse only numeric addresses. getaddrinfo was used to handle both IPv4 and IPv6 by a single call, but maybe it would be better to reimplement parseAllowedAddr to do 2 inet_pton calls (for AF_INET and AF_INET6) Hi Alex, this seems to work (for AIX, and also for the other OpenJDK platforms) . ------------- PR Comment: https://git.openjdk.org/jdk/pull/16561#issuecomment-1864029003 From sroy at openjdk.org Wed Dec 20 08:36:06 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Wed, 20 Dec 2023 08:36:06 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v6] In-Reply-To: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: <_pW-qCHFEz6FfVSqHf2DVilu60sQNs5Lb8iREf1KJO4=.0942e492-1fb0-4e7b-9507-f406e4fb509e@github.com> > J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. > After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. > Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. Suchismith Roy has updated the pull request incrementally with two additional commits since the last revision: - Restore lines - Remove trailing spaces. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16604/files - new: https://git.openjdk.org/jdk/pull/16604/files/cd7e0e64..9df8c2c8 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=04-05 Stats: 4 lines in 1 file changed: 1 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16604.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16604/head:pull/16604 PR: https://git.openjdk.org/jdk/pull/16604 From sroy at openjdk.org Wed Dec 20 08:36:08 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Wed, 20 Dec 2023 08:36:08 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Tue, 28 Nov 2023 11:27:33 GMT, Suchismith Roy wrote: > > > i would have to repeat the line 1132 and 1139 in os_aix.cpp again , if the condition fails for .so files, because i have to reload it again and check if the .a exists. In the shared code i had repeat less number of lines i believe. Do you suggest moving lines 1132 to 1139 to another function then ? > > > > > > @tstuefe Any suggestion on this ? > > ``` > --- a/src/hotspot/os/aix/os_aix.cpp > +++ b/src/hotspot/os/aix/os_aix.cpp > @@ -1108,7 +1108,7 @@ bool os::dll_address_to_library_name(address addr, char* buf, > return true; > } > > -void *os::dll_load(const char *filename, char *ebuf, int ebuflen) { > +static void* dll_load_inner(const char *filename, char *ebuf, int ebuflen) { > > log_info(os)("attempting shared library load of %s", filename); > > @@ -1158,6 +1158,35 @@ void *os::dll_load(const char *filename, char *ebuf, int ebuflen) { > return nullptr; > } > > +void* os::dll_load(const char *filename, char *ebuf, int ebuflen) { > + > + void* result = nullptr; > + > + // First try using *.so suffix; failing that, retry with *.a suffix. > + const size_t len = strlen(filename); > + constexpr size_t safety = 3 + 1; > + constexpr size_t bufsize = len + safety; > + char* buf = NEW_C_HEAP_ARRAY(char, bufsize, mtInternal); > + strcpy(buf, filename); > + char* const dot = strrchr(buf, '.'); > + > + assert(dot != nullptr, "Attempting to load a shared object without extension? %s", filename); > + assert(strcmp(dot, ".a") == 0 || strcmp(dot, ".so") == 0, > + "Attempting to load a shared object that is neither *.so nor *.a", filename); > + > + sprintf(dot, ".so"); > + result = dll_load_inner(buf, ebuf, ebuflen); > + > + if (result == nullptr) { > + sprintf(dot, ".a"); > + result = dll_load_inner(buf, ebuf, ebuflen); > + } > + > + FREE_C_HEAP_ARRAY(char, buf); > + > + return result; > +} > + > ``` Hi Thomas May I know what is the reason to use constexpr over regular datatypes ? Also, I have used strcpy to avoid buffer overflow.(Though we have calculated the exact length). Would that be fine ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1864063261 From sroy at openjdk.org Wed Dec 20 09:24:43 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Wed, 20 Dec 2023 09:24:43 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: <1iM9tYJea_gUItPqqZ32s3lwfYXLZTbb9tlmKUS8OkY=.18b42be6-6c62-4a08-bc37-4ffd8ea5295d@github.com> On Tue, 19 Dec 2023 16:47:33 GMT, Goetz Lindenmaier wrote: > try it! I got the instructions to replicate in my local repo later, so wasn't sure to proceed. Thanks for the suggestion. I think this makes it easier to keep in sync with the other change. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1864127477 From dchuyko at openjdk.org Wed Dec 20 10:20:05 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Wed, 20 Dec 2023 10:20:05 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v16] In-Reply-To: References: Message-ID: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 34 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - ... and 24 more: https://git.openjdk.org/jdk/compare/14dab319...e337e56b ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=15 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From sspitsyn at openjdk.org Wed Dec 20 10:42:59 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 20 Dec 2023 10:42:59 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v8] In-Reply-To: References: Message-ID: On Wed, 20 Dec 2023 08:02:14 GMT, Alan Bateman wrote: >> src/hotspot/share/prims/jvm.cpp line 4024: >> >>> 4022: #else >>> 4023: fatal("Should only be called with JVMTI enabled"); >>> 4024: #endif >> >> You can't do this! The Java code knows nothing about JVM TI being enabled/disabled and will call this function unconditionally. > >> You can't do this! The Java code knows nothing about JVM TI being enabled/disabled and will call this function unconditionally. > > Indeed. I wonder if anyone is testing minimal builds to catch issues like this. Good catch, David! Filed a cleanup bug: https://bugs.openjdk.org/browse/JDK-8322538 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1432548911 From sroy at openjdk.org Wed Dec 20 11:16:03 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Wed, 20 Dec 2023 11:16:03 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v7] In-Reply-To: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: > J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. > After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. > Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. Suchismith Roy has updated the pull request incrementally with one additional commit since the last revision: Spaces fix ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16604/files - new: https://git.openjdk.org/jdk/pull/16604/files/9df8c2c8..ffcbf786 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=05-06 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16604.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16604/head:pull/16604 PR: https://git.openjdk.org/jdk/pull/16604 From sspitsyn at openjdk.org Wed Dec 20 11:29:52 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 20 Dec 2023 11:29:52 GMT Subject: [jdk22] RFR: 8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Tue, 19 Dec 2023 22:48:28 GMT, Alex Menkov wrote: > Hi all, > > This pull request contains a backport of commit [cf948548](https://github.com/openjdk/jdk/commit/cf948548c390c42ca63525d41a9d63ff31349c3a) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository. > > The commit being backported was authored by Alex Menkov on 13 Dec 2023 and was reviewed by Serguei Spitsyn, Yi Yang and David Holmes. > > Thanks! This is a clean backport - approved. ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk22/pull/21#pullrequestreview-1790704261 From jpai at openjdk.org Wed Dec 20 12:21:41 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Wed, 20 Dec 2023 12:21:41 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow [v3] In-Reply-To: References: Message-ID: On Wed, 20 Dec 2023 07:29:07 GMT, Jaikiran Pai wrote: >> Can I please get a review of this change which proposes to improve the code in `get_user_name_slow` function, which is used to identify the target JVM owner's user name? This addresses https://bugs.openjdk.org/browse/JDK-8321971. >> >> As noted in that JBS issue, in its current form, the nested loop ends up iterating over the directory contents of `hsperfdata_xxx` directory and then for each iteration it checks if the name of the entry matches the pid. This iteration shouldn't be needed and instead one could look for a file named `` within that directory. >> >> No new test has been added, given the nature of this change. Existing tier1, tier2, tier3 and svc_tools tests pass with this change on Linux, Windows and macosx. > > Jaikiran Pai has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: > > - remove redundant if block > - merge latest from master branch > - David's review comments - reduce if blocks and release the array outside if block > - David's review comment - punctuation > - 8321971: Improve the user name detection logic in perfMemory get_user_name_slow Just a note - I have incorporated the review comments, except from Johan which I'm still investigating and will update this PR soon. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17104#issuecomment-1864379594 From jpai at openjdk.org Wed Dec 20 12:21:44 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Wed, 20 Dec 2023 12:21:44 GMT Subject: RFR: 8321971: Improve the user name detection logic in perfMemory get_user_name_slow [v2] In-Reply-To: References: Message-ID: On Tue, 19 Dec 2023 20:13:58 GMT, Chris Plummer wrote: >> Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: >> >> - David's review comments - reduce if blocks and release the array outside if block >> - David's review comment - punctuation > > src/hotspot/os/posix/perfMemory_posix.cpp line 609: > >> 607: if (statbuf.st_size > 0 && statbuf.st_ctime > oldest_ctime) { >> 608: >> 609: if (statbuf.st_ctime > oldest_ctime) { > > This `if` expression repeats what we already know to be true from the previous `if` expression. Hello Chris, you are right - this additional `if` isn't needed. It looks like this is only there in `_posix` file and not in the `windows` version (which also intentionally doesn't have the file size check). I've updated this PR to fix this `_posix` one. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17104#discussion_r1432643800 From epeter at openjdk.org Wed Dec 20 12:57:10 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Wed, 20 Dec 2023 12:57:10 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap Message-ID: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. This has a few advantages: - Clear separation between arena (and resource area) allocating array and C-heap allocating array. - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. - We should not have multiple implementations of the same thing (C-Heap backed array). - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. **Bonus** We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. **Testing** Tier1-3 + stress testing: pending ------------- Commit messages: - fix comment about trivial elements - check for trivial destructors - improve comment - add an explicit to constructor - improve comments - remove cheap internals from GrowableArray and fix verification - remove constructors for GrowableArray with Cheap backing - 8322476 Changes: https://git.openjdk.org/jdk/pull/17160/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=17160&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8322476 Stats: 774 lines in 127 files changed: 108 ins; 204 del; 462 mod Patch: https://git.openjdk.org/jdk/pull/17160.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17160/head:pull/17160 PR: https://git.openjdk.org/jdk/pull/17160 From epeter at openjdk.org Wed Dec 20 12:58:53 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Wed, 20 Dec 2023 12:58:53 GMT Subject: RFR: 8319115: GrowableArray: Do not initialize up to capacity In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 22:48:18 GMT, Kim Barrett wrote: >> @eme64 Is it feasible to split this up to solve each of the problems you identify in stages? There is also overlap here with JDK-8319709 IIUC. Thanks. > >> @dholmes-ora These are the "parts": >> >> 1. initialize up to capacity vs length >> >> 2. update the test to verify this (complete refactoring) >> >> 3. remove cheap use of GrowableArray -> use GrowableArrayCHeap instead >> >> >> The first 2 items are inseparable, I cannot make substantial changes to many GrowableArray methods without there even being tests for them. And the tests would not pass before the changes for item 1, since the tests also verify what elements of the array are initialized. So adding the tests first would not be very feasible. >> >> The 3rd item could maybe be split, and be done before the rest. Though it would also require lots of changes to the test, which then I would have to completely refactor with items 1+2 anyway. >> >> And the items are related conceptually, that is why I would felt ok pushing them together. It is all about when (item 1) and what kinds of (item 3) constructors / destructors are called for the elements of the arrays, and verifying that thoroughly (item 2). >> >> Hence: feasible probably, but lots of work overhead. Do you think it is worth it? > > I too would prefer that it be split up. It's very easy to miss important details in amongst all the mostly relatively > simple renamings. That is, I think 3 should be separate from the other changes. @kimbarrett @dholmes-ora I just published this: https://github.com/openjdk/jdk/pull/17160 It removes the C-Heap capability from `GrowableArray`, and replaces usages with `GrowableArrayCHeap`. Bonus: we can now check that all element types of `GrowableArray` should be trivially destructible (that way it is always ok to abandon elements on the array, when the arena or ResourceMark go out of scope). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16918#issuecomment-1864429000 From jkern at openjdk.org Wed Dec 20 13:52:46 2023 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 20 Dec 2023 13:52:46 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v7] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: On Wed, 20 Dec 2023 11:16:03 GMT, Suchismith Roy wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > Suchismith Roy has updated the pull request incrementally with one additional commit since the last revision: > > Spaces fix Only some minor suggestions. src/hotspot/os/aix/os_aix.cpp line 1168: > 1166: int extension_length = 3; > 1167: char* file_path = NEW_C_HEAP_ARRAY(char, buffer_length + extension_length + 1, mtInternal); > 1168: strncpy(file_path,filename, buffer_length + 1); Why not using `char* file_path = os::strdup (filename);` which would replace lines 1167+1168 and use the corresponding `os::free (file_path);` at the end src/hotspot/os/aix/os_aix.cpp line 1174: > 1172: result = dll_load_library(file_path, ebuf, ebuflen); > 1173: // If the load fails,we try to reload by changing the extension to .a for .so files only. > 1174: if(result == nullptr) { Space between if and ( also next line ------------- Changes requested by jkern (Author). PR Review: https://git.openjdk.org/jdk/pull/16604#pullrequestreview-1790895382 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1432716207 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1432738451 From alanb at openjdk.org Wed Dec 20 14:19:04 2023 From: alanb at openjdk.org (Alan Bateman) Date: Wed, 20 Dec 2023 14:19:04 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v8] In-Reply-To: References: Message-ID: On Mon, 18 Dec 2023 17:09:59 GMT, Serguei Spitsyn wrote: >> This fix is for JDK 23 but the intention is to back port it to 22 in RDP-1 time frame. >> It is fixing a deadlock issue between `VirtualThread` class critical sections with the `interruptLock` (in methods: `unpark()`, `interrupt()`, `getAndClearInterrupt()`, `threadState()`, `toString()`), `JvmtiVTMSTransitionDisabler` and JVMTI `Suspend/Resume` mechanisms. >> The deadlocking scenario is well described by Patricio in a bug report comment. >> In simple words, a virtual thread should not be suspended during 'interruptLock' critical sections. >> >> The fix is to record that a virtual thread is in a critical section (`JavaThread`'s `_in_critical_section` bit) by notifying the VM/JVMTI about begin/end of critical section. >> This bit is used in `HandshakeState::get_op_for_self()` to filter out any `HandshakeOperation` if a target `JavaThread` is in a critical section. >> >> Some of new notifications with `notifyJvmtiSync()` method is on a performance critical path. It is why this method has been intrincified. >> >> New test was developed by Patricio: >> `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> The test is very nice as it reliably in 100% reproduces the deadlock without the fix. >> The test is never failing with this fix. >> >> Testing: >> - tested with newly added test: `test/hotspot/jtreg/serviceability/jvmti/vthread/SuspendWithInterruptLock` >> - tested with mach5 tiers 1-6 > > Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits: > > - Merge > - review: improve an assert message > - review: moved a couple of comments out of try blocks > - review: moved notifyJvmtiDisableSuspend(true) out of try-block > - review: 1) replace CriticalLock with DisableSuspend; 2) minor tweaks > - review: (1) rename notifyJvmti method; (2) add try-final statements to VirtualThread methods > - Resolved merge conflict in VirtualThread.java > - added @summary to new test SuspendWithInterruptLock.java > - add new test SuspendWithInterruptLock.java > - 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable src/hotspot/share/runtime/javaThread.hpp line 652: > 650: > 651: bool is_disable_suspend() const { return _is_disable_suspend; } > 652: void toggle_is_disable_suspend() { _is_disable_suspend = !_is_disable_suspend; }; Looking at this again then I don't think it can be a bit that is toggled on and off will work. Consider the case where several threads attempt to poll the state of a virtual Thread with Thread::getState at the same time. This can't work without an atomic counter and further coordination. So I think further work is required on this issue. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1432770204 From stuefe at openjdk.org Wed Dec 20 14:32:55 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 20 Dec 2023 14:32:55 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v7] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: On Wed, 20 Dec 2023 11:16:03 GMT, Suchismith Roy wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > Suchismith Roy has updated the pull request incrementally with one additional commit since the last revision: > > Spaces fix Hi, some requests and questions: - Please modify the JBS title, PR title, and JBS issue text to reflect that this adds an alternative shared object loading path for shared objects on AIX. Something like "Allow loading shared objects with .a extension on AIX". Please describe the new logic in the JBS issue text. - Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from? - What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen? - What happens if the original path handed to os::dll_load is already a *.a file? Should the logic then be reversed? - We really need regression tests for this. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1864572287 From jkern at openjdk.org Wed Dec 20 14:53:06 2023 From: jkern at openjdk.org (Joachim Kern) Date: Wed, 20 Dec 2023 14:53:06 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: Message-ID: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: improve error handling ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/f79c89da..7486ddb9 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=06-07 Stats: 14 lines in 3 files changed: 1 ins; 6 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From sspitsyn at openjdk.org Wed Dec 20 21:06:59 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 20 Dec 2023 21:06:59 GMT Subject: RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable [v8] In-Reply-To: References: Message-ID: <91xGNVFk3N8kP6sdAHhaGp9HhSaGS52_D6xW3aDivSY=.3196af09-9998-462d-9fe0-f91b2afd184a@github.com> On Wed, 20 Dec 2023 14:15:48 GMT, Alan Bateman wrote: > Update: ignore this I mis-read that it updates the current thread's suspend value, not the thread's suspend value. Thanks, Alan. I've also got confused with this and even filed a follow up bug. :) Yes, the initial design was the `_is_disable_suspend` is set/modified/accessed on the current thread only. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17011#discussion_r1433182764 From kbarrett at openjdk.org Wed Dec 20 21:13:53 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 20 Dec 2023 21:13:53 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> Message-ID: <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> On Tue, 19 Dec 2023 16:59:05 GMT, Emanuel Peter wrote: > [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. > > This has a few advantages: > - Clear separation between arena (and resource area) allocating array and C-heap allocating array. > - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. > - We should not have multiple implementations of the same thing (C-Heap backed array). > - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. > > **Bonus** > We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. > > For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. > > **Testing** > Tier1-3 + stress testing: pending pre-existing: There are a lot of non-static class data members that are pointers to GrowableArray that seem like they would be better as direct, e.g. non-pointers. pre-existing: There are a lot of iterations over GrowableArray's that would be simplified by using range-based-for. I'm not a fan of the additional clutter in APIs that the static memory types add. If we had a variant of GrowableArrayCHeap that was not itself dynamically allocatable and took a memory type to use internally as a constructor argument, then I think a lot of that clutter could be eliminated. It could be used for ordinary data members that are direct GAs rather than pointers to GAs. I think there is a way to do something similar for static data members that are pointers that are dynamically allocated later, though that probably requires more work. I've not yet reviewed the changes to growableArray.[ch]pp yet, nor the test changes. But I've run out of time and energy for this for today. src/hotspot/share/cds/dumpTimeClassInfo.hpp line 162: > 160: private: > 161: template > 162: static int array_length_or_zero(GrowableArrayCHeap* array) { Argument could be `GrowableArrayView*`, removing the coupling on the memory type. Also, pre-existing: the argument should be const. src/hotspot/share/cds/metaspaceShared.cpp line 441: > 439: > 440: void dump_java_heap_objects(GrowableArrayCHeap* klasses) NOT_CDS_JAVA_HEAP_RETURN; > 441: void dump_shared_symbol_table(GrowableArrayView* symbols) { pre-existing: Perhaps the arguments to these should be const. src/hotspot/share/cds/metaspaceShared.cpp line 840: > 838: > 839: #if INCLUDE_CDS_JAVA_HEAP > 840: void VM_PopulateDumpSharedSpace::dump_java_heap_objects(GrowableArrayCHeap* klasses) { pre-existing: Perhaps the argument should be const. src/hotspot/share/classfile/compactHashtable.cpp line 54: > 52: > 53: _num_entries_written = 0; > 54: _buckets = NEW_C_HEAP_ARRAY(EntryBucket*, _num_buckets, mtSymbol); pre-existing: It seems like the code could be simpler if the type of _buckets was GrowableArrayCHeap. src/hotspot/share/classfile/javaClasses.cpp line 1824: > 1822: // Pick minimum length that will cover most cases > 1823: int init_length = 64; > 1824: _methods = new GrowableArrayCHeap(init_length); Consider renaming init_length => init_capacity. src/hotspot/share/code/codeCache.hpp line 92: > 90: private: > 91: // CodeHeaps of the cache > 92: typedef GrowableArrayCHeap CodeHeapArray; pre-existing: Consider moving CodeHeapArray to namespace scope and prefer using it to the long-form. If not at namespace scope, it could at least be public in this class and used throughout, including in public APIs. src/hotspot/share/memory/arena.hpp line 209: > 207: > 208: #ifdef ASSERT > 209: bool Arena_contains(const Arena* arena, const void* ptr); This function doesn't seem necessary. Directly calling arena->contains(ptr) in the one place it's being seems like it should suffice. src/hotspot/share/memory/heapInspection.cpp line 282: > 280: KlassInfoHisto::KlassInfoHisto(KlassInfoTable* cit) : > 281: _cit(cit) { > 282: _elements = new GrowableArrayCHeap(_histo_initial_size); pre-existing: Why is this initialization separate from the ctor-initializer? And this looks like an example of where it would be better as a direct GA member rather than a pointer to GA. ------------- Changes requested by kbarrett (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17160#pullrequestreview-1790925376 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1432733463 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433109448 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433110835 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433129906 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433133733 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433140564 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433160909 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433164132 From kbarrett at openjdk.org Wed Dec 20 21:13:54 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Wed, 20 Dec 2023 21:13:54 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: On Wed, 20 Dec 2023 19:37:52 GMT, Kim Barrett wrote: >> [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. >> >> This has a few advantages: >> - Clear separation between arena (and resource area) allocating array and C-heap allocating array. >> - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. >> - We should not have multiple implementations of the same thing (C-Heap backed array). >> - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. >> >> **Bonus** >> We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. >> >> For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. >> >> **Testing** >> Tier1-3 + stress testing: pending > > src/hotspot/share/cds/metaspaceShared.cpp line 840: > >> 838: >> 839: #if INCLUDE_CDS_JAVA_HEAP >> 840: void VM_PopulateDumpSharedSpace::dump_java_heap_objects(GrowableArrayCHeap* klasses) { > > pre-existing: Perhaps the argument should be const. pre-existing and can't attach comment to line#50: `int i;` is dead variable. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433116120 From sspitsyn at openjdk.org Wed Dec 20 21:34:19 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 20 Dec 2023 21:34:19 GMT Subject: [jdk22] RFR: 8311218: fatal error: stuck in JvmtiVTMSTransitionDisabler::VTMS_transition_disable Message-ID: Hi all, This pull request contains a backport of commit [0f8e4e0a](https://github.com/openjdk/jdk/commit/0f8e4e0a81257c678e948c341a241dc0b810494f) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository. The commit being backported was authored by Serguei Spitsyn on 19 Dec 2023 and was reviewed by Leonid Mesnik and Alan Bateman. Thanks! ------------- Commit messages: - Backport 0f8e4e0a81257c678e948c341a241dc0b810494f Changes: https://git.openjdk.org/jdk22/pull/23/files Webrev: https://webrevs.openjdk.org/?repo=jdk22&pr=23&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8311218 Stats: 229 lines in 15 files changed: 196 ins; 0 del; 33 mod Patch: https://git.openjdk.org/jdk22/pull/23.diff Fetch: git fetch https://git.openjdk.org/jdk22.git pull/23/head:pull/23 PR: https://git.openjdk.org/jdk22/pull/23 From mdoerr at openjdk.org Thu Dec 21 00:13:56 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 21 Dec 2023 00:13:56 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Wed, 20 Dec 2023 14:53:06 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > improve error handling A pretty complex solution, but I couldn't spot any real bug. Please consider my suggestions. src/hotspot/os/aix/porting_aix.cpp line 25: > 23: */ > 24: // needs to be defined first, so that the implicit loaded xcoff.h header defines > 25: // the right structures to analyze the loader header of 32 and 64 Bit executable files I don't think we support 32 bit executables. src/hotspot/os/aix/porting_aix.cpp line 916: > 914: constexpr int max_handletable = 1024; > 915: static int g_handletable_used = 0; > 916: static struct handletableentry g_handletable[max_handletable] = {{0, 0, 0, 0}}; Wouldn't `ConcurrentHashTable` be a better data structure? It is already used in hotspot, can grow dynamically and doesn't need linear search. src/hotspot/os/aix/porting_aix.cpp line 921: > 919: // If the libpath cannot be retrieved return an empty path > 920: static const char* rtv_linkedin_libpath() { > 921: static char buffer[4096]; Maybe define a constant for the buffer size? src/hotspot/os/aix/porting_aix.cpp line 927: > 925: // let libpath point to buffer, which then contains a valid libpath > 926: // or an empty string > 927: if (libpath) { `!= nullptr` is common in hotspot. src/hotspot/os/aix/porting_aix.cpp line 934: > 932: // to open it > 933: snprintf(buffer, 100, "/proc/%ld/object/a.out", (long)getpid()); > 934: FILE* f = 0; Should be nullptr. src/hotspot/os/aix/porting_aix.cpp line 990: > 988: } > 989: ret = (0 == stat64x(combined.base(), stat)); > 990: os::free (path2); Please remove the extra whitespace. src/hotspot/os/aix/porting_aix.cpp line 1026: > 1024: > 1025: os::free (libpath); > 1026: os::free (path2); Same here. ------------- PR Review: https://git.openjdk.org/jdk/pull/16920#pullrequestreview-1791807521 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433267331 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433283111 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433273616 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433270399 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433289382 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433290839 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433291127 From amenkov at openjdk.org Thu Dec 21 01:51:48 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 21 Dec 2023 01:51:48 GMT Subject: [jdk22] Integrated: 8321565: [REDO] Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: On Tue, 19 Dec 2023 22:48:28 GMT, Alex Menkov wrote: > Hi all, > > This pull request contains a backport of commit [cf948548](https://github.com/openjdk/jdk/commit/cf948548c390c42ca63525d41a9d63ff31349c3a) from the [openjdk/jdk](https://git.openjdk.org/jdk) repository. > > The commit being backported was authored by Alex Menkov on 13 Dec 2023 and was reviewed by Serguei Spitsyn, Yi Yang and David Holmes. > > Thanks! This pull request has now been integrated. Changeset: fb3cc98d Author: Alex Menkov URL: https://git.openjdk.org/jdk22/commit/fb3cc98da3313e564a7495cc61208dd235e048b1 Stats: 319 lines in 3 files changed: 152 ins; 81 del; 86 mod 8321565: [REDO] Heap dump does not contain virtual Thread stack references Reviewed-by: sspitsyn Backport-of: cf948548c390c42ca63525d41a9d63ff31349c3a ------------- PR: https://git.openjdk.org/jdk22/pull/21 From epeter at openjdk.org Thu Dec 21 06:13:48 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Thu, 21 Dec 2023 06:13:48 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: On Wed, 20 Dec 2023 20:35:05 GMT, Kim Barrett wrote: >> [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. >> >> This has a few advantages: >> - Clear separation between arena (and resource area) allocating array and C-heap allocating array. >> - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. >> - We should not have multiple implementations of the same thing (C-Heap backed array). >> - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. >> >> **Bonus** >> We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. >> >> For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. >> >> **Testing** >> Tier1-3 + stress testing: pending > > src/hotspot/share/memory/arena.hpp line 209: > >> 207: >> 208: #ifdef ASSERT >> 209: bool Arena_contains(const Arena* arena, const void* ptr); > > This function doesn't seem necessary. Directly calling arena->contains(ptr) in the one place it's being seems > like it should suffice. @kimbarrett the reason was that I need to call this from the hpp file, and I encountered some circular dependency I did could not resolve. So I needed to move something off to the cpp files. Either I put it in arena.cpp, or in growableArray.cpp. But If I put things off to growableArray.cpp from the GrowableArray class, then it will not easily instantiate the templates, so that is not possible then. Hence I have to put it into arena.cpp ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433582008 From stuefe at openjdk.org Thu Dec 21 06:39:53 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 21 Dec 2023 06:39:53 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: <8CW_22cMoJAT7fewoZOfo5IwtdVxu1-zHktzkX_8fb4=.c4643f5c-0ba2-4771-ab83-c481bc8857f9@github.com> On Wed, 20 Dec 2023 14:53:06 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > improve error handling still ok, small nit inside src/hotspot/os/aix/porting_aix.cpp line 1033: > 1031: // filled by os::dll_load(). This way we mimic dl handle equality for a library > 1032: // opened a second time, as it is implemented on other platforms. > 1033: void* Aix_dlopen(const char* filename, int Flags, const char** error_report) { add assert for error_report != nullptr ------------- Marked as reviewed by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16920#pullrequestreview-1792301031 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433606032 From epeter at openjdk.org Thu Dec 21 06:45:48 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Thu, 21 Dec 2023 06:45:48 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: On Wed, 20 Dec 2023 21:11:09 GMT, Kim Barrett wrote: >> [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. >> >> This has a few advantages: >> - Clear separation between arena (and resource area) allocating array and C-heap allocating array. >> - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. >> - We should not have multiple implementations of the same thing (C-Heap backed array). >> - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. >> >> **Bonus** >> We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. >> >> For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. >> >> **Testing** >> Tier1-3 + stress testing: pending > > pre-existing: There are a lot of non-static class data members that are pointers to > GrowableArray that seem like they would be better as direct, e.g. non-pointers. > > pre-existing: There are a lot of iterations over GrowableArray's that would be > simplified by using range-based-for. > > I'm not a fan of the additional clutter in APIs that the static memory types add. > If we had a variant of GrowableArrayCHeap that was not itself dynamically allocatable > and took a memory type to use internally as a constructor argument, then I think a > lot of that clutter could be eliminated. It could be used for ordinary data members > that are direct GAs rather than pointers to GAs. I think there is a way to do something > similar for static data members that are pointers that are dynamically allocated later, > though that probably requires more work. > > I've not yet reviewed the changes to growableArray.[ch]pp yet, nor the test changes. > But I've run out of time and energy for this for today. @kimbarrett Thanks for looking at the PR! I see you address a lot of "pre-existing" issues. And you would like GrowableArrayCHeap not have the MEMFLAGS in the template argument but maybe as a constructor argument instead. Or maybe a GACH version that only allocates once, though I guess that would limit what kinds of methods you could call on it... Can we address these issues as separate RFE's? ------------- PR Comment: https://git.openjdk.org/jdk/pull/17160#issuecomment-1865639695 From sroy at openjdk.org Thu Dec 21 08:19:53 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Thu, 21 Dec 2023 08:19:53 GMT Subject: RFR: JDK-8320005 : Allow loading of shared objects with .a extension on AIX [v7] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: On Wed, 20 Dec 2023 14:30:18 GMT, Thomas Stuefe wrote: > Hi, > > some requests and questions: > > * Please modify the JBS title, PR title, and JBS issue text to reflect that this adds an alternative shared object loading path for shared objects on AIX. Something like "Allow loading shared objects with .a extension on AIX". Please describe the new logic in the JBS issue text. > * Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from? > * What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen? > * What happens if the original path handed to os::dll_load is already a *.a file? Should the logic then be reversed? > * We really need regression tests for this. For some of the question I need to consult the folk working on J9. I will answer a few of them if that gives some clarity. > * Please modify the JBS title, PR title, and JBS issue text to reflect that this adds an alternative shared object loading path for shared objects on AIX. Something like "Allow loading shared objects with .an extension on AIX". Please describe the new logic in the JBS issue text. Sure working on it. > * Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from? I am not sure how J9 handles this. I would have to consult . However as per current observation, this issue does not show up on Semuru. This issue is only happening on Adoptium. The team that release these file has always released *.a files which work fine for Semuru. > * What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen? I don't think the problem is with *.a . They would load as the default behaviour of the dlopen. It is only when the dlopen fails for *.so , we give another chance to check for .a file with the same name. > * What happens if the original path handed to os::dll_load is already a *.a file? Should the logic then be reversed? I don't think so. We are not modifying the behaviour to handle *.a files here. We are just adding extra checks for *.so files if they fail to load. In the logic , when a load fails, I just check if it is a .so file and perform the loading again. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1865837275 From stuefe at openjdk.org Thu Dec 21 08:35:57 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 21 Dec 2023 08:35:57 GMT Subject: RFR: JDK-8320005 : Allow loading of shared objects with .a extension on AIX [v7] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: On Thu, 21 Dec 2023 08:16:22 GMT, Suchismith Roy wrote: >> What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen? > I don't think the problem is with *.a . They would load as the default behaviour of the dlopen. It is only when the dlopen fails for *.so , we give another chance to check for .a file with the same name. No, what I meant, and what must be clarified before going forward with this solution, is the following: - is *every* `*.a` object on AIX loadable with `dlopen`, and will the result be the same as when loading a `*.so` object - or, if we present arbitrary `*.a` files to dlopen, is there a chance for dlopen to crash or misbehave. Reason is that I was under the impression that *.a libraries are static libraries and cannot be loaded dynamically. This is what you now try to do. If we cannot safely answer this question, I would opt for a more narrow solution by hard-wiring known alternative names. So, do the second *.a attempt only for your `ibm_16_am.a` which you know works. That could also be done in a reasonably maintainable manner. >> Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from? > I am not sure how J9 handles this. I would have to consult . J9 is Open Source, can't you just look? :) > However as per current observation, this issue does not show up on Semuru. This issue is only happening on Adoptium. The team that release these file has always released *.a files which work fine for Semuru. I don't know what Semuru is. What is the context, is that a different VM? Also OpenJDK? J9 derived? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1865859345 From jkern at openjdk.org Thu Dec 21 09:30:48 2023 From: jkern at openjdk.org (Joachim Kern) Date: Thu, 21 Dec 2023 09:30:48 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Wed, 20 Dec 2023 23:10:29 GMT, Martin Doerr wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> improve error handling > > src/hotspot/os/aix/porting_aix.cpp line 25: > >> 23: */ >> 24: // needs to be defined first, so that the implicit loaded xcoff.h header defines >> 25: // the right structures to analyze the loader header of 32 and 64 Bit executable files > > I don't think we support 32 bit executables. Originally my code worked for 32 & 64 Bit executables, but Thomas mentioned that we have only 64 Bit executables. So I removed the 32 Bit implementation, but this comment was an artefact. I removed the 32 Bit reference now. > src/hotspot/os/aix/porting_aix.cpp line 921: > >> 919: // If the libpath cannot be retrieved return an empty path >> 920: static const char* rtv_linkedin_libpath() { >> 921: static char buffer[4096]; > > Maybe define a constant for the buffer size? Done > src/hotspot/os/aix/porting_aix.cpp line 927: > >> 925: // let libpath point to buffer, which then contains a valid libpath >> 926: // or an empty string >> 927: if (libpath) { > > `!= nullptr` is common in hotspot. Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433797348 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433801010 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433798833 From sroy at openjdk.org Thu Dec 21 09:40:58 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Thu, 21 Dec 2023 09:40:58 GMT Subject: RFR: JDK-8320005 : Allow loading of shared objects with .a extension on AIX [v7] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: <_WrW-iHHdU-IgC7Z1b6oe_Qh0dkC6P3KJAdl7J2S1Do=.712dd065-6207-4632-a82f-8e12ad023cd5@github.com> On Wed, 20 Dec 2023 11:16:03 GMT, Suchismith Roy wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > Suchismith Roy has updated the pull request incrementally with one additional commit since the last revision: > > Spaces fix > > > What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen? > > > I don't think the problem is with *.a . They would load as the default behaviour of the dlopen. It is only when the dlopen fails for *.so , we give another chance to check for .a file with the same name. > > No, what I meant, and what must be clarified before going forward with this solution, is the following: > > * is _every_ `*.a` object on AIX loadable with `dlopen`, and will the result be the same as when loading a `*.so` object > * or, if we present arbitrary `*.a` files to dlopen, is there a chance for dlopen to crash or misbehave. > > Reason is that I was under the impression that *.a libraries are static libraries and cannot be loaded dynamically. This is what you now try to do. > If we cannot safely answer this question, I would opt for a more narrow solution by hard-wiring known alternative names. So, do the second *.a attempt only for your `ibm_16_am.a` which you know works. That could also be done in a reasonably maintainable manner. > In AIX, both static and dynamic libraries have *.a extension. And AIX also supports *.so files.Bascially shared objects in AIX have both *.a and *.so extension. Hence we need to implement this logic. If we try loading a static archive specifically ,how the dlopen would behave , that is something probably @JoKern65 can answer ? > > > Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from? > > > I am not sure how J9 handles this. I would have to consult . > > J9 is Open Source, can't you just look? :) I did try comparing the file structures, and i do not see a similar file structure over there. I am unable to find the jvmTiAgent code and also os_aix file. So i am not sure which functions over there are doing the same functionality. You have any suggestion on how i can check and correlate ? > > > However as per current observation, this issue does not show up on Semuru. This issue is only happening on Adoptium. The team that release these file has always released *.a files which work fine for Semuru. > > I don't know what Semuru is. What is the context, is that a different VM? Also OpenJDK? J9 derived? Semuru is J9 derived. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1865945011 From jkern at openjdk.org Thu Dec 21 09:42:08 2023 From: jkern at openjdk.org (Joachim Kern) Date: Thu, 21 Dec 2023 09:42:08 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Wed, 20 Dec 2023 23:45:16 GMT, Martin Doerr wrote: >> Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: >> >> improve error handling > > src/hotspot/os/aix/porting_aix.cpp line 916: > >> 914: constexpr int max_handletable = 1024; >> 915: static int g_handletable_used = 0; >> 916: static struct handletableentry g_handletable[max_handletable] = {{0, 0, 0, 0}}; > > Wouldn't `ConcurrentHashTable` be a better data structure? It is already used in hotspot, can grow dynamically and doesn't need linear search. There will be only few libraries in the list. With this assumption Thomas suggested to use just a simple array. > src/hotspot/os/aix/porting_aix.cpp line 990: > >> 988: } >> 989: ret = (0 == stat64x(combined.base(), stat)); >> 990: os::free (path2); > > Please remove the extra whitespace. Done > src/hotspot/os/aix/porting_aix.cpp line 1026: > >> 1024: >> 1025: os::free (libpath); >> 1026: os::free (path2); > > Same here. Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433813137 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433814446 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433814755 From jkern at openjdk.org Thu Dec 21 09:55:04 2023 From: jkern at openjdk.org (Joachim Kern) Date: Thu, 21 Dec 2023 09:55:04 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v9] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: cosmetic changes ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/7486ddb9..359080d3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=07-08 Stats: 10 lines in 1 file changed: 2 ins; 0 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From stuefe at openjdk.org Thu Dec 21 10:00:44 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 21 Dec 2023 10:00:44 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Thu, 21 Dec 2023 09:37:57 GMT, Joachim Kern wrote: >> src/hotspot/os/aix/porting_aix.cpp line 916: >> >>> 914: constexpr int max_handletable = 1024; >>> 915: static int g_handletable_used = 0; >>> 916: static struct handletableentry g_handletable[max_handletable] = {{0, 0, 0, 0}}; >> >> Wouldn't `ConcurrentHashTable` be a better data structure? It is already used in hotspot, can grow dynamically and doesn't need linear search. > > There will be only few libraries in the list. With this assumption Thomas suggested to use just a simple array. Let's keep it simple. A linear array of only a few items is easily scanned, probably faster than pointer hopping hash table entries. Not that it matters in any way for the few calls to dlopen. Also, avoiding hotspot structures preserves layer integrity (porting_aix does not pull anything from hotspot so far) and prevents initialisation time dependencies. Not sure whether ConcurrentHashTable works before VM init, but with Joachimes current solution, we can call dlopen at any time in VM life. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433839119 From stuefe at openjdk.org Thu Dec 21 10:03:55 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 21 Dec 2023 10:03:55 GMT Subject: RFR: JDK-8320005 : Allow loading of shared objects with .a extension on AIX [v7] In-Reply-To: <_WrW-iHHdU-IgC7Z1b6oe_Qh0dkC6P3KJAdl7J2S1Do=.712dd065-6207-4632-a82f-8e12ad023cd5@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <_WrW-iHHdU-IgC7Z1b6oe_Qh0dkC6P3KJAdl7J2S1Do=.712dd065-6207-4632-a82f-8e12ad023cd5@github.com> Message-ID: <2sMyJ8mZ6EIULC67tK1IcI4uNnJMvpCzw1BKEDUaIms=.c90f1101-236e-4a80-869c-feca6abd3dc3@github.com> On Thu, 21 Dec 2023 09:37:55 GMT, Suchismith Roy wrote: > > > > What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen? > > > > > > > I don't think the problem is with *.a . They would load as the default behaviour of the dlopen. It is only when the dlopen fails for *.so , we give another chance to check for .a file with the same name. > > > > > > No, what I meant, and what must be clarified before going forward with this solution, is the following: > > > > * is _every_ `*.a` object on AIX loadable with `dlopen`, and will the result be the same as when loading a `*.so` object > > * or, if we present arbitrary `*.a` files to dlopen, is there a chance for dlopen to crash or misbehave. > > > > Reason is that I was under the impression that *.a libraries are static libraries and cannot be loaded dynamically. This is what you now try to do. > > If we cannot safely answer this question, I would opt for a more narrow solution by hard-wiring known alternative names. So, do the second *.a attempt only for your `ibm_16_am.a` which you know works. That could also be done in a reasonably maintainable manner. > > In AIX, both static and dynamic libraries have *.a extension. And AIX also supports *.so files.Bascially shared objects in AIX have both *.a and *.so extension. Hence we need to implement this logic. If we try loading a static archive specifically ,how the dlopen would behave , that is something probably @JoKern65 can answer ? Rather, this is a question you have to ask your collegues at IBM that develop the AIX libc. Since AIX libc is not open source, we cannot look for ourselves, nor can Joachim (her works at SAP). > > > > > Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from? > > > > > > > I am not sure how J9 handles this. I would have to consult . > > > > > > J9 is Open Source, can't you just look? :) > > I did try comparing the file structures, and i do not see a similar file structure over there. I am unable to find the jvmTiAgent code and also os_aix file. So i am not sure which functions over there are doing the same functionality. You have any suggestion on how i can check and correlate ? Someone must implement LoadLibrary. Try looking for places where dlopen() is called. > > > > However as per current observation, this issue does not show up on Semuru. This issue is only happening on Adoptium. The team that release these file has always released *.a files which work fine for Semuru. > > > > > > I don't know what Semuru is. What is the context, is that a different VM? Also OpenJDK? J9 derived? > > Semuru is J9 derived. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1865977132 From mdoerr at openjdk.org Thu Dec 21 10:19:45 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 21 Dec 2023 10:19:45 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Thu, 21 Dec 2023 09:57:08 GMT, Thomas Stuefe wrote: >> There will be only few libraries in the list. With this assumption Thomas suggested to use just a simple array. > > Let's keep it simple. A linear array of only a few items is easily scanned, probably faster than pointer hopping hash table entries. Not that it matters in any way for the few calls to dlopen. > > Also, avoiding hotspot structures preserves layer integrity (porting_aix does not pull anything from hotspot so far) and prevents initialisation time dependencies. Not sure whether ConcurrentHashTable works before VM init, but with Joachimes current solution, we can call dlopen at any time in VM life. I don't like introducing unnecessary limitations. Are we sure nobody will ever need more than 1024 handles? Can't we at least use a GrowableArray or something like that? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433865222 From dchuyko at openjdk.org Thu Dec 21 11:13:25 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Thu, 21 Dec 2023 11:13:25 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v17] In-Reply-To: References: Message-ID: <-u99xleA_S6Se_2wET4iFO7jzqN-GmaiC8fBsAzzcQs=.cf28f08a-7de2-4073-a71b-8fda1620e402@github.com> > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 35 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - ... and 25 more: https://git.openjdk.org/jdk/compare/aff659aa...fbedf276 ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=16 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From jsjolen at openjdk.org Thu Dec 21 11:13:56 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Thu, 21 Dec 2023 11:13:56 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> Message-ID: On Tue, 19 Dec 2023 16:59:05 GMT, Emanuel Peter wrote: > [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. > > This has a few advantages: > - Clear separation between arena (and resource area) allocating array and C-heap allocating array. > - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. > - We should not have multiple implementations of the same thing (C-Heap backed array). > - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. > > **Bonus** > We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. > > For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. > > **Testing** > Tier1-3 + stress testing: pending Wow! Thank you for this Emanuel. I went through your changes and I am happy with them. There are some spelling issues and my opinions on how to write the doc strings. I also asked for some "length"/"size" naming to be changed to "capacity", you don't have to do this as it's pre-existing, but it would make that code clearer. src/hotspot/share/jfr/leakprofiler/chains/edgeStore.cpp line 287: > 285: assert(edge != nullptr, "invariant"); > 286: if (_leak_context_edges == nullptr) { > 287: _leak_context_edges = new GrowableArrayCHeap(initial_size); Pre-existing: `initial_capacity` is a better name. src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleCheckpoint.cpp line 55: > 53: template > 54: static GrowableArrayCHeap* c_heap_allocate_array(int size = initial_array_size) { > 55: return new GrowableArrayCHeap(size); `capacity` instead of `size` src/hotspot/share/jfr/recorder/checkpoint/types/jfrThreadGroup.cpp line 266: > 264: > 265: JfrThreadGroup::JfrThreadGroup() : > 266: _list(new GrowableArrayCHeap(initial_array_size)) {} `capacity` src/hotspot/share/jfr/recorder/jfrRecorder.cpp line 151: > 149: assert(length >= 1, "invariant"); > 150: assert(dcmd_recordings_array == nullptr, "invariant"); > 151: dcmd_recordings_array = new GrowableArrayCHeap(length); `capacity` src/hotspot/share/jfr/support/jfrKlassUnloading.cpp line 38: > 36: > 37: template > 38: static GrowableArrayCHeap* c_heap_allocate_array(int size = initial_array_size) { `capacity` src/hotspot/share/utilities/growableArray.hpp line 618: > 616: > 617: // The GrowableArray internal data is allocated from either: > 618: // - Resrouce area (default) Spelling src/hotspot/share/utilities/growableArray.hpp line 621: > 619: // - Arena > 620: // > 621: // Itself, it can be embedded, on stack, resource_arena or arena allocated. "Itself can be allocated on stack, resource area or arena allocated." src/hotspot/share/utilities/growableArray.hpp line 629: > 627: // For C-Heap allocation use GrowableArrayCHeap. > 628: // > 629: // Note, that with GrowableArray does not deallocate the allocated memory from "that the" not "that with" src/hotspot/share/utilities/growableArray.hpp line 638: > 636: // GrowableArray is copyable, but it only creates a shallow copy. Hence, one has > 637: // to be careful not to duplicate the state and then diverge while sharing the > 638: // underlying data. Sad but true :-( src/hotspot/share/utilities/growableArray.hpp line 644: > 642: friend class GrowableArrayWithAllocator >; > 643: > 644: // Since GrowableArray is arena / resource area allocated, it is a custom to "it is a custom to" can basically be removed "Since Growable array is arena/resource area allocated it does not destruct its elements. Therefore, ..." is sufficient. ------------- PR Review: https://git.openjdk.org/jdk/pull/17160#pullrequestreview-1792708591 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433894629 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433894947 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433895133 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433895741 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433896219 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433901496 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433904262 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433916326 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433918209 PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433920731 From jsjolen at openjdk.org Thu Dec 21 11:13:57 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Thu, 21 Dec 2023 11:13:57 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: On Wed, 20 Dec 2023 20:39:27 GMT, Kim Barrett wrote: >> [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. >> >> This has a few advantages: >> - Clear separation between arena (and resource area) allocating array and C-heap allocating array. >> - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. >> - We should not have multiple implementations of the same thing (C-Heap backed array). >> - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. >> >> **Bonus** >> We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. >> >> For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. >> >> **Testing** >> Tier1-3 + stress testing: pending > > src/hotspot/share/memory/heapInspection.cpp line 282: > >> 280: KlassInfoHisto::KlassInfoHisto(KlassInfoTable* cit) : >> 281: _cit(cit) { >> 282: _elements = new GrowableArrayCHeap(_histo_initial_size); > > pre-existing: Why is this initialization separate from the ctor-initializer? And this looks like an example of > where it would be better as a direct GA member rather than a pointer to GA. Can name be changed to `_histo_initial_capacity`? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433897055 From jsjolen at openjdk.org Thu Dec 21 11:13:58 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Thu, 21 Dec 2023 11:13:58 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> Message-ID: On Thu, 21 Dec 2023 10:48:30 GMT, Johan Sj?len wrote: >> [JDK-8247755](https://bugs.openjdk.org/browse/JDK-8247755) introduced the `GrowableArrayCHeap`. This duplicates the current C-Heap allocation capability in `GrowableArray`. I now remove that from `GrowableArray` and move all usages to `GrowableArrayCHeap`. >> >> This has a few advantages: >> - Clear separation between arena (and resource area) allocating array and C-heap allocating array. >> - We can prevent assigning / copying between arrays of different allocation strategies already at compile time, and not only with asserts at runtime. >> - We should not have multiple implementations of the same thing (C-Heap backed array). >> - `GrowableArrayCHeap` is NONCOPYABLE. This is a nice restriction, we now know that C-Heap backed arrays do not get copied unknowingly. >> >> **Bonus** >> We can now restrict `GrowableArray` element type `E` to be `std::is_trivially_destructible::value == true`. The idea is that arena / resource allocated arrays get abandoned, often without being even cleared. Hence, the elements in the array are never destructed. But if we only use elements that are trivially destructible, then it makes no difference if the destructors are ever called, or the elements simply abandoned. >> >> For `GrowableArrayCHeap`, we expect that the user eventually calls the destructor for the array, which in turn calls the destructors of the remaining elements. Hence, it is up to the user to ensure the cleanup. And so we can allow non-trivial destructors. >> >> **Testing** >> Tier1-3 + stress testing: pending > > src/hotspot/share/utilities/growableArray.hpp line 621: > >> 619: // - Arena >> 620: // >> 621: // Itself, it can be embedded, on stack, resource_arena or arena allocated. > > "Itself can be allocated on stack, resource area or arena allocated." That it can be embedded into another class/struct is a given, imho. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1433904533 From jkern at openjdk.org Thu Dec 21 11:26:44 2023 From: jkern at openjdk.org (Joachim Kern) Date: Thu, 21 Dec 2023 11:26:44 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Thu, 21 Dec 2023 10:17:18 GMT, Martin Doerr wrote: >> Let's keep it simple. A linear array of only a few items is easily scanned, probably faster than pointer hopping hash table entries. Not that it matters in any way for the few calls to dlopen. >> >> Also, avoiding hotspot structures preserves layer integrity (porting_aix does not pull anything from hotspot so far) and prevents initialisation time dependencies. Not sure whether ConcurrentHashTable works before VM init, but with Joachimes current solution, we can call dlopen at any time in VM life. > > I don't like introducing unnecessary limitations. Are we sure nobody will ever need more than 1024 handles? > Can't we at least use a GrowableArray or something like that? In principle you are right, but in my opinion 1024 is an academical limit. I never saw processes with more than a few dozen loaded libraries. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433942205 From stuefe at openjdk.org Thu Dec 21 11:49:44 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 21 Dec 2023 11:49:44 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Thu, 21 Dec 2023 11:23:46 GMT, Joachim Kern wrote: >> I don't like introducing unnecessary limitations. Are we sure nobody will ever need more than 1024 handles? >> Can't we at least use a GrowableArray or something like that? > > In principle you are right, but in my opinion 1024 is an academical limit. I never saw processes with more than a few dozen loaded libraries. Dynamic allocation also opens us up to potential initialization issues, unless we explicitly use raw ::malloc. It should work, but I think its better avoided unless we really need it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433963889 From mdoerr at openjdk.org Thu Dec 21 11:56:53 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 21 Dec 2023 11:56:53 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Thu, 21 Dec 2023 11:45:36 GMT, Thomas Stuefe wrote: >> In principle you are right, but in my opinion 1024 is an academical limit. I never saw processes with more than a few dozen loaded libraries. > > Dynamic allocation also opens us up to potential initialization issues, unless we explicitly use raw ::malloc. It should work, but I think its better avoided unless we really need it. Well we're fixing an academic issue by introducing another one? Doesn't make sense to me. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433972207 From stuefe at openjdk.org Thu Dec 21 12:16:54 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 21 Dec 2023 12:16:54 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v8] In-Reply-To: References: <1jQHeWyTuhMCLg2f-3-oMUECl3sjvkJNXmQoHYwxu1c=.d3355a3c-fe6f-46bb-bcff-968401a830f6@github.com> Message-ID: On Thu, 21 Dec 2023 11:54:17 GMT, Martin Doerr wrote: >> Dynamic allocation also opens us up to potential initialization issues, unless we explicitly use raw ::malloc. It should work, but I think its better avoided unless we really need it. > > Well we're fixing an academic issue by introducing another one? Doesn't make sense to me. Okay, I butt out, I don't care enough. Up to you both to decide what to do. My recommendation would still be to avoid hotspot infrastructure that relies on os::malloc and friends; other than that, rewriting this table to make it growable using realloc should be trivial. Note that we need *some* sort of limit though. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1433996779 From sspitsyn at openjdk.org Thu Dec 21 12:29:47 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 21 Dec 2023 12:29:47 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Tue, 19 Dec 2023 21:34:10 GMT, Alex Menkov wrote: > I mean race between virtual thread state change and the thread stack switch (to/from carrier). I'm not sure if I understand you correctly or if we can call it a race. Alan will correct me if I'm wrong. You are talking about thread state change. At least, mount state transition happens on the same JavaThread (it seems, you call it thread state switch). Mount state transition can go over a safepoint. But it should not progress while in a safepoint. David pointed out, "this was all happening at a global safepoint". My understanding is this assumption is correct. Then your approach `to identify that a virtual thread is mounted or not` should work in general. The condition `java_lang_VirtualThread::carrier_thread(vt) != nullptr` should indicate that the `vt` is mounted or is being in mount or unmount transition. If the `vt` is in mount or unmount transition then (it is a gray zone) the way we identify mounted state should match the way we did it when dumped mounted virtual threads. It is done this way: `oop mounted_vt = thread->is_vthread_mounted() ? thread->vthread() : nullptr;` So, it seems any of yous suggestion should work here. Though, it would be nice to simplify it a little if possible. Again, to be consistent, a `vt` in mount state transition just have to be identified as mounted or unmounted in both fragments in a similar way . ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1434006417 From dchuyko at openjdk.org Thu Dec 21 13:38:15 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Thu, 21 Dec 2023 13:38:15 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v18] In-Reply-To: References: Message-ID: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 36 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - ... and 26 more: https://git.openjdk.org/jdk/compare/6de23bf3...b348ebed ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=17 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From amenkov at openjdk.org Thu Dec 21 23:18:45 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 21 Dec 2023 23:18:45 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Thu, 21 Dec 2023 12:23:35 GMT, Serguei Spitsyn wrote: >> I mean race between virtual thread state change and the thread stack switch (to/from carrier). >> Correct condition here is "dump the virtual thread if it was not dumped as mounted virtual thread". >> Most likely for RUNNABLE/PINNED/TIMED_PINNED we can be sure the thread is mounted, but we can get vthread in transition (PARKING, YIELDING, etc). Virtual threads may be in transition at safepoints (but they can't change mounted/unmounted state). >> So I think `is_vthread_mounted` can be implemented in 2 ways: >> 1) copy logic of JvmtiEnvBase::get_JavaThread_or_null: >> get JavaThread for java_lang_VirtualThread::carrier_thread(vt); >> if not null, check Continuation::is_continuation_mounted(java_thread, java_lang_VirtualThread::continuation(vt)) - this is to handle transition, when vthread is already unmounted, but carrierThread is not yet set to null; >> 2) check that java_lang_VirtualThread::continuation(vt) doesn't have non-empty chunk. >> AFAIU this is true for mounted vthreads. If we get it for unmounted vt, its stack trace of the thread is empty anyway, so it's ok to skip it (most likely it can happen only if thread state is NEW or TERMINATED, we already skip such vthreads). >> >> @AlanBateman could you please comment if my understanding is correct > >> I mean race between virtual thread state change and the thread stack switch (to/from carrier). > > I'm not sure if I understand you correctly or if we can call it a race. Alan will correct me if I'm wrong. > You are talking about thread state change. At least, mount state transition happens on the same JavaThread (it seems, you call it thread state switch). Mount state transition can go over a safepoint. But it should not progress while in a safepoint. David pointed out, "this was all happening at a global safepoint". My understanding is this assumption is correct. Then your approach `to identify that a virtual thread is mounted or not` should work in general. The condition `java_lang_VirtualThread::carrier_thread(vt) != nullptr` should indicate that the `vt` is mounted or is being in mount or unmount transition. > If the `vt` is in mount or unmount transition then (it is a gray zone) the way we identify mounted state should match the way we did it when dumped mounted virtual threads. > It is done this way: `oop mounted_vt = thread->is_vthread_mounted() ? thread->vthread() : nullptr;` > So, it seems any of yous suggestion should work here. Though, it would be nice to simplify it a little if possible. Again, to be consistent, a `vt` in mount state transition just have to be identified as mounted or unmounted in both fragments in a similar way . Looks like "race" is wrong word here. There is no race between different threads, we just cannot rely on vt state or carrierThread value when the thread in mount/unmount transition. Sorry for the confusion. Serguei, thank you for the analysis. I agree, the code for mounted and unmounted vthreads should be consistent. For unmounted threads we have to get JavaThread of the carrier thread and if it's not null, check java_thread->is_vthread_mounted(). We don't need to check `is_continuation_mounted` as we are at safepoint. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1434583304 From amenkov at openjdk.org Fri Dec 22 00:31:51 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 22 Dec 2023 00:31:51 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads [v2] In-Reply-To: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: > HeapDumper dumps virtual threads in 2 places: > - dumping platform threads (mounted virtual threads are dumped as separate thread object); > - dumping heap objects when the object is `java.lang.VirtualThread`. > > In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) > Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. > > Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: feedback: reimplemented ThreadDumpe::is_vthread_mounted() ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17134/files - new: https://git.openjdk.org/jdk/pull/17134/files/3fc64e3a..0afd2a4f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17134&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17134&range=00-01 Stats: 11 lines in 1 file changed: 9 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/17134.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17134/head:pull/17134 PR: https://git.openjdk.org/jdk/pull/17134 From dholmes at openjdk.org Fri Dec 22 00:47:51 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 22 Dec 2023 00:47:51 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads [v2] In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Fri, 22 Dec 2023 00:31:51 GMT, Alex Menkov wrote: >> HeapDumper dumps virtual threads in 2 places: >> - dumping platform threads (mounted virtual threads are dumped as separate thread object); >> - dumping heap objects when the object is `java.lang.VirtualThread`. >> >> In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) >> Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. >> >> Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > feedback: reimplemented ThreadDumpe::is_vthread_mounted() This seems good to me know - thanks for the updates. One minor suggestion below. src/hotspot/share/services/heapDumper.cpp line 1934: > 1932: // create a HPROF_GC_INSTANCE record for each object > 1933: DumperSupport::dump_instance(writer(), o, &_class_cache); > 1934: if (java_lang_VirtualThread::is_instance(o) Suggestion. Just to be clear add a comment // If we encounter an unmounted virtual thread it needs to be dumped explicitly. or something to that effect. Thanks. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17134#pullrequestreview-1793820556 PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1434616383 From amenkov at openjdk.org Fri Dec 22 01:43:07 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 22 Dec 2023 01:43:07 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads [v3] In-Reply-To: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: > HeapDumper dumps virtual threads in 2 places: > - dumping platform threads (mounted virtual threads are dumped as separate thread object); > - dumping heap objects when the object is `java.lang.VirtualThread`. > > In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) > Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. > > Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: feedback: comment added ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17134/files - new: https://git.openjdk.org/jdk/pull/17134/files/0afd2a4f..c6ad90ca Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17134&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17134&range=01-02 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/17134.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17134/head:pull/17134 PR: https://git.openjdk.org/jdk/pull/17134 From amenkov at openjdk.org Fri Dec 22 01:43:09 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 22 Dec 2023 01:43:09 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads [v2] In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Fri, 22 Dec 2023 00:44:16 GMT, David Holmes wrote: >> Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: >> >> feedback: reimplemented ThreadDumpe::is_vthread_mounted() > > src/hotspot/share/services/heapDumper.cpp line 1934: > >> 1932: // create a HPROF_GC_INSTANCE record for each object >> 1933: DumperSupport::dump_instance(writer(), o, &_class_cache); >> 1934: if (java_lang_VirtualThread::is_instance(o) > > Suggestion. Just to be clear add a comment > > // If we encounter an unmounted virtual thread it needs to be dumped explicitly. > > or something to that effect. Thanks. Thank you for review Added comment as suggested ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1434638886 From kbarrett at openjdk.org Fri Dec 22 02:00:50 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 22 Dec 2023 02:00:50 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> Message-ID: On Thu, 21 Dec 2023 11:10:43 GMT, Johan Sj?len wrote: > ... I also asked for some "length"/"size" naming to be changed to "capacity", you don't have to do this as it's pre-existing, but it would make that code clearer. I think I only commented on one in my pass over the code, but I agree with all of @jdksjolen suggestions for those. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17160#issuecomment-1867135604 From kbarrett at openjdk.org Fri Dec 22 02:25:47 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 22 Dec 2023 02:25:47 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: On Thu, 21 Dec 2023 06:11:03 GMT, Emanuel Peter wrote: >> src/hotspot/share/memory/arena.hpp line 209: >> >>> 207: >>> 208: #ifdef ASSERT >>> 209: bool Arena_contains(const Arena* arena, const void* ptr); >> >> This function doesn't seem necessary. Directly calling arena->contains(ptr) in the one place it's being seems >> like it should suffice. > > @kimbarrett the reason was that I need to call this from the hpp file, and I encountered some circular dependency I did could not resolve. So I needed to move something off to the cpp files. Either I put it in arena.cpp, or in growableArray.cpp. But If I put things off to growableArray.cpp from the GrowableArray class, then it will not easily instantiate the templates, so that is not possible then. Hence I have to put it into arena.cpp I don't think a global API is warranted to support that local debug-only implementation detail. The inclusion problem arises because the PR eliminates GrowableArrayMetadata (a non-templated class), thereby forcing the init_checks helper to be moved to GrowableArray (a class template). That forced moving the implementation from the .cpp to the .hpp. One solution might be to move the new version of init_checks to growableArray.inline.hpp. After all, breaking circularities is one of the reasons one might have an inline.hpp file. However, that file doesn't currently exist, and I think introducing it would have too much fannout. And it might not even work without even more effort; there might be .hpp files that contain allocations of GrowableArray. Much simpler, and I think probably better, is to keep GrowableArrayMetadata (perhaps under a different name - GrowableArrayArenaHolder?), now holding the allocation arena and providing init_checks (and anything else that seems appropriate), with init_checks defined similarly to the current definition in the .cpp file. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/17160#discussion_r1434655989 From dchuyko at openjdk.org Fri Dec 22 09:33:08 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Fri, 22 Dec 2023 09:33:08 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v15] In-Reply-To: References: Message-ID: <8HNg975y467K6xKiocEvZQkjHB6GHeBRM-oKXgrmxOo=.2d0233a3-1887-4c14-b158-27d1402ee659@github.com> On Wed, 20 Dec 2023 02:40:40 GMT, Andrei Pangin wrote: >> Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits: >> >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - ... and 23 more: https://git.openjdk.org/jdk/compare/fde5b168...44d680cd > > src/hotspot/share/code/codeCache.cpp line 1413: > >> 1411: ResourceMark rm; >> 1412: // Try the max level and let the directives be applied during the compilation. >> 1413: int complevel = CompLevel::CompLevel_full_optimization; > > Should the highest level depend on the configuration instead of the hard-coded constant? Perhaps, needs to be `highest_compile_level()` Yes, changed to use `highest_compile_level()`. > src/hotspot/share/compiler/compilerDirectives.cpp line 750: > >> 748: if (!dir->is_default_directive() && dir->match(method)) { >> 749: match_found = true; >> 750: break; > > `match_found` is redundant: for better readability, you may just return true. Curly braces around MutexLocker won't be needed either. Thanks, that's indeed simpler. > src/hotspot/share/oops/method.hpp line 820: > >> 818: // Clear the flags related to compiler directives that were set by the compilerBroker, >> 819: // because the directives can be updated. >> 820: void clear_method_flags() { > > The function name is a bit misleading - it clears only flags related to directives. Changed to `clear_directive_flags`. > src/hotspot/share/oops/methodFlags.hpp line 61: > >> 59: status(has_loops_flag_init , 1 << 14) /* The loop flag has been initialized */ \ >> 60: status(on_stack_flag , 1 << 15) /* RedefineClasses support to keep Metadata from being cleaned */ \ >> 61: status(has_matching_directives , 1 << 16) /* The method has matching directives */ \ > > It's worth noting that the flag is temporary and is valid only during DCmd execution. Good point, updated the comment. This btw means that in another places this flag can be reused. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1434883459 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1434884163 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1434884612 PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1434885291 From dchuyko at openjdk.org Fri Dec 22 09:33:06 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Fri, 22 Dec 2023 09:33:06 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v19] In-Reply-To: References: Message-ID: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request incrementally with one additional commit since the last revision: Deopt osr, cleanups ------------- Changes: - all: https://git.openjdk.org/jdk/pull/14111/files - new: https://git.openjdk.org/jdk/pull/14111/files/b348ebed..d75daf64 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=18 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=17-18 Stats: 41 lines in 4 files changed: 15 ins; 6 del; 20 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From dchuyko at openjdk.org Fri Dec 22 09:35:53 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Fri, 22 Dec 2023 09:35:53 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v15] In-Reply-To: References: Message-ID: On Wed, 20 Dec 2023 02:57:29 GMT, Andrei Pangin wrote: >> Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 33 commits: >> >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - Merge branch 'openjdk:master' into compiler-directives-force-update >> - ... and 23 more: https://git.openjdk.org/jdk/compare/fde5b168...44d680cd > > src/hotspot/share/code/codeCache.cpp line 1409: > >> 1407: while(iter.next()) { >> 1408: CompiledMethod* nm = iter.method(); >> 1409: methodHandle mh(thread, nm->method()); > > If there are two CompiledMethods for the same Java method, will it be scheduled for recompilation twice? Related question: if `nm` is an OSR method, does it make sense to go directly for deoptimization rather than compiling a non-OSR version? If there are multiple method versions, it will be recompiled several times. The alternative is too keep some additional information which may complicate the code. OSRs is a good catch, I changed their handling to deopt. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/14111#discussion_r1434887812 From sroy at openjdk.org Fri Dec 22 10:14:55 2023 From: sroy at openjdk.org (Suchismith Roy) Date: Fri, 22 Dec 2023 10:14:55 GMT Subject: RFR: JDK-8320005 : Allow loading of shared objects with .a extension on AIX [v7] In-Reply-To: <2sMyJ8mZ6EIULC67tK1IcI4uNnJMvpCzw1BKEDUaIms=.c90f1101-236e-4a80-869c-feca6abd3dc3@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <_WrW-iHHdU-IgC7Z1b6oe_Qh0dkC6P3KJAdl7J2S1Do=.712dd065-6207-4632-a82f-8e12ad023cd5@github.com> <2sMyJ8mZ6EIULC67tK1IcI4uNnJMvpCzw1BKEDUaIms=.c90f1101-236e-4a80-869c-feca6abd3dc3@github.com> Message-ID: On Thu, 21 Dec 2023 10:01:04 GMT, Thomas Stuefe wrote: > > > > > What happens if we accidentally attempt to load a "real" static library, which is also named *.a? Would dlopen() then crash? What would happen? > > > > > > > > > > I don't think the problem is with *.a . They would load as the default behaviour of the dlopen. It is only when the dlopen fails for *.so , we give another chance to check for .a file with the same name. > > > > > > > > > No, what I meant, and what must be clarified before going forward with this solution, is the following: > > > > > > * is _every_ `*.a` object on AIX loadable with `dlopen`, and will the result be the same as when loading a `*.so` object > > > * or, if we present arbitrary `*.a` files to dlopen, is there a chance for dlopen to crash or misbehave. > > > > > > Reason is that I was under the impression that *.a libraries are static libraries and cannot be loaded dynamically. This is what you now try to do. > > > If we cannot safely answer this question, I would opt for a more narrow solution by hard-wiring known alternative names. So, do the second *.a attempt only for your `ibm_16_am.a` which you know works. That could also be done in a reasonably maintainable manner. > > > > > > In AIX, both static and dynamic libraries have *.a extension. And AIX also supports *.so files.Bascially shared objects in AIX have both *.a and *.so extension. Hence we need to implement this logic. If we try loading a static archive specifically ,how the dlopen would behave , that is something probably @JoKern65 can answer ? > > Rather, this is a question you have to ask your collegues at IBM that develop the AIX libc. > > Since AIX libc is not open source, we cannot look for ourselves, nor can Joachim (her works at SAP). > > > > > > Does this really have to be handled in the OpenJDK? What does J9 on AIX do? Could this be done in a simpler way outside OpenJDK, e.g. by providing an *.so variant of the library in question? Where does this library come from? > > > > > > > > > > I am not sure how J9 handles this. I would have to consult . > > > > > > > > > J9 is Open Source, can't you just look? :) > > > > > > I did try comparing the file structures, and i do not see a similar file structure over there. I am unable to find the jvmTiAgent code and also os_aix file. So i am not sure which functions over there are doing the same functionality. You have any suggestion on how i can check and correlate ? > > Someone must implement LoadLibrary. Try looking for places where dlopen() is called. > > > > > However as per current observation, this issue does not show up on Semuru. This issue is only happening on Adoptium. The team that release these file has always released *.a files which work fine for Semuru. > > > > > > > > > I don't know what Semuru is. What is the context, is that a different VM? Also OpenJDK? J9 derived? > > > > > > Semuru is J9 derived. Ok , i was not able to find the right file yet. I will collaborate on this further once i am back from vacation, in January. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1867498645 From sspitsyn at openjdk.org Fri Dec 22 10:58:39 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 22 Dec 2023 10:58:39 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads [v3] In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Fri, 22 Dec 2023 01:43:07 GMT, Alex Menkov wrote: >> HeapDumper dumps virtual threads in 2 places: >> - dumping platform threads (mounted virtual threads are dumped as separate thread object); >> - dumping heap objects when the object is `java.lang.VirtualThread`. >> >> In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) >> Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. >> >> Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > feedback: comment added The fix looks good. Thank you for the update. Also, added a minor comment. src/hotspot/share/services/heapDumper.cpp line 1934: > 1932: // create a HPROF_GC_INSTANCE record for each object > 1933: DumperSupport::dump_instance(writer(), o, &_class_cache); > 1934: // If we encounter an unmounted virtual thread it needs to be dumped explicitly. Nit: It is nice to have a comment here. I'm thinking if it'd make sens to shortly explain why mounted threads are not dumped here. ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17134#pullrequestreview-1794318747 PR Review Comment: https://git.openjdk.org/jdk/pull/17134#discussion_r1434952089 From stefank at openjdk.org Fri Dec 22 13:24:37 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 22 Dec 2023 13:24:37 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: On Wed, 20 Dec 2023 21:11:09 GMT, Kim Barrett wrote: > I'm not a fan of the additional clutter in APIs that the static memory types add. If we had a variant of GrowableArrayCHeap that was not itself dynamically allocatable and took a memory type to use internally as a constructor argument, then I think a lot of that clutter could be eliminated. It could be used for ordinary data members that are direct GAs rather than pointers to GAs. I think there is a way to do something similar for static data members that are pointers that are dynamically allocated later, though that probably requires more work. FWIW, I added the GrowableArrayCHeap and the static memory type. I did that because there was a perceived need to minimize the memory usage, because we were going to use an extreme amount of these arrays for one of our subsystems in ZGC. It later turned out that we really didn't need to squeeze out the last bit of memory for that use-case. I would really like to get rid of the the static memory type from GrowableArrayCHeap, and just add it as an instance member. ------------- PR Comment: https://git.openjdk.org/jdk/pull/17160#issuecomment-1867683570 From epeter at openjdk.org Fri Dec 22 13:33:39 2023 From: epeter at openjdk.org (Emanuel Peter) Date: Fri, 22 Dec 2023 13:33:39 GMT Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: On Fri, 22 Dec 2023 13:22:19 GMT, Stefan Karlsson wrote: >> pre-existing: There are a lot of non-static class data members that are pointers to >> GrowableArray that seem like they would be better as direct, e.g. non-pointers. >> >> pre-existing: There are a lot of iterations over GrowableArray's that would be >> simplified by using range-based-for. >> >> I'm not a fan of the additional clutter in APIs that the static memory types add. >> If we had a variant of GrowableArrayCHeap that was not itself dynamically allocatable >> and took a memory type to use internally as a constructor argument, then I think a >> lot of that clutter could be eliminated. It could be used for ordinary data members >> that are direct GAs rather than pointers to GAs. I think there is a way to do something >> similar for static data members that are pointers that are dynamically allocated later, >> though that probably requires more work. >> >> I've not yet reviewed the changes to growableArray.[ch]pp yet, nor the test changes. >> But I've run out of time and energy for this for today. > >> I'm not a fan of the additional clutter in APIs that the static memory types add. If we had a variant of GrowableArrayCHeap that was not itself dynamically allocatable and took a memory type to use internally as a constructor argument, then I think a lot of that clutter could be eliminated. It could be used for ordinary data members that are direct GAs rather than pointers to GAs. I think there is a way to do something similar for static data members that are pointers that are dynamically allocated later, though that probably requires more work. > > FWIW, I added the GrowableArrayCHeap and the static memory type. I did that because there was a perceived need to minimize the memory usage, because we were going to use an extreme amount of these arrays for one of our subsystems in ZGC. It later turned out that we really didn't need to squeeze out the last bit of memory for that use-case. I would really like to get rid of the the static memory type from GrowableArrayCHeap, and just add it as an instance member. @stefank Maybe it would then make sense to do that before this change here? Because we will really have to touch all these changes here again for that. @kimbarrett @stefank @jdksjolen Or alternatively, we just say that we want to keep the C-Heap functionality inside `GrowableArray`, and simply remove `GrowableArrayCHeap`? Though I honestly would prefer a world where every allocation strategy has its own `GrowableArray*` variant, so that it is clear statically that they cannot be assigned to each other. So my preference would even be to have these: CHeapGrowableArray ArenaGrowableArray ResourceAreaGrowableArray What do you think? And how important is it that we do not use `virtual` with `GrowableArray`? Because the implementation and testing is much nastier without it. Is the V-table still such a overhead that we need to avoid it? ------------- PR Comment: https://git.openjdk.org/jdk/pull/17160#issuecomment-1867691744 From jkern at openjdk.org Fri Dec 22 14:50:16 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 22 Dec 2023 14:50:16 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v10] In-Reply-To: References: Message-ID: > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: additional fix of sideeffect reported in JDK-8322691 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/359080d3..dc2ea51b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=08-09 Stats: 44 lines in 1 file changed: 22 ins; 0 del; 22 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From jkern at openjdk.org Fri Dec 22 15:57:05 2023 From: jkern at openjdk.org (Joachim Kern) Date: Fri, 22 Dec 2023 15:57:05 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v11] In-Reply-To: References: Message-ID: <5LVpPG0ADhvALrcHghtZ95N1-SYcgrnrHl704svJStY=.e498d17a-3534-4ad9-8a74-004b40f7487a@github.com> > On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. > > This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). > > We propose a different, cleaner way of handling this: > > - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. > - Cache dl handles; repeated opening of a library should return the cached handle. > - Increase handle-local ref counter on open, Decrease it on close > - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). > > This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: No need for malloc ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16920/files - new: https://git.openjdk.org/jdk/pull/16920/files/dc2ea51b..acf306d4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16920&range=09-10 Stats: 28 lines in 1 file changed: 3 ins; 14 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/16920.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16920/head:pull/16920 PR: https://git.openjdk.org/jdk/pull/16920 From stuefe at openjdk.org Fri Dec 22 15:57:06 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 22 Dec 2023 15:57:06 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v10] In-Reply-To: References: Message-ID: On Fri, 22 Dec 2023 14:50:16 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > additional fix of sideeffect reported in JDK-8322691 src/hotspot/os/aix/porting_aix.cpp line 1071: > 1069: if (max_handletable == 0) { > 1070: // First time we allocate memory for 128 Entries > 1071: char* ptmp = (char*)::malloc(128 * sizeof(struct handletableentry)); No need for malloc. You can start with realloc, since realloc(NULL, ...) is malloc. static handletablentry* tab = nullptr; static unsigned max_handles = 0; ... if (need more handles) unsigned new_max = MAX2(max_handles * 2, init_num_handles); handleentry* new_tab = ::realloc(p_handletable, sizeof(handleentry) * new_max); if (new_tab != nullptr) { max_handles = new_max; tab= new_tab; } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1435139923 From amenkov at openjdk.org Fri Dec 22 20:52:48 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 22 Dec 2023 20:52:48 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads [v4] In-Reply-To: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: > HeapDumper dumps virtual threads in 2 places: > - dumping platform threads (mounted virtual threads are dumped as separate thread object); > - dumping heap objects when the object is `java.lang.VirtualThread`. > > In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) > Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. > > Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: feedback: extended comment ------------- Changes: - all: https://git.openjdk.org/jdk/pull/17134/files - new: https://git.openjdk.org/jdk/pull/17134/files/c6ad90ca..011ed399 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=17134&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=17134&range=02-03 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/17134.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/17134/head:pull/17134 PR: https://git.openjdk.org/jdk/pull/17134 From kim.barrett at oracle.com Fri Dec 22 23:22:24 2023 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 22 Dec 2023 23:22:24 +0000 Subject: RFR: 8322476: Remove GrowableArray C-Heap version, replace usages with GrowableArrayCHeap In-Reply-To: References: <7BF6OZ3vRH791MKbVeJqQ5foScHax_gLMFjSKkm3J68=.f29e5b1c-0751-4257-b253-261c6e20a7b9@github.com> <-cpYKTYXVhVfplPdXUKYt0ZcT-ql497vy0UZRrGREtk=.8fd5f503-ae50-4e8e-a5d8-65824465e5fd@github.com> Message-ID: <300914D8-7893-4859-A32F-3250CF7C3C84@oracle.com> [Kim Barret wrote:] >>> pre-existing: There are a lot of non-static class data members that are pointers to >>> GrowableArray that seem like they would be better as direct, e.g. non-pointers. >>> >>> pre-existing: There are a lot of iterations over GrowableArray's that would be >>> simplified by using range-based-for. >>> >>> I'm not a fan of the additional clutter in APIs that the static memory types add. >>> If we had a variant of GrowableArrayCHeap that was not itself dynamically allocatable >>> and took a memory type to use internally as a constructor argument, then I think a >>> lot of that clutter could be eliminated. It could be used for ordinary data members >>> that are direct GAs rather than pointers to GAs. I think there is a way to do something >>> similar for static data members that are pointers that are dynamically allocated later, >>> though that probably requires more work. >>> >> >> On Fri, 22 Dec 2023 13:22:19 GMT, Stefan Karlsson wrote: >> FWIW, I added the GrowableArrayCHeap and the static memory type. I did that because there was a perceived need to minimize the memory usage, because we were going to use an extreme amount of these arrays for one of our subsystems in ZGC. It later turned out that we really didn't need to squeeze out the last bit of memory for that use-case. I would really like to get rid of the the static memory type from GrowableArrayCHeap, and just add it as an instance member. Currently the basic overhead for a static-MEMTYPE GA is 16 bytes (data pointer, int length and capacity). So 16 bytes on 64bit platforms. Adding the memory type adds 8 bytes (due to alignment), so 50% increase. Though it may not matter for dynamically allocated GAs, since std::max_align_t is probably at least 32 bytes. Though I think we seriously overuse dynamic allocation for GAs. Does it really matter. I don't know. StringDedup uses a lot of GAs. (Though it could use 1/2 as many with some work, since they are used in pairs that always have the same length and capacity. At the time I was working on it I was feeling lazy and just used pairs of GAs, with the intention of coming back to that at some point if it proved to be a significant problem.) > On Dec 22, 2023, at 8:33 AM, Emanuel Peter wrote: > @stefank Maybe it would then make sense to do that before this change here? Because we will really have to touch all these changes here again for that. > > @kimbarrett @stefank @jdksjolen > Or alternatively, we just say that we want to keep the C-Heap functionality inside `GrowableArray`, and simply remove `GrowableArrayCHeap`? Though I honestly would prefer a world where every allocation strategy has its own `GrowableArray*` variant, so that it is clear statically that they cannot be assigned to each other. So my preference would even be to have these: > > CHeapGrowableArray > ArenaGrowableArray > ResourceAreaGrowableArray > > What do you think? I think having distinct CHeap, Resource, and Arena allocated types would be an improvement. If we were using std::vector with associated allocators we'd have something like that. (The allocator type is a template parameter for std::vector.) Refactoring GA in that direction is certainly possible, and might be an improvement. We could also have two variants of CHeap GAs, one with the memtype being a constructor argument (and possibly different from the memtype used to dynamically allocate the GA, as is currently the case, although I think that feature is never used), and a more compact one with a static memtype. I did some work on HotSpot allocators for standard containers like std::vector a while ago that could be applied to GA. It includes support for dynamic and static memtypes, and shows that isn't hard to do. I thought about whether this PR should go ahead without some of that stuff in place. It touches a lot of places that would probably be touched again if we went in that sort of direction. OTOH, a lot of these same places would probably need to be touched repeatedly anyway, one way or another. For example, dealing with what appears to be a large amount of unnecessary dynamic allocation of GAs would also touch a lot of these places, and I'm suspicious of combining the type changes and the allocation changes in one PR. > And how important is it that we do not use `virtual` with `GrowableArray`? Because the implementation and testing is much nastier without it. Is the V-table still such a overhead that we need to avoid it? I don't think there is any need for virtual functions in GA. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From sspitsyn at openjdk.org Sun Dec 24 11:06:51 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 24 Dec 2023 11:06:51 GMT Subject: RFR: JDK-8322237: Heap dump contains duplicate thread records for mounted virtual threads [v4] In-Reply-To: References: <8erTsH14ewZCVLtTZe0AZjGljqpum_VxBQQq2Shn-EE=.da5df507-2878-4775-bc07-f6d642283b19@github.com> Message-ID: On Fri, 22 Dec 2023 20:52:48 GMT, Alex Menkov wrote: >> HeapDumper dumps virtual threads in 2 places: >> - dumping platform threads (mounted virtual threads are dumped as separate thread object); >> - dumping heap objects when the object is `java.lang.VirtualThread`. >> >> In the 2nd case mounted virtual threads should be skipped (as they are already dumped with correct stack traces/stack references) >> Check that a virtual thread is mounted is non-trivial, method from JvmtiEnvBase was used for this. >> >> Testing: tier1..3, heapdump-related tests: open/test/hotspot/jtreg/serviceability,open/test/hotspot/jtreg/runtime/ErrorHandling,open/test/hotspot/jtreg/gc/epsilon,open/test/jdk/sun/tools/jhsdb > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > feedback: extended comment Thank you for the update! ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/17134#pullrequestreview-1795555865 From duke at openjdk.org Tue Dec 26 14:15:17 2023 From: duke at openjdk.org (Taizo Kurashige) Date: Tue, 26 Dec 2023 14:15:17 GMT Subject: RFR: 8313710: jcmd: typo in the documentation of JFR.start and JFR.dump [v3] In-Reply-To: References: Message-ID: > Hi, > > I fixed the typos for JFR.start and JFR.dump. > Acconding to issue's description, there is some typo in JFR.stop documentation, but I couldn't find that. I confirmed that there is no such typo in this repository. So I thought there was no need to fix JFR.stop documentation. > > I confirmed that the fixes are reflected and that all of the jdk_jfr tests pass. > > Could someone please review it? Taizo Kurashige has updated the pull request incrementally with one additional commit since the last revision: 8313710: jcmd: typo in the documentation of JFR.start and JFR.dump ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16413/files - new: https://git.openjdk.org/jdk/pull/16413/files/ec5b71ab..5fe59b39 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16413&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16413&range=01-02 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16413.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16413/head:pull/16413 PR: https://git.openjdk.org/jdk/pull/16413 From duke at openjdk.org Tue Dec 26 14:15:17 2023 From: duke at openjdk.org (Taizo Kurashige) Date: Tue, 26 Dec 2023 14:15:17 GMT Subject: RFR: 8313710: jcmd: typo in the documentation of JFR.start and JFR.dump In-Reply-To: References: Message-ID: On Thu, 30 Nov 2023 11:37:07 GMT, Erik Gahlin wrote: > > @egahlin > > > Looks like the added character and shift to the right will make the description exceed 80 characters in some lines, for example "included" for the begin option. > > > > > > I'm sorry, I don't know from which letter to which letter is more than 80 characters, for example in line of begin option. Can you tell me that? Also, I didn't know the rule that the number of characters per line should not exceed 80. If possible, could you please tell me where such a rule is stated? > > You could look at line 319 in DCmdStart.java and 198 in DcmdDump.java. Make sure characters don't exceed. "...1234567890" Sorry for the late response. Thanks, I understand that characters should not exceed "... 1234567890" (88 characters) . I fixed line 204 in DcmdDump.java. In addition, the starting position of "...1234567890" in DcmdStart.java does not match the beginning of "Syntax :..", and it is hard to see visually whether characters exceed "...1234567890". So I fixed it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16413#issuecomment-1869571223 From duke at openjdk.org Tue Dec 26 14:15:18 2023 From: duke at openjdk.org (Taizo Kurashige) Date: Tue, 26 Dec 2023 14:15:18 GMT Subject: RFR: 8313710: jcmd: typo in the documentation of JFR.start and JFR.dump [v3] In-Reply-To: References: Message-ID: On Tue, 28 Nov 2023 07:44:27 GMT, David Holmes wrote: >> Thank you for telling me about how to jcmd.1 nroff file is managed. >> I will split this change into separate subtasks. Then if one of the Oracle engineers finds that subtask, will he/she work on it like any other bug? > > Yes it will be assigned and addressed. Thanks Sorry for the late response. I will ask my colleague to make it, so please wait a moment. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16413#discussion_r1436476251 From mdoerr at openjdk.org Wed Dec 27 10:52:59 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 27 Dec 2023 10:52:59 GMT Subject: RFR: JDK-8320890: [AIX] Find a better way to mimic dl handle equality [v11] In-Reply-To: <5LVpPG0ADhvALrcHghtZ95N1-SYcgrnrHl704svJStY=.e498d17a-3534-4ad9-8a74-004b40f7487a@github.com> References: <5LVpPG0ADhvALrcHghtZ95N1-SYcgrnrHl704svJStY=.e498d17a-3534-4ad9-8a74-004b40f7487a@github.com> Message-ID: On Fri, 22 Dec 2023 15:57:05 GMT, Joachim Kern wrote: >> On AIX, repeated calls to dlopen referring to the same shared library may result in different, unique dl handles to be returned from libc. In that it differs from typical libc implementations that cache dl handles. >> >> This causes problems in the JVM with code that assumes equality of handles. One such problem is in the JVMTI agent handler. That problem was fixed with a local fix to said handler ([JDK-8315706](https://bugs.openjdk.org/browse/JDK-8315706)). However, this fix causes follow-up problems since it assumes that the file name passed to `os::dll_load()` is the file that has been opened. It prevents efficient, os_aix.cpp-local workarounds for other AIX issues like the *.so/*.a duality. See [JDK-8320005](https://bugs.openjdk.org/browse/JDK-8320005). As such, it is a hack that causes other, more uglier hacks to follow (see discussion of https://github.com/openjdk/jdk/pull/16604). >> >> We propose a different, cleaner way of handling this: >> >> - Handle this entirely inside the AIX versions of os::dll_load and os::dll_unload. >> - Cache dl handles; repeated opening of a library should return the cached handle. >> - Increase handle-local ref counter on open, Decrease it on close >> - Make sure calls to os::dll_load are matched to os::dll_unload (See [JDK-8320830](https://bugs.openjdk.org/browse/JDK-8320830)). >> >> This way we mimic dl handle equality as it is implemented on other platforms, and this works for all callers of os::dll_load. > > Joachim Kern has updated the pull request incrementally with one additional commit since the last revision: > > No need for malloc src/hotspot/os/aix/porting_aix.cpp line 975: > 973: return false; > 974: > 975: char* path2 = os::strdup (path); Whitespace between `os::strdup` and `(path)`. src/hotspot/os/aix/porting_aix.cpp line 1019: > 1017: } > 1018: > 1019: char* libpath = os::strdup (Libpath.base()); Whitespace! ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1436947479 PR Review Comment: https://git.openjdk.org/jdk/pull/16920#discussion_r1436947817 From apangin at openjdk.org Thu Dec 28 00:25:48 2023 From: apangin at openjdk.org (Andrei Pangin) Date: Thu, 28 Dec 2023 00:25:48 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v19] In-Reply-To: References: Message-ID: On Fri, 22 Dec 2023 09:33:06 GMT, Dmitry Chuyko wrote: >> Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. >> >> A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. >> >> It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). >> >> Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. >> >> A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. >> >> In addition, a new diagnostic command `Compiler.replace_directives... > > Dmitry Chuyko has updated the pull request incrementally with one additional commit since the last revision: > > Deopt osr, cleanups The logic looks good to me now, thanks. ------------- Marked as reviewed by apangin (no project role). PR Review: https://git.openjdk.org/jdk/pull/14111#pullrequestreview-1797576900