From stuefe at openjdk.org Wed Nov 1 06:05:05 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 1 Nov 2023 06:05:05 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: <6C1LXsILE-RYGk4Nbaoa5xQH_04T7dRpHwuHXqOJDx4=.bfb6bc7e-11f3-454f-9aa2-f1368654567a@github.com> References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> <6C1LXsILE-RYGk4Nbaoa5xQH_04T7dRpHwuHXqOJDx4=.bfb6bc7e-11f3-454f-9aa2-f1368654567a@github.com> Message-ID: On Mon, 30 Oct 2023 10:29:56 GMT, Johan Sj?len wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > src/hotspot/os/linux/memMapPrinter_linux.cpp line 80: > >> 78: FILE* f = os::fopen("/proc/self/maps", "r"); >> 79: if (f != nullptr) { >> 80: static constexpr size_t linesize = sizeof(ProcMapsInfo); > > Make this `constexpr const`, remove `static`. Lets go with constexpr; should imply const. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378423904 From stuefe at openjdk.org Wed Nov 1 06:09:04 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 1 Nov 2023 06:09:04 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Mon, 30 Oct 2023 17:31:30 GMT, Gerard Ziemski wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > src/hotspot/os/linux/memMapPrinter_linux.cpp line 83: > >> 81: char line[linesize]; >> 82: while(fgets(line, sizeof(line), f) == line) { >> 83: line[sizeof(line) - 1] = '\0'; > > What would happen if the read line is empty, i.e. sizeof(line)==0, can that happen? sizeof(line) is the size of the character array, not strlen. It is linesize bytes. `x[sizeof(x) - 1] = '\0';` is the typical pattern one uses to ensure a character array is zero-terminated. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378425890 From stuefe at openjdk.org Wed Nov 1 06:30:04 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 1 Nov 2023 06:30:04 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Tue, 31 Oct 2023 16:58:19 GMT, Gerard Ziemski wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > src/hotspot/os/linux/memMapPrinter_linux.cpp line 59: > >> 57: void print_OS_specific_details(outputStream* st) const override { >> 58: st->print("%s %s ", _info.prot, _info.offset); >> 59: } > > If that's all this is doing, do we really need it? This prints protection and offset. The former is interesting for obvious reasons (e.g. lets you tell apart stack guard regions from stack, or uncommitted from committed regions) and the latter interesting for some corner cases (e.g. ZGC mapped regions). I don't want to do this in shared code though since I am not sure every OS has this information or whether I can obtain this information. I rather not print out a bunch of 0s or empty strings. The alternative would be carrying a non-descriptive text string around with "OS information", but I don't want to have to think about storage and copying stuff around. It would not be simpler than this. BTW, the way to implement this on MacOS would be probably by spawning off `vmmap` and parsing the output; on Windows one would use `VirtualQuery`. But I leave this for other folks, I am happy with Linux for now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378436209 From stuefe at openjdk.org Wed Nov 1 07:38:08 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 1 Nov 2023 07:38:08 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: <1CItO6JntzhVYf09HjjGLUAOBtFx8J3ZToTM5nvhGpg=.76f36955-92df-4379-9a5b-f6eac683d469@github.com> On Tue, 31 Oct 2023 17:08:15 GMT, Gerard Ziemski wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > src/hotspot/share/nmt/memMapPrinter.cpp line 79: > >> 77: const void* const min = MAX2(from1, from2); >> 78: const void* const max = MIN2(to1, to2); >> 79: return min < max; > > I had to rewrite it as: > > `return MAX2(from1, from2) < MIN2(to1, to2);` > > to make sure I understand it, or better yet, why not have it as a macros? > > `#define RANGE_INTERSECT(min, max) (return MAX2(from1, from2) < MIN2(to1, to2))` > > MAX2 and MIN2 are macros already and we're not doing anything fancy here. I'll do the former, that is clearer I agree, but leave the latter out (I assume with the macro you mean add it to globalDefenitions.hpp). I fear that a lot of bikeshedding and general discussions would start, and to do it properly needs a bit more time. Because we really would benefit from having a nice templatized utility classes for ranges. Like MemRegion, but that one is tied to HeapWord* I think. > src/hotspot/share/nmt/memMapPrinter.cpp line 89: > >> 87: // NMT deadlocks (ThreadCritical). >> 88: Range* _ranges; >> 89: MEMFLAGS* _flags; > > Why did you decide to have two separate memory chunks? > > I think I'd have used a struct to hold Range* and MEMFLAGS* and use that struct in a single array. Two reasons: - more condensed, needs less memory. MEMFLAGS is a byte, Range is 16 bytes. A combined structure would be 24 bytes with padding (since the structure needs to be pointer aligned). We'd loose 7 bytes. - It is more cache friendly, hence faster. I need to iterate through all ranges, but only need to access the MEMFLAGS for the one range I find. By keeping the ranges condensed, more of them fit into a cache line, so iteration is faster. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378475306 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378476876 From sjohanss at openjdk.org Wed Nov 1 09:37:08 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 1 Nov 2023 09:37:08 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: References: Message-ID: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> On Tue, 31 Oct 2023 04:23:13 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Replace NULL with nullptr Sorry for being a bit late to this PR. I think the addition of CPU time tracking is good, but I wonder if we could do it in a way that is a bit more general. A more general way of tracking CPU time for a set of threads and we could then have different consumers of this data. In addition to hsperf counters I think having logging and JFR events for this could be interesting as well. Have you had any thought along those lines, any obvious problems with such approach? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1788665727 From jsjolen at openjdk.org Wed Nov 1 09:41:05 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 1 Nov 2023 09:41:05 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: <1CItO6JntzhVYf09HjjGLUAOBtFx8J3ZToTM5nvhGpg=.76f36955-92df-4379-9a5b-f6eac683d469@github.com> References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> <1CItO6JntzhVYf09HjjGLUAOBtFx8J3ZToTM5nvhGpg=.76f36955-92df-4379-9a5b-f6eac683d469@github.com> Message-ID: On Wed, 1 Nov 2023 07:32:23 GMT, Thomas Stuefe wrote: >> src/hotspot/share/nmt/memMapPrinter.cpp line 79: >> >>> 77: const void* const min = MAX2(from1, from2); >>> 78: const void* const max = MIN2(to1, to2); >>> 79: return min < max; >> >> I had to rewrite it as: >> >> `return MAX2(from1, from2) < MIN2(to1, to2);` >> >> to make sure I understand it, or better yet, why not have it as a macros? >> >> `#define RANGE_INTERSECT(min, max) (return MAX2(from1, from2) < MIN2(to1, to2))` >> >> MAX2 and MIN2 are macros already and we're not doing anything fancy here. > > I'll do the former, that is clearer I agree, but leave the latter out (I assume with the macro you mean add it to globalDefenitions.hpp). > > I fear that a lot of bikeshedding and general discussions would start, and to do it properly needs a bit more time. Because we really would benefit from having a nice templatized utility classes for ranges. Like MemRegion, but that one is tied to HeapWord* I think. `MAX2` and `MIN2` are not macros, they're `constexpr` free functions, and that's what a `RANGE_INTERSECT` free function should be also. Unexpected because of their names, I know. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378579302 From jsjolen at openjdk.org Wed Nov 1 09:41:05 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 1 Nov 2023 09:41:05 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> <1CItO6JntzhVYf09HjjGLUAOBtFx8J3ZToTM5nvhGpg=.76f36955-92df-4379-9a5b-f6eac683d469@github.com> Message-ID: On Wed, 1 Nov 2023 09:37:22 GMT, Johan Sj?len wrote: >> I'll do the former, that is clearer I agree, but leave the latter out (I assume with the macro you mean add it to globalDefenitions.hpp). >> >> I fear that a lot of bikeshedding and general discussions would start, and to do it properly needs a bit more time. Because we really would benefit from having a nice templatized utility classes for ranges. Like MemRegion, but that one is tied to HeapWord* I think. > > `MAX2` and `MIN2` are not macros, they're `constexpr` free functions, and that's what a `RANGE_INTERSECT` free function should be also. Unexpected because of their names, I know. FWIW: Yeah, if it's going to be in globalDefinitions.hpp then do that in a separate RFE. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378579763 From iwalulya at openjdk.org Wed Nov 1 09:42:02 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 1 Nov 2023 09:42:02 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v5] In-Reply-To: References: Message-ID: On Tue, 31 Oct 2023 19:14:13 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Fix compilation src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp line 266: > 264: > 265: inline void G1CollectedHeap::pin_object(JavaThread* thread, oop obj) { > 266: assert(obj != NULL, "obj must not be null"); please update the `NULL`s to `nullptr` ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1378580763 From jsjolen at openjdk.org Wed Nov 1 09:52:08 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 1 Nov 2023 09:52:08 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Sat, 28 Oct 2023 13:04:05 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > fix various builds src/hotspot/share/nmt/memMapPrinter.cpp line 99: > 97: } > 98: > 99: bool add(const void* from, const void* to, MEMFLAGS f) { Please mention that we're `add`ing in sorted order, that is that `forall R \in _ranges: R.to <= from` holds. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378589809 From stuefe at openjdk.org Wed Nov 1 09:55:06 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 1 Nov 2023 09:55:06 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Wed, 1 Nov 2023 09:48:39 GMT, Johan Sj?len wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > src/hotspot/share/nmt/memMapPrinter.cpp line 99: > >> 97: } >> 98: >> 99: bool add(const void* from, const void* to, MEMFLAGS f) { > > Please mention that we're `add`ing in sorted order, that is that `forall R \in _ranges: R.to <= from` holds. I wasn't sure about that. Do we always? Are NMT regions guaranteed to be sorted? (Dimly remember cases where that weren't so) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378593459 From stuefe at openjdk.org Wed Nov 1 10:09:40 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 1 Nov 2023 10:09:40 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v7] In-Reply-To: References: Message-ID: <8tOkabtdC8K8xjHBVgKio1y14oJ4VAJPSuYFFTUs0xI=.f6363762-2997-4ceb-822a-b92695db760f@github.com> > Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. > > Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. > > That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. > > Example output (from a spring petstore run): > > [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) > > This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. Thomas Stuefe has updated the pull request incrementally with three additional commits since the last revision: - Provide both print and dump command - Feedback Gerard - Feedbacj Johan ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16301/files - new: https://git.openjdk.org/jdk/pull/16301/files/a3506fd6..3829abbf Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=05-06 Stats: 200 lines in 9 files changed: 164 ins; 7 del; 29 mod Patch: https://git.openjdk.org/jdk/pull/16301.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16301/head:pull/16301 PR: https://git.openjdk.org/jdk/pull/16301 From jsjolen at openjdk.org Wed Nov 1 10:09:40 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 1 Nov 2023 10:09:40 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Sat, 28 Oct 2023 13:04:05 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > fix various builds OK, went through the cache. Will continue later. src/hotspot/share/nmt/memMapPrinter.cpp line 105: > 103: _ranges[_count - 1].to = to; > 104: return true; > 105: } I'm pretty sure that the virtual memory tracker already gives you the minimal set of regions, have you observed this branch being taken? src/hotspot/share/nmt/memMapPrinter.cpp line 119: > 117: assert(_capacity > _count, "Sanity"); > 118: _ranges[_count].from = from; > 119: _ranges[_count].to = to; Or just `_ranges[_count] = Range{from, to}` ------------- PR Review: https://git.openjdk.org/jdk/pull/16301#pullrequestreview-1707901319 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378602552 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378604692 From jsjolen at openjdk.org Wed Nov 1 10:12:06 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 1 Nov 2023 10:12:06 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: <6TtseorItrazZpn9oSLplBh78aFmOzrsCeRTtBBGe-I=.78ed35ac-f772-4941-9c74-4cf3c33efdac@github.com> On Wed, 1 Nov 2023 09:52:24 GMT, Thomas Stuefe wrote: >> src/hotspot/share/nmt/memMapPrinter.cpp line 99: >> >>> 97: } >>> 98: >>> 99: bool add(const void* from, const void* to, MEMFLAGS f) { >> >> Please mention that we're `add`ing in sorted order, that is that `forall R \in _ranges: R.to <= from` holds. > > I wasn't sure about that. Do we always? Are NMT regions guaranteed to be sorted? (Dimly remember cases where that weren't so) It's always sorted in some way, we save the linked lists in some different sorting orders for baselining, but in the `VirtualMemoryTracker` they're guaranteed to be sorted by address. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378609954 From stuefe at openjdk.org Wed Nov 1 10:16:09 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 1 Nov 2023 10:16:09 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Wed, 1 Nov 2023 10:01:43 GMT, Johan Sj?len wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > src/hotspot/share/nmt/memMapPrinter.cpp line 105: > >> 103: _ranges[_count - 1].to = to; >> 104: return true; >> 105: } > > I'm pretty sure that the virtual memory tracker already gives you the minimal set of regions, have you observed this branch being taken? Yes, it resulted in a significant performance gain for a test run where I interleaved non-committed and committed memory of the same tag. I may have been mistaken, of course. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378613209 From jsjolen at openjdk.org Wed Nov 1 10:49:05 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 1 Nov 2023 10:49:05 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: <9KmsnKM_nlIfD2gtEkUGSVXy7Zo3mx9xMuJCO_pucxE=.84d0979c-9454-4b2d-a7b7-06cf1170a2b2@github.com> On Wed, 1 Nov 2023 10:13:03 GMT, Thomas Stuefe wrote: >> src/hotspot/share/nmt/memMapPrinter.cpp line 105: >> >>> 103: _ranges[_count - 1].to = to; >>> 104: return true; >>> 105: } >> >> I'm pretty sure that the virtual memory tracker already gives you the minimal set of regions, have you observed this branch being taken? > > Yes, it resulted in a significant performance gain for a test run where I interleaved non-committed and committed memory of the same tag. I may have been mistaken, of course. It being committed or reserved shouldn't matter (I assume non-committed = reserved), as committed just means that there's a pointer to the committed memory region in the reserved memory region. Well, all I can say is that I'm surprised. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1378643276 From gziemski at openjdk.org Wed Nov 1 15:41:08 2023 From: gziemski at openjdk.org (Gerard Ziemski) Date: Wed, 1 Nov 2023 15:41:08 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v7] In-Reply-To: <8tOkabtdC8K8xjHBVgKio1y14oJ4VAJPSuYFFTUs0xI=.f6363762-2997-4ceb-822a-b92695db760f@github.com> References: <8tOkabtdC8K8xjHBVgKio1y14oJ4VAJPSuYFFTUs0xI=.f6363762-2997-4ceb-822a-b92695db760f@github.com> Message-ID: On Wed, 1 Nov 2023 10:09:40 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with three additional commits since the last revision: > > - Provide both print and dump command > - Feedback Gerard > - Feedbacj Johan LGTM Thank you for doing this! ------------- Marked as reviewed by gziemski (Committer). PR Review: https://git.openjdk.org/jdk/pull/16301#pullrequestreview-1708486866 From kevinw at openjdk.org Wed Nov 1 18:16:40 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Wed, 1 Nov 2023 18:16:40 GMT Subject: RFR: 8319238: JMX ThreadPoolAccTest.java is too verbose and should fail before timeout Message-ID: Discovered while testing changes that made this test fail. The test failure is hard to diagnose as it logs and retries at full speed, possibly forever, until timeout. This can hit a log file limit. We can save thousands of lines of text being printed when the test runs normally and successfully, by waiting half a second before doing the Principal-checking which is the purpose of the test. ------------- Commit messages: - 8319238: JMX ThreadPoolAccTest.java is too verbose and should fail before timeout Changes: https://git.openjdk.org/jdk/pull/16456/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16456&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319238 Stats: 26 lines in 1 file changed: 23 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16456.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16456/head:pull/16456 PR: https://git.openjdk.org/jdk/pull/16456 From sspitsyn at openjdk.org Wed Nov 1 18:49:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 1 Nov 2023 18:49:11 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads Message-ID: The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. At the low level, the JVMTI code supporting platform and virtual threads still can be different. This implementation is based on the `JvmtiVTMSTransitionDisabler` class. The interface includes, at least, two new classes: - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` Testing: - the mach5 tiers 1-6 are all passed ------------- Commit messages: - 8319244: implement JVMTI handshakes support for virtual threads Changes: https://git.openjdk.org/jdk/pull/16460/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319244 Stats: 532 lines in 5 files changed: 176 ins; 329 del; 27 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From amenkov at openjdk.org Wed Nov 1 19:40:09 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 1 Nov 2023 19:40:09 GMT Subject: RFR: JDK-8319053: Segment dump files remain after parallel heap dump on Windows Message-ID: Segment file is closed from DumpWriter dtor. On Unix systems `remove` can delete an opened file, on Windows it fails with "access denied". The fix destroys DumpWriter objects for segment files after all data are dumped ------------- Commit messages: - jcheck - fix - fix Changes: https://git.openjdk.org/jdk/pull/16462/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16462&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319053 Stats: 22 lines in 2 files changed: 17 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16462.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16462/head:pull/16462 PR: https://git.openjdk.org/jdk/pull/16462 From jjoo at openjdk.org Wed Nov 1 19:50:45 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 1 Nov 2023 19:50:45 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v36] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 36 additional commits since the last revision: - Merge branch 'openjdk:master' into master - Replace NULL with nullptr - Implement hsperf counter for G1ServiceThread - Remove StringDedup from GC thread list - Use 64-bit atomic add for incrementing counters - Merge branch 'openjdk:master' into master - Add call to publish in parallel gc and update counter names - Add Copyright header to test and formatting changes - Fix test - add comment and change if defined to ifdef - ... and 26 more: https://git.openjdk.org/jdk/compare/d748dccb...2446149c ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/be104e16..2446149c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=35 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=34-35 Stats: 27809 lines in 1182 files changed: 15264 ins; 4162 del; 8383 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Wed Nov 1 23:56:25 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 1 Nov 2023 23:56:25 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v37] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - revert gitignore change - Attempt to fix broken test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/2446149c..9fb36a9e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=36 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=35-36 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 2 01:24:08 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 2 Nov 2023 01:24:08 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> Message-ID: On Wed, 1 Nov 2023 09:34:01 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Replace NULL with nullptr > > Sorry for being a bit late to this PR. I think the addition of CPU time tracking is good, but I wonder if we could do it in a way that is a bit more general. A more general way of tracking CPU time for a set of threads and we could then have different consumers of this data. In addition to hsperf counters I think having logging and JFR events for this could be interesting as well. > > Have you had any thought along those lines, any obvious problems with such approach? @kstefanj thank you for taking a look! caoman@ and I discussed your comment and our thoughts are below: > A more general way of tracking CPU time for a set of threads and we could then have different consumers of this data. Could you elaborate a bit on what you were thinking of here? If we are assuming something like a thread that updates all other threads, I think this implementation could get a bit complicated. There are two main issues that we can see with a generic thread approach: 1. We would have to figure out how often to pull metrics from the various gc threads from the central thread, and possibly determine this frequency separately for every thread. Instead with our current implementation, we can manually trigger publishes based on when the GC thread is done doing work. 2. We would still need to tag each thread we want to track somewhere, and keep track of a mapping from thread to its counter name, etc. which doesn't seem to simplify things too much. (I imagine we will still need to touch numerous files to "tag" each thread with whether we want to track it or not?) The existing `sun.management:type=HotspotThreading` MBean approach (discussed [here]( https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) could be another general way to track CPU. However, the discussion concludes that it is an internal API, and discourages users from using it. > In addition to hsperf counters I think having logging and JFR events for this could be interesting as well. There's actually already an existing implementation that covers process CPU from GC pauses, but it doesn't handle concurrent work: https://bugs.openjdk.org/browse/JDK-8291753. However, for our implementation of AHS, we decided against a JFR-based approach, as it has a bit more overhead and the hsperf-based implementation seemed simpler. So while a JFR-based approach to track these counters might be feasible, we believe it would be considerably more work, and could be done in a separate PR later if there is sufficient user interest. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1789914128 From dholmes at openjdk.org Thu Nov 2 06:18:00 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 2 Nov 2023 06:18:00 GMT Subject: RFR: JDK-8319053: Segment dump files remain after parallel heap dump on Windows In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 19:23:14 GMT, Alex Menkov wrote: > Segment file is closed from DumpWriter dtor. > On Unix systems `remove` can delete an opened file, on Windows it fails with "access denied". > The fix destroys DumpWriter objects for segment files after all data are dumped Sorry I don't understand the fix here. You changed the AbstractDumpWriter to be a C-heap object instead of ResourceObject and then delete it, so invoking a destructor, which AFAICS does nothing in terms of removing any files. ??? BTW with this change you may be able to remove the ResourceMark prior to allocating `local_writer` (line 2350). src/hotspot/share/services/heapDumper.cpp line 1959: > 1957: // Delete selected segmented heap file nevertheless > 1958: if (remove(path) != 0) { > 1959: log_info(heapdump)("Remove segment file (%d) failed (%d)", i, errno); Suggestion: "Removal of segment file ..." ------------- PR Review: https://git.openjdk.org/jdk/pull/16462#pullrequestreview-1709514524 PR Review Comment: https://git.openjdk.org/jdk/pull/16462#discussion_r1379635376 From tschatzl at openjdk.org Thu Nov 2 08:33:46 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 08:33:46 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v6] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: NULL -> nullptr ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/e5dfbb73..fb1deac4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=04-05 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From mbaesken at openjdk.org Thu Nov 2 10:09:13 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 2 Nov 2023 10:09:13 GMT Subject: RFR: JDK-8318957: enhance agentlib:jdwp help output by info about allow option [v2] In-Reply-To: References: Message-ID: <9Djxt6_rtquouSPP_w7UJpE8elauxJjEGqLbmhAYIug=.a5b165e6-c6a6-4868-8407-ee5f1cc14484@github.com> On Mon, 30 Oct 2023 08:55:55 GMT, Matthias Baesken wrote: >> The allow option of agentlib:jdwp has been introduced a long time ago (see JDK-8061228) and is documented here : >> https://docs.oracle.com/en/java/javase/17/docs/specs/jpda/conninv.html#oracle-vm-invocation-options >> However it is still missing in the agentlib:jdwp help output and should be added there too. > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > More info about address lists Hi Chris, thanks for the review ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16393#issuecomment-1790426332 From mbaesken at openjdk.org Thu Nov 2 10:09:14 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 2 Nov 2023 10:09:14 GMT Subject: Integrated: JDK-8318957: enhance agentlib:jdwp help output by info about allow option In-Reply-To: References: Message-ID: On Fri, 27 Oct 2023 08:19:13 GMT, Matthias Baesken wrote: > The allow option of agentlib:jdwp has been introduced a long time ago (see JDK-8061228) and is documented here : > https://docs.oracle.com/en/java/javase/17/docs/specs/jpda/conninv.html#oracle-vm-invocation-options > However it is still missing in the agentlib:jdwp help output and should be added there too. This pull request has now been integrated. Changeset: 53bb7cd4 Author: Matthias Baesken URL: https://git.openjdk.org/jdk/commit/53bb7cd415f1d2e87d8f06ad5eb611bfdf8ef0ad Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod 8318957: enhance agentlib:jdwp help output by info about allow option Reviewed-by: cjplummer ------------- PR: https://git.openjdk.org/jdk/pull/16393 From simonis at openjdk.org Thu Nov 2 10:46:11 2023 From: simonis at openjdk.org (Volker Simonis) Date: Thu, 2 Nov 2023 10:46:11 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v37] In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 23:56:25 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - revert gitignore change > - Attempt to fix broken test @kstefanj , it is a pity that the `sun.management:type=HotspotThreading` MBean is not exported any more. If we move that or a similar functionality to a new MBean under `com.sun.management` (as proposed in the [cited discussion](https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) then we might reuse these new hsperf counters in the same way this is already done by some other MBeans which already use hsperf counters as their information source. I think logging or JFR functionality could also easily be implemented on top of the new hsperf counters. As for @albertnetymk's proposal to simple use the `/proc` file system to retrieve thread CPU time information, that is not only inconvenient for the reasons detailed by @caoman. It will also not work for compiler threads (which I think should have similar counters in the future) because compiler threads can be created and and destroyed dynamically (due to [`-XX:+UseDynamicNumberOfCompilerThreads`](https://bugs.openjdk.org/browse/JDK-8198756)). The current PR still looks good to me. ------------- Marked as reviewed by simonis (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1709943605 From jbechberger at openjdk.org Thu Nov 2 11:23:29 2023 From: jbechberger at openjdk.org (Johannes Bechberger) Date: Thu, 2 Nov 2023 11:23:29 GMT Subject: RFR: JDK-8318736: com/sun/jdi/JdwpOnThrowTest.java failed with "transport error 202: bind failed: Address already in use" [v4] In-Reply-To: References: Message-ID: > Fix race condition in debugger port selection, introduced with [JDK-8317920](https://bugs.openjdk.org/browse/JDK-8317920). > > Tested on my Mac M1, but it doesn't contain platform-dependent code. Johannes Bechberger has updated the pull request incrementally with one additional commit since the last revision: Update test/jdk/com/sun/jdi/lib/jdb/Debuggee.java Co-authored-by: Alex Menkov <69548902+alexmenkov at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16358/files - new: https://git.openjdk.org/jdk/pull/16358/files/ff651a94..8788131e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16358&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16358&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16358.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16358/head:pull/16358 PR: https://git.openjdk.org/jdk/pull/16358 From jbechberger at openjdk.org Thu Nov 2 11:35:29 2023 From: jbechberger at openjdk.org (Johannes Bechberger) Date: Thu, 2 Nov 2023 11:35:29 GMT Subject: RFR: JDK-8318736: com/sun/jdi/JdwpOnThrowTest.java failed with "transport error 202: bind failed: Address already in use" [v5] In-Reply-To: References: Message-ID: > Fix race condition in debugger port selection, introduced with [JDK-8317920](https://bugs.openjdk.org/browse/JDK-8317920). > > Tested on my Mac M1, but it doesn't contain platform-dependent code. Johannes Bechberger has updated the pull request incrementally with one additional commit since the last revision: Refactor as requested ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16358/files - new: https://git.openjdk.org/jdk/pull/16358/files/8788131e..8a9b7133 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16358&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16358&range=03-04 Stats: 36 lines in 1 file changed: 14 ins; 16 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/16358.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16358/head:pull/16358 PR: https://git.openjdk.org/jdk/pull/16358 From tschatzl at openjdk.org Thu Nov 2 11:42:54 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 11:42:54 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v7] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/fb1deac4..73f61da9 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=05-06 Stats: 180 lines in 18 files changed: 54 ins; 28 del; 98 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Thu Nov 2 11:46:02 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 11:46:02 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v4] In-Reply-To: References: <8BH2UtnHn-DYz3c80Su4v9BF_v0w-N4fHkASCXP_E2c=.70c7ff8f-32e2-4970-87e3-fe22f7b08e6b@github.com> Message-ID: On Tue, 31 Oct 2023 18:54:26 GMT, Thomas Schatzl wrote: > Had a discussion with @albertnetymk and we came to the following agreement about naming: > >"allocation failure" - allocation failed in the to-space due to memory exhaustion >"pinned" - the region/object has been pinned >"evacuation failure" - either pinned or allocation failure > >I will apply this new naming asap. Done. I left out the `G1EvacFailureInjector` (it only injects allocation failures, not evacuation failures) related renamings as this adds lots of noise (including the debug options). I'll file a follow-up and assign it to me. Tier1 seems to pass, will redo upper tiers again. The only noteworthy externally visible change is that the `(Evacuation Failure)` tag in log messages is now `(Allocation Failure)`. I did not want combinations of `(Evacuation Failure)` and additionally `(Pinned) (Allocation Failure)`, but maybe it is fine, or just fine to keep only `(Evacuation Failure)` as before and assume that users enable higher level logging to find out details. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16342#issuecomment-1790570048 From stuefe at openjdk.org Thu Nov 2 12:35:06 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 2 Nov 2023 12:35:06 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: <9KmsnKM_nlIfD2gtEkUGSVXy7Zo3mx9xMuJCO_pucxE=.84d0979c-9454-4b2d-a7b7-06cf1170a2b2@github.com> References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> <9KmsnKM_nlIfD2gtEkUGSVXy7Zo3mx9xMuJCO_pucxE=.84d0979c-9454-4b2d-a7b7-06cf1170a2b2@github.com> Message-ID: On Wed, 1 Nov 2023 10:46:21 GMT, Johan Sj?len wrote: >> Yes, it resulted in a significant performance gain for a test run where I interleaved non-committed and committed memory of the same tag. I may have been mistaken, of course. > > It being committed or reserved shouldn't matter (I assume non-committed = reserved), as committed just means that there's a pointer to the committed memory region in the reserved memory region. Well, all I can say is that I'm surprised. They differ by permissions. I guess NMT keeps them separate for that reason. In any case, the code snippet makes a huge difference. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1380031435 From stuefe at openjdk.org Thu Nov 2 12:35:09 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 2 Nov 2023 12:35:09 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Wed, 1 Nov 2023 10:04:04 GMT, Johan Sj?len wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > src/hotspot/share/nmt/memMapPrinter.cpp line 119: > >> 117: assert(_capacity > _count, "Sanity"); >> 118: _ranges[_count].from = from; >> 119: _ranges[_count].to = to; > > Or just `_ranges[_count] = Range{from, to}` ok ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1380031671 From stuefe at openjdk.org Thu Nov 2 12:52:38 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 2 Nov 2023 12:52:38 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v8] In-Reply-To: References: Message-ID: > Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. > > Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. > > That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. > > Example output (from a spring petstore run): > > [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) > > This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 24 commits: - Feedback Johan; fix windows build errors - Merge branch 'master' into JDK-8318636-Add-jcmd-to-print-annotated-process-memory-map - Provide both print and dump command - Feedback Gerard - Feedbacj Johan - fix various builds - NMT_FLAGS_DO - small fixes - adapt test - remove test code - ... and 14 more: https://git.openjdk.org/jdk/compare/2d4a4d04...fe891f5c ------------- Changes: https://git.openjdk.org/jdk/pull/16301/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=07 Stats: 889 lines in 17 files changed: 887 ins; 1 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16301.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16301/head:pull/16301 PR: https://git.openjdk.org/jdk/pull/16301 From stuefe at openjdk.org Thu Nov 2 12:52:39 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 2 Nov 2023 12:52:39 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v7] In-Reply-To: References: <8tOkabtdC8K8xjHBVgKio1y14oJ4VAJPSuYFFTUs0xI=.f6363762-2997-4ceb-822a-b92695db760f@github.com> Message-ID: On Wed, 1 Nov 2023 15:38:46 GMT, Gerard Ziemski wrote: > LGTM > > Thank you for doing this! Thank you for reviewing, @gerard-ziemski ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1790665200 From bdutheil at openjdk.org Thu Nov 2 15:28:06 2023 From: bdutheil at openjdk.org (Brice Dutheil) Date: Thu, 2 Nov 2023 15:28:06 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Wed, 1 Nov 2023 06:26:57 GMT, Thomas Stuefe wrote: >> src/hotspot/os/linux/memMapPrinter_linux.cpp line 59: >> >>> 57: void print_OS_specific_details(outputStream* st) const override { >>> 58: st->print("%s %s ", _info.prot, _info.offset); >>> 59: } >> >> If that's all this is doing, do we really need it? > > This prints protection and offset. The former is interesting for obvious reasons (e.g. lets you tell apart stack guard regions from stack, or uncommitted from committed regions) and the latter interesting for some corner cases (e.g. ZGC mapped regions). > > I don't want to do this in shared code though since I am not sure every OS has this information or whether I can obtain this information. I rather not print out a bunch of 0s or empty strings. > > The alternative would be carrying a non-descriptive text string around with "OS information", but I don't want to have to think about storage and copying stuff around. It would not be simpler than this. > > BTW, the way to implement this on MacOS would be probably by spawning off `vmmap` and parsing the output; on Windows one would use `VirtualQuery`. But I leave this for other folks, I am happy with Linux for now. On ZGC, by the way, it's possible to see many `memfd:java_heap (deleted)`, does this work allow annotate these mappings ? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1380330786 From sspitsyn at openjdk.org Thu Nov 2 15:50:05 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 2 Nov 2023 15:50:05 GMT Subject: RFR: 8319238: JMX ThreadPoolAccTest.java is too verbose and should fail before timeout In-Reply-To: References: Message-ID: <32Rz8M0A865KTiasBJpx8UcEFTJGTfQ4heV0OLw6RkM=.eb68f176-f04e-44af-8170-77fb2e631c8e@github.com> On Wed, 1 Nov 2023 17:10:34 GMT, Kevin Walls wrote: > Discovered while testing changes that made this test fail. The test failure is hard to diagnose as it logs and retries at full speed, possibly forever, until timeout. This can hit a log file limit. We can save thousands of lines of text being printed when the test runs normally and successfully, by waiting half a second before doing the Principal-checking which is the purpose of the test. Looks reasonable. Thanks, Serguei test/jdk/javax/management/monitor/ThreadPoolAccTest.java line 167: > 165: String expected = principals[i / 3]; > 166: > 167: echo("testPrincipals: monitored: " + monitored[i] + " principal: " + principal + " expected: " + expected); With this in place, should printing of principal at line 176 be removed? ------------- Marked as reviewed by sspitsyn (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16456#pullrequestreview-1710643924 PR Review Comment: https://git.openjdk.org/jdk/pull/16456#discussion_r1380362829 From tschatzl at openjdk.org Thu Nov 2 15:51:35 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 15:51:35 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Add documentation about why and how we handle pinned regions in the young/old generation. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/73f61da9..5ae05e4c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=06-07 Stats: 18 lines in 2 files changed: 16 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Thu Nov 2 15:55:05 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 2 Nov 2023 15:55:05 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Mon, 30 Oct 2023 17:12:19 GMT, Thomas Schatzl wrote: >> I do not think so. I will do some more testing about this. > > I (still) do not think it is possible after some more re-testing. There are the following situations I can think of: > > * string deduplication is a need-to-be-supported case where only the C code may have a reference to a pinned object: thread A critical sections a string, gets the char array address, locking the region containing the char array. Then string dedup goes ahead and replaces the original char array with something else. Now the C code has the only reference to that char array. > There is no API to convert a raw array pointer back to a Java object so destroying the header is fine; unpinning does not need the header. > > * there is some other case I can think of that could be problematic, but is actually a spec violation: the array is critical-locked by thread A, then shared with other C code (not critical-unlocked), resumes with Java code that forgets that reference. At some point other C code accesses that locked memory and (hopefully) critically-unlocks it. > Again, there is no API to convert a raw array pointer back to a Java object so destroying the header is fine. > > In all other cases I can think of there is always a reference to the encapsulating java object either from the stack frame (when passing in the object into the JNI function they are part of the oop maps) or if you create a new array object (via `NewArray` and lock it, the VM will add a handle to it. > > There is also no API to inspect the array header using the raw pointer (e.g. passing the raw pointer to `GetArrayLength` - doesn't compile as it expects a `jarray`, and in debug VMs there is actually a check that the passed argument is something that resembles a handle), so modifications are already invalid, and the change is fine imo. > > hth, > Thomas Here is some example (pseudo-) code for the first case mentioned above that should be valid JNI code: Java code: String x = ...; native_f1(x); [ some java code, x.chars gets deduplicated, its char array pointing to somewhere else now. Now native code is the only one having a reference to the old char array ] native_f2(); ----------- sample native code: void native_f1(jobject jstring) { global_string = NewGlobalRef(jstring); global_raw_chars = GetStringChars(global_string); } void native_f2() { ReleaseStringChars(global_string, global_raw_chars); DeleteGlobalRef(global_string); } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1380370821 From stuefe at openjdk.org Thu Nov 2 17:42:29 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 2 Nov 2023 17:42:29 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: Message-ID: > Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. > > Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. > > That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. > > Example output (from a spring petstore run): > > [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) > > This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: Fix another windows error ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16301/files - new: https://git.openjdk.org/jdk/pull/16301/files/fe891f5c..7dd5a911 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=07-08 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16301.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16301/head:pull/16301 PR: https://git.openjdk.org/jdk/pull/16301 From stuefe at openjdk.org Thu Nov 2 17:42:30 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 2 Nov 2023 17:42:30 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Thu, 2 Nov 2023 15:25:14 GMT, Brice Dutheil wrote: >> This prints protection and offset. The former is interesting for obvious reasons (e.g. lets you tell apart stack guard regions from stack, or uncommitted from committed regions) and the latter interesting for some corner cases (e.g. ZGC mapped regions). >> >> I don't want to do this in shared code though since I am not sure every OS has this information or whether I can obtain this information. I rather not print out a bunch of 0s or empty strings. >> >> The alternative would be carrying a non-descriptive text string around with "OS information", but I don't want to have to think about storage and copying stuff around. It would not be simpler than this. >> >> BTW, the way to implement this on MacOS would be probably by spawning off `vmmap` and parsing the output; on Windows one would use `VirtualQuery`. But I leave this for other folks, I am happy with Linux for now. > > On ZGC, by the way, it's possible to see many `memfd:java_heap (deleted)`, does this work allow annotate these mappings ? Yes, they should show up as JAVAHEAP. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1380570741 From kevinw at openjdk.org Thu Nov 2 18:19:05 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Thu, 2 Nov 2023 18:19:05 GMT Subject: RFR: 8319238: JMX ThreadPoolAccTest.java is too verbose and should fail before timeout In-Reply-To: <32Rz8M0A865KTiasBJpx8UcEFTJGTfQ4heV0OLw6RkM=.eb68f176-f04e-44af-8170-77fb2e631c8e@github.com> References: <32Rz8M0A865KTiasBJpx8UcEFTJGTfQ4heV0OLw6RkM=.eb68f176-f04e-44af-8170-77fb2e631c8e@github.com> Message-ID: On Thu, 2 Nov 2023 15:47:00 GMT, Serguei Spitsyn wrote: >> Discovered while testing changes that made this test fail. The test failure is hard to diagnose as it logs and retries at full speed, possibly forever, until timeout. This can hit a log file limit. We can save thousands of lines of text being printed when the test runs normally and successfully, by waiting half a second before doing the Principal-checking which is the purpose of the test. > > test/jdk/javax/management/monitor/ThreadPoolAccTest.java line 167: > >> 165: String expected = principals[i / 3]; >> 166: >> 167: echo("testPrincipals: monitored: " + monitored[i] + " principal: " + principal + " expected: " + expected); > > With this in place, should printing of principal at line 176 be removed? Right, it is duplicated. That new line is useful, I really wanted to see something before we check for null. Then we have the existing printing before we check further, which I did not disturb. I'm sure it's not a big deal either way. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16456#discussion_r1380610946 From amenkov at openjdk.org Thu Nov 2 20:26:01 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 2 Nov 2023 20:26:01 GMT Subject: RFR: JDK-8319053: Segment dump files remain after parallel heap dump on Windows In-Reply-To: References: Message-ID: <2WABOcF-1FVRWZVqLWmYNrNXY8GuH34KSdIcxc5Xecg=.cdba32fb-7ed9-4b96-966c-8bcd8237dbdf@github.com> On Thu, 2 Nov 2023 06:14:59 GMT, David Holmes wrote: > Sorry I don't understand the fix here. You changed the AbstractDumpWriter to be a C-heap object instead of ResourceObject and then delete it, so invoking a destructor, which AFAICS does nothing in terms of removing any files. ??? The files are removed by DumpMerger outside of the VM operation. On Windows opened files cannot be removed ("access denied"). DumpWriter dtor closes the file (it has `FileWriter* _writer` field; dtor deletes the object and FileWriter dtor closes the file). > > BTW with this change you may be able to remove the ResourceMark prior to allocating `local_writer` (line 2350). VM_HeapDumper::create_local_writer() uses resource area to allocate buffer for file path, so ResourceMark is needed there anyway. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16462#issuecomment-1791485769 From amenkov at openjdk.org Thu Nov 2 20:44:14 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 2 Nov 2023 20:44:14 GMT Subject: RFR: JDK-8319053: Segment dump files remain after parallel heap dump on Windows [v2] In-Reply-To: References: Message-ID: > Segment file is closed from DumpWriter dtor. > On Unix systems `remove` can delete an opened file, on Windows it fails with "access denied". > The fix destroys DumpWriter objects for segment files after all data are dumped Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: updated log text ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16462/files - new: https://git.openjdk.org/jdk/pull/16462/files/8a1f0eb0..6474afaf Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16462&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16462&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16462.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16462/head:pull/16462 PR: https://git.openjdk.org/jdk/pull/16462 From amenkov at openjdk.org Thu Nov 2 21:42:04 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 2 Nov 2023 21:42:04 GMT Subject: RFR: JDK-8319053: Segment dump files remain after parallel heap dump on Windows [v2] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 06:09:22 GMT, David Holmes wrote: >> Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: >> >> updated log text > > src/hotspot/share/services/heapDumper.cpp line 1959: > >> 1957: // Delete selected segmented heap file nevertheless >> 1958: if (remove(path) != 0) { >> 1959: log_info(heapdump)("Remove segment file (%d) failed (%d)", i, errno); > > Suggestion: "Removal of segment file ..." Fixed ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16462#discussion_r1380811214 From amenkov at openjdk.org Thu Nov 2 23:25:06 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 2 Nov 2023 23:25:06 GMT Subject: RFR: JDK-8318736: com/sun/jdi/JdwpOnThrowTest.java failed with "transport error 202: bind failed: Address already in use" [v5] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 11:35:29 GMT, Johannes Bechberger wrote: >> Fix race condition in debugger port selection, introduced with [JDK-8317920](https://bugs.openjdk.org/browse/JDK-8317920). >> >> Tested on my Mac M1, but it doesn't contain platform-dependent code. > > Johannes Bechberger has updated the pull request incrementally with one additional commit since the last revision: > > Refactor as requested Marked as reviewed by amenkov (Reviewer). test/jdk/com/sun/jdi/lib/jdb/Debuggee.java line 132: > 130: onthrow.isEmpty() ? > 131: JDWP::parseListenAddress : > 132: Launcher::parseLaunchEchoListenAddress Please increase indent for conditional operator Suggestion: onthrow.isEmpty() ? JDWP::parseListenAddress : Launcher::parseLaunchEchoListenAddress or Suggestion: onthrow.isEmpty() ? JDWP::parseListenAddress : Launcher::parseLaunchEchoListenAddress ------------- PR Review: https://git.openjdk.org/jdk/pull/16358#pullrequestreview-1711398441 PR Review Comment: https://git.openjdk.org/jdk/pull/16358#discussion_r1380882396 From yyang at openjdk.org Fri Nov 3 03:29:04 2023 From: yyang at openjdk.org (Yi Yang) Date: Fri, 3 Nov 2023 03:29:04 GMT Subject: RFR: JDK-8319053: Segment dump files remain after parallel heap dump on Windows [v2] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 20:44:14 GMT, Alex Menkov wrote: >> Segment file is closed from DumpWriter dtor. >> On Unix systems `remove` can delete an opened file, on Windows it fails with "access denied". >> The fix destroys DumpWriter objects for segment files after all data are dumped > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > updated log text Looks reasonable. Thanks. ------------- Marked as reviewed by yyang (Committer). PR Review: https://git.openjdk.org/jdk/pull/16462#pullrequestreview-1711780491 From sspitsyn at openjdk.org Fri Nov 3 04:18:26 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 3 Nov 2023 04:18:26 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v2] In-Reply-To: References: Message-ID: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes three new classes: > - `JvmtiHandshake`, `JvmtiUnifiedHandshakeClosure`, VM_HandshakeUnmountedVirtualThread > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with two additional commits since the last revision: - address review: remove fix in libGetStackTraceSuspendedStress.cpp - addressed initial minor review comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/218d439f..7ef7dbbc Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=00-01 Stats: 64 lines in 4 files changed: 33 ins; 29 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From sspitsyn at openjdk.org Fri Nov 3 04:29:14 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 3 Nov 2023 04:29:14 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v3] In-Reply-To: References: Message-ID: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes three new classes: > - `JvmtiHandshake`, `JvmtiUnifiedHandshakeClosure`, VM_HandshakeUnmountedVirtualThread > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: remove unneeded ResourceMark from JVMTI GetStackTrace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/7ef7dbbc..720c9c7e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From dholmes at openjdk.org Fri Nov 3 06:25:02 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 3 Nov 2023 06:25:02 GMT Subject: RFR: JDK-8319053: Segment dump files remain after parallel heap dump on Windows [v2] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 20:44:14 GMT, Alex Menkov wrote: >> Segment file is closed from DumpWriter dtor. >> On Unix systems `remove` can delete an opened file, on Windows it fails with "access denied". >> The fix destroys DumpWriter objects for segment files after all data are dumped > > Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: > > updated log text Thanks for the explanation. Looks fine. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16462#pullrequestreview-1711885352 From sspitsyn at openjdk.org Fri Nov 3 06:39:01 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 3 Nov 2023 06:39:01 GMT Subject: RFR: 8319238: JMX ThreadPoolAccTest.java is too verbose and should fail before timeout In-Reply-To: References: <32Rz8M0A865KTiasBJpx8UcEFTJGTfQ4heV0OLw6RkM=.eb68f176-f04e-44af-8170-77fb2e631c8e@github.com> Message-ID: On Thu, 2 Nov 2023 18:16:49 GMT, Kevin Walls wrote: >> test/jdk/javax/management/monitor/ThreadPoolAccTest.java line 167: >> >>> 165: String expected = principals[i / 3]; >>> 166: >>> 167: echo("testPrincipals: monitored: " + monitored[i] + " principal: " + principal + " expected: " + expected); >> >> With this in place, should printing of principal at line 176 be removed? > > Right, it is duplicated. That new line is useful, I really wanted to see something before we check for null. > Then we have the existing printing before we check further, which I did not disturb. I'm sure it's not a big deal either way. Up to you. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16456#discussion_r1381203169 From jbechberger at openjdk.org Fri Nov 3 07:22:26 2023 From: jbechberger at openjdk.org (Johannes Bechberger) Date: Fri, 3 Nov 2023 07:22:26 GMT Subject: RFR: JDK-8318736: com/sun/jdi/JdwpOnThrowTest.java failed with "transport error 202: bind failed: Address already in use" [v6] In-Reply-To: References: Message-ID: > Fix race condition in debugger port selection, introduced with [JDK-8317920](https://bugs.openjdk.org/browse/JDK-8317920). > > Tested on my Mac M1, but it doesn't contain platform-dependent code. Johannes Bechberger has updated the pull request incrementally with one additional commit since the last revision: Update test/jdk/com/sun/jdi/lib/jdb/Debuggee.java Co-authored-by: Alex Menkov <69548902+alexmenkov at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16358/files - new: https://git.openjdk.org/jdk/pull/16358/files/8a9b7133..8168a8ef Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16358&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16358&range=04-05 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16358.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16358/head:pull/16358 PR: https://git.openjdk.org/jdk/pull/16358 From stuefe at openjdk.org Fri Nov 3 08:29:07 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 3 Nov 2023 08:29:07 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 17:42:29 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > Fix another windows error Windows GHA errors unrelated. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1792037200 From ayang at openjdk.org Fri Nov 3 10:19:10 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 10:19:10 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 15:51:35 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Add documentation about why and how we handle pinned regions in the young/old generation. src/hotspot/share/gc/g1/g1FullGCPrepareTask.inline.hpp line 76: > 74: } > 75: > 76: inline bool G1DetermineCompactionQueueClosure::has_pinned_objects(HeapRegion* hr) const { Could this be a static-local function so that it doesn't appear in the header file? (Its name is the same as the public API in heap-region.) src/hotspot/share/gc/g1/g1GCPhaseTimes.cpp line 482: > 480: } > 481: > 482: double G1GCPhaseTimes::print_post_evacuate_collection_set(bool evacuation_retained) const { Why the renaming here? src/hotspot/share/gc/g1/g1GCPhaseTimes.hpp line 150: > 148: > 149: enum RestoreRetainedRegionsWorkItems { > 150: RestoreRetainedRegionsEvacFailedNum, // How many regions experienced an evacuation failure (pinned or allocation failure) Kind of a preexisting issue. "retained" here seems to mean evac-fail, not "kept in retain-list". src/hotspot/share/gc/g1/g1HeapRegionAttr.hpp line 43: > 41: remset_is_tracked_t _remset_is_tracked; > 42: region_type_t _type; > 43: bool _is_pinned; Maybe `uint8_t` as documented above? src/hotspot/share/gc/g1/g1Policy.cpp line 547: > 545: } > 546: > 547: log_trace(gc, ergo, heap)("Selected %u of %u retained candidates (unreclaimable %u) taking %1.3fms additional time", I actually think calling it "pinned", instead of "unreclaimable", is more informative (to users/dev). (And other places when it is shown in logs.) src/hotspot/share/gc/g1/g1YoungCollector.cpp line 1102: > 1100: jtm.report_pause_type(collector_state()->young_gc_pause_type(_concurrent_operation_is_full_mark)); > 1101: > 1102: policy()->record_young_collection_end(_concurrent_operation_is_full_mark, evacuation_alloc_failed()); The arg name (where this method is defined) should be updated to sth like `evac_alloc_failed` from `evacuation_failure`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381415671 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381418678 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381440184 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381419525 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381423626 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381433739 From ayang at openjdk.org Fri Nov 3 10:19:12 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 10:19:12 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: <-W9sGrhwC6dMCJZrvR8cYU_YzEoE23CMbsU9WGXSocs=.4771371a-5cf9-4694-b32e-f173a14dcb7c@github.com> On Thu, 2 Nov 2023 15:52:09 GMT, Thomas Schatzl wrote: >> I (still) do not think it is possible after some more re-testing. There are the following situations I can think of: >> >> * string deduplication is a need-to-be-supported case where only the C code may have a reference to a pinned object: thread A critical sections a string, gets the char array address, locking the region containing the char array. Then string dedup goes ahead and replaces the original char array with something else. Now the C code has the only reference to that char array. >> There is no API to convert a raw array pointer back to a Java object so destroying the header is fine; unpinning does not need the header. >> >> * there is some other case I can think of that could be problematic, but is actually a spec violation: the array is critical-locked by thread A, then shared with other C code (not critical-unlocked), resumes with Java code that forgets that reference. At some point other C code accesses that locked memory and (hopefully) critically-unlocks it. >> Again, there is no API to convert a raw array pointer back to a Java object so destroying the header is fine. >> >> In all other cases I can think of there is always a reference to the encapsulating java object either from the stack frame (when passing in the object into the JNI function they are part of the oop maps) or if you create a new array object (via `NewArray` and lock it, the VM will add a handle to it. >> >> There is also no API to inspect the array header using the raw pointer (e.g. passing the raw pointer to `GetArrayLength` - doesn't compile as it expects a `jarray`, and in debug VMs there is actually a check that the passed argument is something that resembles a handle), so modifications are already invalid, and the change is fine imo. >> >> hth, >> Thomas > > Here is some example (pseudo-) code for the first case mentioned above that should be valid JNI code: > > > Java code: > > String x = ...; > native_f1(x); > [ some java code, x.chars gets deduplicated, its char array pointing to somewhere else now. Now native code is the only one having a reference to the old char array ] > native_f2(); > > ----------- sample native code: > > void native_f1(jobject jstring) { > global_string = NewGlobalRef(jstring); > global_raw_chars = GetStringChars(global_string); > } > > void native_f2() { > ReleaseStringChars(global_string, global_raw_chars); > DeleteGlobalRef(global_string); > } > string deduplication is a need-to-be-supported case... OK, so this is the only valid scenario where a type-array should be kept live even though it's not reachable from GC's perspective. Could you describe it in the comment? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381436094 From tschatzl at openjdk.org Fri Nov 3 10:59:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 10:59:06 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 09:56:43 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> Add documentation about why and how we handle pinned regions in the young/old generation. > > src/hotspot/share/gc/g1/g1GCPhaseTimes.cpp line 482: > >> 480: } >> 481: >> 482: double G1GCPhaseTimes::print_post_evacuate_collection_set(bool evacuation_retained) const { > > Why the renaming here? Probably forgot to undo the rename. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381501087 From tschatzl at openjdk.org Fri Nov 3 12:32:02 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 12:32:02 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: ayang review - renamings + documentation ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/5ae05e4c..8342b80b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=07-08 Stats: 79 lines in 17 files changed: 8 ins; 3 del; 68 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Fri Nov 3 12:32:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 12:32:06 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v8] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 15:51:35 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Add documentation about why and how we handle pinned regions in the young/old generation. Fwiw, recent changes (without the most recent renamings) passed tier1-5 ------------- PR Comment: https://git.openjdk.org/jdk/pull/16342#issuecomment-1792352540 From jbechberger at openjdk.org Fri Nov 3 12:37:12 2023 From: jbechberger at openjdk.org (Johannes Bechberger) Date: Fri, 3 Nov 2023 12:37:12 GMT Subject: Integrated: JDK-8318736: com/sun/jdi/JdwpOnThrowTest.java failed with "transport error 202: bind failed: Address already in use" In-Reply-To: References: Message-ID: On Wed, 25 Oct 2023 11:27:18 GMT, Johannes Bechberger wrote: > Fix race condition in debugger port selection, introduced with [JDK-8317920](https://bugs.openjdk.org/browse/JDK-8317920). > > Tested on my Mac M1, but it doesn't contain platform-dependent code. This pull request has now been integrated. Changeset: 1a21c1a7 Author: Johannes Bechberger URL: https://git.openjdk.org/jdk/commit/1a21c1a783d64ca0930c358c06a43975f96ffac6 Stats: 50 lines in 2 files changed: 12 ins; 18 del; 20 mod 8318736: com/sun/jdi/JdwpOnThrowTest.java failed with "transport error 202: bind failed: Address already in use" Reviewed-by: amenkov ------------- PR: https://git.openjdk.org/jdk/pull/16358 From sspitsyn at openjdk.org Fri Nov 3 12:38:20 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 3 Nov 2023 12:38:20 GMT Subject: RFR: 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) Message-ID: It is a fix of a minor test issue. The test should not fail when the JVMTI function `SetEventNotificationMode()` returns errors codes: - `JVMTI_ERROR_THREAD_NOT_ALIVE` - `JVMTI_ERROR_WRONG_PHASE` Tested the fix locally and with mach5 test runs. ------------- Commit messages: - 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) Changes: https://git.openjdk.org/jdk/pull/16488/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16488&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8318631 Stats: 8 lines in 1 file changed: 8 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16488.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16488/head:pull/16488 PR: https://git.openjdk.org/jdk/pull/16488 From rkennke at openjdk.org Fri Nov 3 12:51:14 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Fri, 3 Nov 2023 12:51:14 GMT Subject: RFR: 8319376: Parallel: Forwarded objects during heap inspection Message-ID: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> See JBS issue for details. Testing: - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC - [ ] tier1 -XX:+UseParallelGC - [ ] tier2 -XX:+UseParallelGC - [ ] hotspot_gc ------------- Commit messages: - 8319376: Parallel: Forwarded objects during heap inspection Changes: https://git.openjdk.org/jdk/pull/16494/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319376 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From ayang at openjdk.org Fri Nov 3 12:53:08 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 12:53:08 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 12:32:02 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > ayang review - renamings + documentation src/hotspot/share/gc/g1/g1Policy.hpp line 275: > 273: double start, > 274: double end, > 275: bool alloocation_failure = false); Typo. src/hotspot/share/gc/g1/g1Policy.hpp line 314: > 312: // Record the start and end of the actual collection part of the evacuation pause. > 313: void record_young_collection_start(); > 314: void record_young_collection_end(bool concurrent_operation_is_full_mark, bool alllocation_failure); Typo. src/hotspot/share/gc/g1/g1YoungCollector.cpp line 87: > 85: GCCause::to_string(_pause_cause), > 86: _collector->evacuation_pinned() ? " (Pinned)" : "", > 87: _collector->evacuation_alloc_failed() ? " (Allocation Failure)" : ""); > GC(6) Pause Young (Normal) (Pinned) (Evacuation Failure) I wonder if the last two can be merged into one `()`, sth like `(Pinned / ...)`, because they are on the same abstraction level. src/hotspot/share/gc/g1/g1_globals.hpp line 327: > 325: range(1, 256) \ > 326: \ > 327: product(uint, G1NumCollectionsKeepPinned, 8, DIAGNOSTIC, \ Any particular reason this is not `EXPERIMENTAL`? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381627320 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381627700 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381628625 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381624492 From tschatzl at openjdk.org Fri Nov 3 13:46:51 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 13:46:51 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v10] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: typos ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/8342b80b..f9735539 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=08-09 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Fri Nov 3 13:50:09 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 13:50:09 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: <4uKST6lml9Okm18TVjp2hgBUQHBPH0FP_Uv13Pr7CLE=.dab77bbb-eec6-41d7-820d-3ed89779feb0@github.com> On Fri, 3 Nov 2023 12:41:05 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review - renamings + documentation > > src/hotspot/share/gc/g1/g1_globals.hpp line 327: > >> 325: range(1, 256) \ >> 326: \ >> 327: product(uint, G1NumCollectionsKeepPinned, 8, DIAGNOSTIC, \ > > Any particular reason this is not `EXPERIMENTAL`? Changing this does not in any way enable risky/experimental code not fit for production. This knob is for helping diagnose performance issues. G1 does have its fair share of experimental options, but all/most of these were from the initial import where G1 as a whole had been experimental (unstable) for some time. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381706769 From nikhil.a.agarwal at oracle.com Fri Nov 3 13:50:28 2023 From: nikhil.a.agarwal at oracle.com (Nikhil Agarwal) Date: Fri, 3 Nov 2023 13:50:28 +0000 Subject: getProcessCPULoad returns different CPU percentage as comparead to top linux command Message-ID: Hi All, getProcessCPULoad returns different CPU percentage as compared to top linux command intermittently . Does it supposed to returned same data as top command always . Should the getProcessCPULoad value be same as top/number of cpus always ? regards Nikhil -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayang at openjdk.org Fri Nov 3 14:17:07 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 14:17:07 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v10] In-Reply-To: References: Message-ID: <1-i3-5OmZbuCNUlpfv31Kr3eiBXEd4Si8F5gsbPHuBQ=.1d97dcac-4662-4482-842c-ce86315ba61a@github.com> On Fri, 3 Nov 2023 13:46:51 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > typos Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16342#pullrequestreview-1712794411 From tschatzl at openjdk.org Fri Nov 3 14:17:11 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Nov 2023 14:17:11 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 12:44:10 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review - renamings + documentation > > src/hotspot/share/gc/g1/g1YoungCollector.cpp line 87: > >> 85: GCCause::to_string(_pause_cause), >> 86: _collector->evacuation_pinned() ? " (Pinned)" : "", >> 87: _collector->evacuation_alloc_failed() ? " (Allocation Failure)" : ""); > >> GC(6) Pause Young (Normal) (Pinned) (Evacuation Failure) > > I wonder if the last two can be merged into one `()`, sth like `(Pinned / ...)`, because they are on the same abstraction level. Parsing the separate components is easier :) Not sure if these tags in any way ever indicated some level of abstraction. I do not have a strong opinion here. The combinations (Pinned) (Allocation Failure) (Pinned + Allocation Failure) // or the other way around, or some other symbol for "+" or no symbol at all? are fine with me (and I thought about doing something more elaborate here), but my concern has been that any complicated string makes it less unique (e.g. `(Allocation Failure)` vs. "Allocation Failure") and adds code both to implement and parse the result. Much more disrupting is likely that there is no "Evacuation Failure" string any more. But log messages are not part of the external interface, and we should not want to change them just because. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381716129 From ayang at openjdk.org Fri Nov 3 14:17:13 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Fri, 3 Nov 2023 14:17:13 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 13:53:35 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/g1YoungCollector.cpp line 87: >> >>> 85: GCCause::to_string(_pause_cause), >>> 86: _collector->evacuation_pinned() ? " (Pinned)" : "", >>> 87: _collector->evacuation_alloc_failed() ? " (Allocation Failure)" : ""); >> >>> GC(6) Pause Young (Normal) (Pinned) (Evacuation Failure) >> >> I wonder if the last two can be merged into one `()`, sth like `(Pinned / ...)`, because they are on the same abstraction level. > > Parsing the separate components is easier :) Not sure if these tags in any way ever indicated some level of abstraction. > > I do not have a strong opinion here. The combinations > > (Pinned) > (Allocation Failure) > (Pinned + Allocation Failure) // or the other way around, or some other symbol for "+" or no symbol at all? > > are fine with me (and I thought about doing something more elaborate here), but my concern has been that any complicated string makes it less unique (e.g. `(Allocation Failure)` vs. "Allocation Failure") and adds code both to implement and parse the result. > > Much more disrupting is likely that there is no "Evacuation Failure" string any more. But log messages are not part of the external interface, and we should not want to change them just because. The example looks good to me. >> src/hotspot/share/gc/g1/g1_globals.hpp line 327: >> >>> 325: range(1, 256) \ >>> 326: \ >>> 327: product(uint, G1NumCollectionsKeepPinned, 8, DIAGNOSTIC, \ >> >> Any particular reason this is not `EXPERIMENTAL`? > > Changing this does not in any way enable risky/experimental code not fit for production. This knob is for helping diagnose performance issues. > > G1 does have its fair share of experimental options, but all/most of these were from the initial import where G1 as a whole had been experimental (unstable) for some time. This flag conceptually related (or similar) to `G1RetainRegionLiveThresholdPercent`, which is an exp, so I thought they should be the same category. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381748512 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1381747902 From amenkov at openjdk.org Fri Nov 3 20:12:20 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 3 Nov 2023 20:12:20 GMT Subject: RFR: 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 04:31:33 GMT, Serguei Spitsyn wrote: > It is a fix of a minor test issue. > The test should not fail when the JVMTI function `SetEventNotificationMode()` returns errors codes: > - `JVMTI_ERROR_THREAD_NOT_ALIVE` > - `JVMTI_ERROR_WRONG_PHASE` > > Tested the fix locally and with mach5 test runs. test/hotspot/jtreg/serviceability/jvmti/stress/StackTrace/Suspended/libGetStackTraceSuspendedStress.cpp line 125: > 123: err = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_SINGLE_STEP, vthread); > 124: if (err == JVMTI_ERROR_THREAD_NOT_ALIVE || > 125: err == JVMTI_ERROR_WRONG_PHASE) { WRONG_PHASE looks good to me, but why THREAD_NOT_ALIVE is considered expected for suspended thread? if the thread was terminated, SuspendThread should return THREAD_NOT_ALIVE, but once SuspendThread returns ERROR_NONE, how the thread can terminates before ResumeThread? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16488#discussion_r1382154927 From amenkov at openjdk.org Fri Nov 3 20:47:24 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Fri, 3 Nov 2023 20:47:24 GMT Subject: Integrated: JDK-8319053: Segment dump files remain after parallel heap dump on Windows In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 19:23:14 GMT, Alex Menkov wrote: > Segment file is closed from DumpWriter dtor. > On Unix systems `remove` can delete an opened file, on Windows it fails with "access denied". > The fix destroys DumpWriter objects for segment files after all data are dumped This pull request has now been integrated. Changeset: 29cf2c47 Author: Alex Menkov URL: https://git.openjdk.org/jdk/commit/29cf2c471bf046cd58bd6fefd617a2b03040d4ff Stats: 22 lines in 2 files changed: 17 ins; 2 del; 3 mod 8319053: Segment dump files remain after parallel heap dump on Windows Reviewed-by: dholmes, yyang ------------- PR: https://git.openjdk.org/jdk/pull/16462 From iwalulya at openjdk.org Fri Nov 3 21:25:13 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Fri, 3 Nov 2023 21:25:13 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v10] In-Reply-To: References: Message-ID: On Fri, 3 Nov 2023 13:46:51 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > typos LGTM! Nits: src/hotspot/share/gc/g1/g1FullCollector.cpp line 465: > 463: continue; > 464: } else if (is_compaction_target(region_index)) { > 465: assert(!hr->has_pinned_objects(), "pinned objects should not be compaction targets"); Suggestion: assert(!hr->has_pinned_objects(), "pinned regions should not be compaction targets"); src/hotspot/share/gc/g1/g1YoungCollector.cpp line 430: > 428: _claimer(_g1h->workers()->active_workers()), > 429: _humongous_total(0), > 430: _humongous_candidates(0) { } Suggestion: _humongous_candidates(0) { } ------------- Marked as reviewed by iwalulya (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16342#pullrequestreview-1707910606 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1382164253 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1382135577 From iwalulya at openjdk.org Fri Nov 3 21:25:16 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Fri, 3 Nov 2023 21:25:16 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v5] In-Reply-To: References: Message-ID: On Tue, 31 Oct 2023 19:14:13 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Fix compilation src/hotspot/share/gc/g1/g1CollectionSet.cpp line 355: > 353: move_pinned_marking_to_retained(&pinned_marking_regions); > 354: // Drop pinned retained regions to make progress with retained regions. Regions > 355: // in that list have must have been pinned for at least Suggestion: // in that list must have been pinned for at least ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1378608614 From stuefe at openjdk.org Sat Nov 4 06:49:15 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Sat, 4 Nov 2023 06:49:15 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Wed, 1 Nov 2023 10:04:30 GMT, Johan Sj?len wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > OK, went through the cache. Will continue later. @jdksjolen anything more needed from my side? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1793362385 From david.holmes at oracle.com Mon Nov 6 05:28:16 2023 From: david.holmes at oracle.com (David Holmes) Date: Mon, 6 Nov 2023 15:28:16 +1000 Subject: getProcessCPULoad returns different CPU percentage as comparead to top linux command In-Reply-To: References: Message-ID: <45180289-8dd1-4c53-b822-7abc5602ed52@oracle.com> Hi, On 3/11/2023 11:50 pm, Nikhil Agarwal wrote: > Hi All, > > ????????????? getProcessCPULoad returns different CPU percentage as > compared to top linux command intermittently . Does it supposed to > returned same data as top command always . ?Should the getProcessCPULoad > value be same as ?top/number of cpus always ? I've no idea how top calculates what it does. The VM reads the first line of /proc/stat to get the various tick counters. HTH. David > regards > > Nikhil > From dholmes at openjdk.org Mon Nov 6 05:37:09 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 6 Nov 2023 05:37:09 GMT Subject: RFR: 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) In-Reply-To: References: Message-ID: <_nZqf39Y7sE1-0vEZfiqvc7Bni2Bw7ctPst0IJrDWbg=.5e1ea5bc-d0f7-4fd9-ab0a-9387902eeac2@github.com> On Fri, 3 Nov 2023 20:09:28 GMT, Alex Menkov wrote: >> It is a fix of a minor test issue. >> The test should not fail when the JVMTI function `SetEventNotificationMode()` returns errors codes: >> - `JVMTI_ERROR_THREAD_NOT_ALIVE` >> - `JVMTI_ERROR_WRONG_PHASE` >> >> Tested the fix locally and with mach5 test runs. > > test/hotspot/jtreg/serviceability/jvmti/stress/StackTrace/Suspended/libGetStackTraceSuspendedStress.cpp line 125: > >> 123: err = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_SINGLE_STEP, vthread); >> 124: if (err == JVMTI_ERROR_THREAD_NOT_ALIVE || >> 125: err == JVMTI_ERROR_WRONG_PHASE) { > > WRONG_PHASE looks good to me, but why THREAD_NOT_ALIVE is considered expected for suspended thread? > if the thread was terminated, SuspendThread should return THREAD_NOT_ALIVE, but once SuspendThread returns ERROR_NONE, how the thread can terminates before ResumeThread? My question exactly. I'm not even sure why wrong phase is allowed here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16488#discussion_r1382799696 From nikhil.a.agarwal at oracle.com Mon Nov 6 05:53:52 2023 From: nikhil.a.agarwal at oracle.com (Nikhil Agarwal) Date: Mon, 6 Nov 2023 05:53:52 +0000 Subject: getProcessCPULoad returns different CPU percentage as comparead to top linux command In-Reply-To: <45180289-8dd1-4c53-b822-7abc5602ed52@oracle.com> References: <45180289-8dd1-4c53-b822-7abc5602ed52@oracle.com> Message-ID: Hi David, Thanks for your reply , It seems top command also uses /proc/stat to do the magic .Reading a similar issue for top vs another monitor tool on https://medium.com/@yogita088/how-to-calculate-cpu-usage-proc-stat-vs-top-e74f99f02d08 , What is the duration for which getProcessCPUload reports CPU utilisation , is it from system start time or 1 sec or x seconds ? It is configurable in top command as -d parameter. This might help to discover difference btw top vs getprocesscpuload regards Nikhil -----Original Message----- From: David Holmes Sent: 06 November 2023 10:58 To: Nikhil Agarwal ; serviceability-dev at openjdk.org Subject: Re: getProcessCPULoad returns different CPU percentage as comparead to top linux command Hi, On 3/11/2023 11:50 pm, Nikhil Agarwal wrote: > Hi All, > > ????????????? getProcessCPULoad returns different CPU percentage as > compared to top linux command intermittently . Does it supposed to > returned same data as top command always . ?Should the > getProcessCPULoad value be same as ?top/number of cpus always ? I've no idea how top calculates what it does. The VM reads the first line of /proc/stat to get the various tick counters. HTH. David > regards > > Nikhil > From dholmes at openjdk.org Mon Nov 6 06:00:07 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 6 Nov 2023 06:00:07 GMT Subject: RFR: 8319376: Parallel: Forwarded objects during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc @rkennke could you update the title to "ParallelGC: Forwarded objects found during heap inspection" please. Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1794136928 From alanb at openjdk.org Mon Nov 6 06:19:09 2023 From: alanb at openjdk.org (Alan Bateman) Date: Mon, 6 Nov 2023 06:19:09 GMT Subject: RFR: 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) In-Reply-To: <_nZqf39Y7sE1-0vEZfiqvc7Bni2Bw7ctPst0IJrDWbg=.5e1ea5bc-d0f7-4fd9-ab0a-9387902eeac2@github.com> References: <_nZqf39Y7sE1-0vEZfiqvc7Bni2Bw7ctPst0IJrDWbg=.5e1ea5bc-d0f7-4fd9-ab0a-9387902eeac2@github.com> Message-ID: On Mon, 6 Nov 2023 05:34:04 GMT, David Holmes wrote: >> test/hotspot/jtreg/serviceability/jvmti/stress/StackTrace/Suspended/libGetStackTraceSuspendedStress.cpp line 125: >> >>> 123: err = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_SINGLE_STEP, vthread); >>> 124: if (err == JVMTI_ERROR_THREAD_NOT_ALIVE || >>> 125: err == JVMTI_ERROR_WRONG_PHASE) { >> >> WRONG_PHASE looks good to me, but why THREAD_NOT_ALIVE is considered expected for suspended thread? >> if the thread was terminated, SuspendThread should return THREAD_NOT_ALIVE, but once SuspendThread returns ERROR_NONE, how the thread can terminates before ResumeThread? > > My question exactly. I'm not even sure why wrong phase is allowed here. Is the issue here that agent thread started by Debuggee.checkStatus is racing with the test? The producer/consumer thread do 1000 put/take ops and it looks like it can complete and VM commence shutdown while the agent thread is observing, do I read this correctly? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16488#discussion_r1382823031 From david.holmes at oracle.com Mon Nov 6 06:36:19 2023 From: david.holmes at oracle.com (David Holmes) Date: Mon, 6 Nov 2023 16:36:19 +1000 Subject: getProcessCPULoad returns different CPU percentage as comparead to top linux command In-Reply-To: References: <45180289-8dd1-4c53-b822-7abc5602ed52@oracle.com> Message-ID: On 6/11/2023 3:53 pm, Nikhil Agarwal wrote: > Hi David, > Thanks for your reply , It seems top command also uses /proc/stat to do the magic .Reading a similar issue for top vs another monitor tool on > https://medium.com/@yogita088/how-to-calculate-cpu-usage-proc-stat-vs-top-e74f99f02d08 , What is the duration for which getProcessCPUload reports CPU utilisation , is it from system start time or 1 sec or x seconds ? It is configurable in top command as -d parameter. This might help to discover difference btw top vs getprocesscpuload For getProcessCpuLoad it is the time since the JVM process initialized the performance counters. At VM init the counters are read from /proc/stat and /proc/self/stat and stashed away. Then when you ask for the load the counters are read again and the initial values subtracted to give the load across the lifetime of the process. You can see the code here: https://github.com/openjdk/jdk/blob/master/src/jdk.management/linux/native/libmanagement_ext/UnixOperatingSystem.c David > regards > Nikhil > > > -----Original Message----- > From: David Holmes > Sent: 06 November 2023 10:58 > To: Nikhil Agarwal ; serviceability-dev at openjdk.org > Subject: Re: getProcessCPULoad returns different CPU percentage as comparead to top linux command > > Hi, > > On 3/11/2023 11:50 pm, Nikhil Agarwal wrote: >> Hi All, >> >> ????????????? getProcessCPULoad returns different CPU percentage as >> compared to top linux command intermittently . Does it supposed to >> returned same data as top command always . ?Should the >> getProcessCPULoad value be same as ?top/number of cpus always ? > > I've no idea how top calculates what it does. The VM reads the first line of /proc/stat to get the various tick counters. > > HTH. > > David > >> regards >> >> Nikhil >> From jpai at openjdk.org Mon Nov 6 07:02:19 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Mon, 6 Nov 2023 07:02:19 GMT Subject: RFR: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods Message-ID: Can I please get a review of this PR which fixes the typos in the method javadocs of com.sun.management.OperatingSystemMXBean? As noted in https://bugs.openjdk.org/browse/JDK-8319465, this PR replaces the word "betweens" by "between" ------------- Commit messages: - 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods Changes: https://git.openjdk.org/jdk/pull/16516/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16516&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319465 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16516.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16516/head:pull/16516 PR: https://git.openjdk.org/jdk/pull/16516 From dholmes at openjdk.org Mon Nov 6 07:55:07 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 6 Nov 2023 07:55:07 GMT Subject: RFR: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods In-Reply-To: References: Message-ID: <2Tal46ut9irLVgKlnknF58R2lJqhaSBRUWDT98rAuhM=.d586df2a-95a9-48c8-835f-60c52defaeb2@github.com> On Mon, 6 Nov 2023 06:55:21 GMT, Jaikiran Pai wrote: > Can I please get a review of this PR which fixes the typos in the method javadocs of com.sun.management.OperatingSystemMXBean? > > As noted in https://bugs.openjdk.org/browse/JDK-8319465, this PR replaces the word "betweens" by "between" Looks good and trivial. Thanks ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16516#pullrequestreview-1714446331 From kevinw at openjdk.org Mon Nov 6 11:22:12 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Mon, 6 Nov 2023 11:22:12 GMT Subject: RFR: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods In-Reply-To: References: Message-ID: <36GlvvOjhpn8DBopNnUItDelKQAV7LYv-izg58OwQIY=.f0f5db1c-2c76-416b-91b5-e2e4df6984aa@github.com> On Mon, 6 Nov 2023 06:55:21 GMT, Jaikiran Pai wrote: > Can I please get a review of this PR which fixes the typos in the method javadocs of com.sun.management.OperatingSystemMXBean? > > As noted in https://bugs.openjdk.org/browse/JDK-8319465, this PR replaces the word "betweens" by "between" Marked as reviewed by kevinw (Committer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16516#pullrequestreview-1714886414 From sjohanss at openjdk.org Mon Nov 6 12:14:21 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 6 Nov 2023 12:14:21 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> Message-ID: <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> On Thu, 2 Nov 2023 01:20:59 GMT, Jonathan Joo wrote: > Could you elaborate a bit on what you were thinking of here? If we are assuming something like a thread that updates all other threads, I think this implementation could get a bit complicated. > > There are two main issues that we can see with a generic thread approach: > > 1. We would have to figure out how often to pull metrics from the various gc threads from the central thread, and possibly determine this frequency separately for every thread. Instead with our current implementation, we can manually trigger publishes based on when the GC thread is done doing work. > > 2. We would still need to tag each thread we want to track somewhere, and keep track of a mapping from thread to its counter name, etc. which doesn't seem to simplify things too much. (I imagine we will still need to touch numerous files to "tag" each thread with whether we want to track it or not?) I agree, I was not thinking about having a separate thread, more about trying to group this information in a way that it would be easier, for example, to provide periodic JFR events for the CPU times collected. Having something like a `CollectorCPUTimeCounters` (we already have `CollectorCounters`). Such a class could keep the different counters making it easier to get an overview of which CPU time counters are present. But this would also require a mapping between thread and counter (but it might be as simple as having an enum). I played around a bit instead of trying to explain what I mean and this is not very polished, but I was thinking something like this: https://github.com/openjdk/jdk/compare/pr/15082...kstefanj:jdk:pull/15082-idea What do you think? This way we don't add things to `CollectedHeap` as well, which is usually good unless really needed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1794686146 From sjohanss at openjdk.org Mon Nov 6 12:20:21 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Mon, 6 Nov 2023 12:20:21 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v37] In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 23:56:25 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - revert gitignore change > - Attempt to fix broken test > The existing sun.management:type=HotspotThreading MBean approach (discussed [here](https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) could be another general way to track CPU. However, the discussion concludes that it is an internal API, and discourages users from using it. > @kstefanj , it is a pity that the `sun.management:type=HotspotThreading` MBean is not exported any more. If we move that or a similar functionality to a new MBean under `com.sun.management` (as proposed in the [cited discussion](https://mail.openjdk.org/pipermail/core-libs-dev/2023-September/111397.html)) then we might reuse these new hsperf counters in the same way this is already done by some other MBeans which already use hsperf counters as their information source. I think logging or JFR functionality could also easily be implemented on top of the new hsperf counters. I haven't looked at the details around this, but extracting some useful information from the internal bean and making it public sound reasonable to me. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1794693970 From dfuchs at openjdk.org Mon Nov 6 12:29:08 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Mon, 6 Nov 2023 12:29:08 GMT Subject: RFR: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods In-Reply-To: References: Message-ID: <6JbKmC6mi4Zv57ozdszwv7VC3Itm-CvpOlMbg7sYAEE=.9bd5c172-2698-4803-a018-fe4227c5f533@github.com> On Mon, 6 Nov 2023 06:55:21 GMT, Jaikiran Pai wrote: > Can I please get a review of this PR which fixes the typos in the method javadocs of com.sun.management.OperatingSystemMXBean? > > As noted in https://bugs.openjdk.org/browse/JDK-8319465, this PR replaces the word "betweens" by "between" Marked as reviewed by dfuchs (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16516#pullrequestreview-1715042344 From jpai at openjdk.org Mon Nov 6 12:58:11 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Mon, 6 Nov 2023 12:58:11 GMT Subject: RFR: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods In-Reply-To: References: Message-ID: <99949XXBV5pfmhYOakTZ4qfiHFxELtjE6KlkK2eFaq4=.c4a82dd9-a2c5-4b79-9016-06ff5174d258@github.com> On Mon, 6 Nov 2023 06:55:21 GMT, Jaikiran Pai wrote: > Can I please get a review of this PR which fixes the typos in the method javadocs of com.sun.management.OperatingSystemMXBean? > > As noted in https://bugs.openjdk.org/browse/JDK-8319465, this PR replaces the word "betweens" by "between" Thank you everyone for the reviews. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16516#issuecomment-1794756060 From jpai at openjdk.org Mon Nov 6 13:01:19 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Mon, 6 Nov 2023 13:01:19 GMT Subject: Integrated: 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods In-Reply-To: References: Message-ID: On Mon, 6 Nov 2023 06:55:21 GMT, Jaikiran Pai wrote: > Can I please get a review of this PR which fixes the typos in the method javadocs of com.sun.management.OperatingSystemMXBean? > > As noted in https://bugs.openjdk.org/browse/JDK-8319465, this PR replaces the word "betweens" by "between" This pull request has now been integrated. Changeset: 2d4bbf47 Author: Jaikiran Pai URL: https://git.openjdk.org/jdk/commit/2d4bbf478745e62584420bfdef5a4948edac54ad Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod 8319465: Typos in javadoc of com.sun.management.OperatingSystemMXBean methods Reviewed-by: dholmes, kevinw, dfuchs ------------- PR: https://git.openjdk.org/jdk/pull/16516 From ayang at openjdk.org Mon Nov 6 13:39:14 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 13:39:14 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc > It is not totally clear how we can get a forwarded object in eden in the first place. This can happen when promotion-failure occurs. Serial and G1 have special code to fix self-forwarded (promotion-fail) objs. Maybe Parallel should do sth similar. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1794843503 From tschatzl at openjdk.org Mon Nov 6 14:51:49 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 6 Nov 2023 14:51:49 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v11] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 15 additional commits since the last revision: - Merge tag 'jdk-22+21' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+21 for changeset d96f38b8 - iwalulya review - typos - ayang review - renamings + documentation - Add documentation about why and how we handle pinned regions in the young/old generation. - Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. - NULL -> nullptr - Fix compilation - Improve TestPinnedOldObjectsEvacuation test - Move tests into gc.g1.pinnedobjs package - ... and 5 more: https://git.openjdk.org/jdk/compare/24b37ee3...251f4d38 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/f9735539..251f4d38 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=09-10 Stats: 4795 lines in 166 files changed: 2660 ins; 1554 del; 581 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From rkennke at openjdk.org Mon Nov 6 16:23:11 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 16:23:11 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc > * JDK-8319376 > > It is not totally clear how we can get a forwarded object in eden in the first place. > > This can happen when promotion-failure occurs. Serial and G1 have special code to fix self-forwarded (promotion-fail) objs. Maybe Parallel should do sth similar. Maybe. But that is not (only) what is happening here. I added a check for self-forwarded into my proposed change and it asserts like this: # Internal Error (/home/rkennke/src/openjdk/jdk/src/hotspot/share/memory/heapInspection.cpp:217), pid=7955, tid=8208 # assert(obj->forwardee() == obj) failed: expect self-forwarded: obj: 0x00000000fac62258, fwd: 0x00000000fd6bac78 ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1795334683 From rkennke at openjdk.org Mon Nov 6 16:27:26 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 16:27:26 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v2] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: <3PbfIJdubuqzX5XYYHQARHUB0HW1PSP3tRH9DMsSz_s=.72565146-c053-42d8-8d9f-dc41b76cfb54@github.com> > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Handle self-forwarded objects correctly ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/4db0df9f..08336840 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From tschatzl at openjdk.org Mon Nov 6 17:20:39 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Mon, 6 Nov 2023 17:20:39 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v12] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 16 additional commits since the last revision: - Merge tag 'jdk-22+22' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+22 for changeset d354141a - Merge tag 'jdk-22+21' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+21 for changeset d96f38b8 - iwalulya review - typos - ayang review - renamings + documentation - Add documentation about why and how we handle pinned regions in the young/old generation. - Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. - NULL -> nullptr - Fix compilation - Improve TestPinnedOldObjectsEvacuation test - ... and 6 more: https://git.openjdk.org/jdk/compare/3af918f1...2ad39680 ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/251f4d38..2ad39680 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=10-11 Stats: 27458 lines in 1170 files changed: 14959 ins; 4156 del; 8343 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From ayang at openjdk.org Mon Nov 6 17:42:09 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Mon, 6 Nov 2023 17:42:09 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v2] In-Reply-To: <3PbfIJdubuqzX5XYYHQARHUB0HW1PSP3tRH9DMsSz_s=.72565146-c053-42d8-8d9f-dc41b76cfb54@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> <3PbfIJdubuqzX5XYYHQARHUB0HW1PSP3tRH9DMsSz_s=.72565146-c053-42d8-8d9f-dc41b76cfb54@github.com> Message-ID: On Mon, 6 Nov 2023 16:27:26 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Handle self-forwarded objects correctly I don't get the assertion any more after adding the following to `PSScavenge::clean_up_failed_promotion`: ParallelScavengeHeap* heap = ParallelScavengeHeap::heap(); struct ResetForwardedMarkWord : ObjectClosure { void do_object(oop obj) override { if (obj->is_forwarded()) { obj->init_mark(); } } } cl; heap->young_gen()->eden_space()->object_iterate(&cl); heap->young_gen()->from_space()->object_iterate(&cl); (Essentially copied from Serial.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1795627388 From rkennke at openjdk.org Mon Nov 6 18:15:00 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 18:15:00 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Restore forwardings after failed promotions ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/08336840..e65910d3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=01-02 Stats: 16 lines in 2 files changed: 12 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From rkennke at openjdk.org Mon Nov 6 19:48:29 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Mon, 6 Nov 2023 19:48:29 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Mon, 6 Nov 2023 18:15:00 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Restore forwardings after failed promotions > I don't get the assertion any more after adding the following to `PSScavenge::clean_up_failed_promotion`: > > ```c++ > ParallelScavengeHeap* heap = ParallelScavengeHeap::heap(); > > struct ResetForwardedMarkWord : ObjectClosure { > void do_object(oop obj) override { > if (obj->is_forwarded()) { > obj->init_mark(); > } > } > } cl; > > heap->young_gen()->eden_space()->object_iterate(&cl); > heap->young_gen()->from_space()->object_iterate(&cl); > ``` > > (Essentially copied from Serial.) Nice, I changed this PR accordingly. I'm curious (and puzzled), though. How would this situation normally have been repaired without this explicit header-resetting code? Would the failed-promotion slide directly into full-GC which repairs it? OTOH, I don't even see code in paralle mark-compact which would explicitely repair forwardings. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1796198151 From sspitsyn at openjdk.org Mon Nov 6 20:07:32 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 6 Nov 2023 20:07:32 GMT Subject: RFR: 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) In-Reply-To: References: <_nZqf39Y7sE1-0vEZfiqvc7Bni2Bw7ctPst0IJrDWbg=.5e1ea5bc-d0f7-4fd9-ab0a-9387902eeac2@github.com> Message-ID: <35zdXMHLcxNkSDr4P-jRNLjZ0ilKCzfD_83fyLoXUlY=.1c1760f4-8806-4487-8094-87ae62921056@github.com> On Mon, 6 Nov 2023 06:16:20 GMT, Alan Bateman wrote: >> My question exactly. I'm not even sure why wrong phase is allowed here. > > Is the issue here that agent thread started by Debuggee.checkStatus is racing with the test? The producer/consumer thread do 1000 put/take ops and it looks like it can complete and VM commence shutdown while the agent thread is observing, do I read this correctly? Thank you for the comments. Alex, you are right. I've misread this code at line 201: 199 err = jvmti->SuspendThread(vthread); 200 if (err == JVMTI_ERROR_THREAD_NOT_ALIVE) { 201 continue; 202 } 203 check_jvmti_status(jni, err, "Error in SuspendThread"); 204 // LOG("Agent: suspended vt: %s ct: %s\n", vname, cname); 205 206 check_vthread_consistency_suspended(jvmti, jni, vthread); The function `check_vthread_consistency_suspended()` is not called when if the `SuspendThread` returned `JVMTI_ERROR_THREAD_NOT_ALIVE` error code. It does not look as a test bug then. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16488#discussion_r1383912422 From sspitsyn at openjdk.org Mon Nov 6 20:16:29 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 6 Nov 2023 20:16:29 GMT Subject: RFR: 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) In-Reply-To: <35zdXMHLcxNkSDr4P-jRNLjZ0ilKCzfD_83fyLoXUlY=.1c1760f4-8806-4487-8094-87ae62921056@github.com> References: <_nZqf39Y7sE1-0vEZfiqvc7Bni2Bw7ctPst0IJrDWbg=.5e1ea5bc-d0f7-4fd9-ab0a-9387902eeac2@github.com> <35zdXMHLcxNkSDr4P-jRNLjZ0ilKCzfD_83fyLoXUlY=.1c1760f4-8806-4487-8094-87ae62921056@github.com> Message-ID: On Mon, 6 Nov 2023 20:04:44 GMT, Serguei Spitsyn wrote: >> Is the issue here that agent thread started by Debuggee.checkStatus is racing with the test? The producer/consumer thread do 1000 put/take ops and it looks like it can complete and VM commence shutdown while the agent thread is observing, do I read this correctly? > > Thank you for the comments. > Alex, you are right. > I've misread this code at line 201: > > 199 err = jvmti->SuspendThread(vthread); > 200 if (err == JVMTI_ERROR_THREAD_NOT_ALIVE) { > 201 continue; > 202 } > 203 check_jvmti_status(jni, err, "Error in SuspendThread"); > 204 // LOG("Agent: suspended vt: %s ct: %s\n", vname, cname); > 205 > 206 check_vthread_consistency_suspended(jvmti, jni, vthread); > > The function `check_vthread_consistency_suspended()` is not called when if the `SuspendThread` returned `JVMTI_ERROR_THREAD_NOT_ALIVE` error code. > > It does not look as a test bug then. David and Alan, Yes, there is always a race between the app and the JVMTI agent execution. This was always a general issue with the JVMTI. The agent code executing JVMTI functions and events is not protected from the `JVMTI_ERROR_WRONG_PHASE` errors. This error code can be returned by JVMTI functions even in the middle of the event callback execution. We had this problem in the JDWP agent and fixed it by adding proper synchronization into the agent itself. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16488#discussion_r1383923778 From sspitsyn at openjdk.org Mon Nov 6 23:22:03 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 6 Nov 2023 23:22:03 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: Message-ID: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes three new classes: > - `JvmtiHandshake`, `JvmtiUnifiedHandshakeClosure`, VM_HandshakeUnmountedVirtualThread > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: get rid of the VM_HandshakeUnmountedVirtualThread ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/720c9c7e..ca2fbb98 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=02-03 Stats: 38 lines in 4 files changed: 0 ins; 29 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From sspitsyn at openjdk.org Mon Nov 6 23:31:58 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 6 Nov 2023 23:31:58 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> Message-ID: On Mon, 6 Nov 2023 23:22:03 GMT, Serguei Spitsyn wrote: >> The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. >> At the low level, the JVMTI code supporting platform and virtual threads still can be different. >> This implementation is based on the `JvmtiVTMSTransitionDisabler` class. >> >> The internal API includes two new classes: >> - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` >> >> The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. >> >> The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: >> - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` >> >> To get the test results clean, the update also fixes the test issue: >> [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" >> >> Testing: >> - the mach5 tiers 1-6 are all passed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: get rid of the VM_HandshakeUnmountedVirtualThread I've pushed an update which removes newly introduced VM_op and its use: `VM_HandshakeUnmountedVirtualThread`. Patricio convinced me that it has to be handshake-safe to execute a `HanshakeClosure` callback on the current (handshake requesting) thread when target thread is an unmounted virtual threads. At an earlier development stage I saw various intermittent crashes and concluded it is not handshake-safe. It is why there was a decision to use the `VM_HandshakeUnmountedVirtualThread`. I do not see these crashes anymore after a full testing cycle. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16460#issuecomment-1797026470 From jjoo at openjdk.org Tue Nov 7 00:52:07 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 7 Nov 2023 00:52:07 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v38] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Attempt to fix duplicate name error in test ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/9fb36a9e..ac780c5e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=37 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=36-37 Stats: 8 lines in 1 file changed: 0 ins; 5 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From manc at openjdk.org Tue Nov 7 01:08:33 2023 From: manc at openjdk.org (Man Cao) Date: Tue, 7 Nov 2023 01:08:33 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> Message-ID: On Mon, 6 Nov 2023 12:11:47 GMT, Stefan Johansson wrote: > I played around a bit instead of trying to explain what I mean and this is not very polished, but I was thinking something like this: https://github.com/openjdk/jdk/compare/pr/15082...kstefanj:jdk:pull/15082-idea > > What do you think? This way we don't add things to CollectedHeap as well, which is usually good unless really needed. I think it looks great. It is mainly refactoring that consolidates the declarations/definitions of the hsperf counters in to a single file. Would it be better to name that class `CPUTimeCounters`, so we could move `sun.threads.cpu_time.vm` and `sun.threads.cpu_time.conc_dedup`, and future JIT thread CPU counters to that class? Then we could also change the constructor of `ThreadTotalCPUTimeClosure` to `ThreadTotalCPUTimeClosure(CPUTimeCounters* counters, CPUTimeGroups::Name name)`, then it could set `_update_gc_counters` based on `name`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1797109487 From tschatzl at openjdk.org Tue Nov 7 08:31:54 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 08:31:54 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v13] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Fix tests after merge ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/2ad39680..d9ccccff Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=12 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=11-12 Stats: 36 lines in 3 files changed: 0 ins; 0 del; 36 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From sjohanss at openjdk.org Tue Nov 7 09:57:40 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Tue, 7 Nov 2023 09:57:40 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v35] In-Reply-To: References: <6tngC-Jwyx8e25LGT8dAwKbaPb9qb_w5ONctnFieH3o=.61b013b2-beb9-4053-8c06-86a700208d77@github.com> <3iJqYOiXeO6bwtUmjzlO3tyFR9Uc28YAJ8aQbKbqKJM=.fda95001-86f3-4f09-8a74-e15be1987c4f@github.com> Message-ID: <1CHY1oIjteVS6FU__jUkfO6nTijg9j3tRbv_QJl3QLI=.18e3b2ba-3ab2-4ee8-ab5f-1dd30323f671@github.com> On Tue, 7 Nov 2023 01:06:12 GMT, Man Cao wrote: > I think it looks great. It is mainly refactoring that consolidates the declarations/definitions of the hsperf counters in to a single file. Would it be better to name that class `CPUTimeCounters`, so we could move `sun.threads.cpu_time.vm` and `sun.threads.cpu_time.conc_dedup`, and future JIT thread CPU counters to that class? > Yes, mainly refactoring and I was thinking along the same lines, but since this patch was just for GC and we had `CollectorCounters` already I went with this. I think calling the class `CPUTimeCounters` would be good and place it outside GC makes sense if we plan to include even more CPU time counters. Another name that we could improve is `CPUTimeGroups` and maybe also the enum name `Name`, they are ok, but we might come up with something better. > Then we could also change the constructor of `ThreadTotalCPUTimeClosure` to `ThreadTotalCPUTimeClosure(CPUTimeCounters* counters, CPUTimeGroups::Name name)`, then it could set `_update_gc_counters` based on `name`. I was looking at this too, but had to restructure the code more to avoid circular deps. If we create the general `CPUTImeCounters` we could move this closure to that file and then things would fit better I believe. So I like your proposals. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1798170871 From ayang at openjdk.org Tue Nov 7 10:04:31 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 10:04:31 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> On Mon, 6 Nov 2023 19:46:07 GMT, Roman Kennke wrote: > Would the failed-promotion slide directly into full-GC which repairs it? No, promotion-fail objects are specially handled, as documented in `PSPromotionManager::oop_promotion_failed`. Other successfully forwarded objects are essentially dead, so during a full garbage collection, they are skipped. If a young garbage collection is successful, the eden/from space is cleared, which is why object iteration doesn't expose the "forwarded" markword. The patch above is just intended to demonstrate a way of suppressing the assertion, but it's likely not the best or correct way to proceed. Perhaps object-iteration logic in ParallelGC should skip those with the "forwarded" markword (because they are dead objs)? (This makes sense to me semantically, but I'm not sure about the potential consequences, given that object iteration is possibly used in many places.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798184871 From kevinw at openjdk.org Tue Nov 7 10:18:40 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Tue, 7 Nov 2023 10:18:40 GMT Subject: Integrated: 8319238: JMX ThreadPoolAccTest.java is too verbose and should fail before timeout In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 17:10:34 GMT, Kevin Walls wrote: > Discovered while testing changes that made this test fail. The test failure is hard to diagnose as it logs and retries at full speed, possibly forever, until timeout. This can hit a log file limit. We can save thousands of lines of text being printed when the test runs normally and successfully, by waiting half a second before doing the Principal-checking which is the purpose of the test. This pull request has now been integrated. Changeset: a7c01902 Author: Kevin Walls URL: https://git.openjdk.org/jdk/commit/a7c0190230825e998bb534721ed3c22904efdbb4 Stats: 26 lines in 1 file changed: 23 ins; 0 del; 3 mod 8319238: JMX ThreadPoolAccTest.java is too verbose and should fail before timeout Reviewed-by: sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/16456 From tschatzl at openjdk.org Tue Nov 7 10:52:35 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 10:52:35 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: <8-NYDad3UQZM_JyiOqxRmfLVcD4L0_W0tX6LrpKmuZk=.abc09ef7-b6bc-4720-a8ab-46c193b68330@github.com> On Mon, 6 Nov 2023 19:46:07 GMT, Roman Kennke wrote: > I'm curious (and puzzled), though. How would this situation normally have been repaired without this explicit header-resetting code? Would the failed-promotion slide directly into full-GC which repairs it? OTOH, I don't even see code in paralle mark-compact which would explicitely repair forwardings. After full gc the from/to/survivor spaces should be reset to have their tops point to bottom. Maybe this is not done for all of its space, e.g. only for eden and to-space. At least I could not find anything related at a quick glance. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798264144 From azafari at openjdk.org Tue Nov 7 11:40:02 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Tue, 7 Nov 2023 11:40:02 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v8] In-Reply-To: References: Message-ID: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> > The `find` method now is > ```C++ > template > int find(T* token, bool f(T*, E)) const { > ... > > Any other functions which use this are also changed. > Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: function pointer is replaced with template Functor. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15418/files - new: https://git.openjdk.org/jdk/pull/15418/files/6d9288e7..7665b878 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=06-07 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/15418.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15418/head:pull/15418 PR: https://git.openjdk.org/jdk/pull/15418 From rkennke at openjdk.org Tue Nov 7 12:40:32 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 12:40:32 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> Message-ID: On Tue, 7 Nov 2023 10:02:04 GMT, Albert Mingkun Yang wrote: > > Would the failed-promotion slide directly into full-GC which repairs it? > > No, promotion-fail objects are specially handled, as documented in `PSPromotionManager::oop_promotion_failed`. Other successfully forwarded objects are essentially dead, so during a full garbage collection, they are skipped. Ok. Just to clarify: After failed promotion, all references to original copies of promoted objects are updated, and therefore the original copies are dead (and can safely be skipped)? Or is has no reference been updated yet, and therefore the new copies are dead and can be skipped? > If a young garbage collection is successful, the eden/from space is cleared, which is why object iteration doesn't expose the "forwarded" markword. Yes, that makes sense. > The patch above is just intended to demonstrate a way of suppressing the assertion, but it's likely not the best or correct way to proceed. I thought so. > Perhaps object-iteration logic in ParallelGC should skip those with the "forwarded" markword (because they are dead objs)? (This makes sense to me semantically, but I'm not sure about the potential consequences, given that object iteration is possibly used in many places.) If the first statement above is true, then this seems reasonable. I will give it a try. Roman ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798419996 From ayang at openjdk.org Tue Nov 7 12:47:33 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 12:47:33 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v3] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> <1KlNGWoYqYFKUBBqosFwRiAST-R-fjH5aiYXv7BCImE=.c8b568ad-21b2-42c5-bae1-4413707a878f@github.com> Message-ID: <6nUhJZh2N3GblEkFDLXQ9o2QSEreD8DR8YbQQlJSooY=.2941018d-27ba-4639-99aa-7e6fa13136b3@github.com> On Tue, 7 Nov 2023 12:38:14 GMT, Roman Kennke wrote: > all references to original copies of promoted objects are updated, and therefore the original copies are dead (and can safely be skipped) Yes; after scavenging, all pointers are correct (same value for promotion-fail and new value otherwise). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1798430442 From tschatzl at openjdk.org Tue Nov 7 14:11:20 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 14:11:20 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v14] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: "GCLocker Initiated GC" is not a valid GC cause for G1 any more ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/d9ccccff..c272a736 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=13 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=12-13 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From rkennke at openjdk.org Tue Nov 7 14:31:49 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 14:31:49 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v4] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Don't expose forwarded objects during object iteration ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/e65910d3..78ffecb1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=02-03 Stats: 16 lines in 2 files changed: 3 ins; 12 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From ayang at openjdk.org Tue Nov 7 14:40:33 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 14:40:33 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v4] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 14:31:49 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Don't expose forwarded objects during object iteration src/hotspot/share/gc/parallel/mutableSpace.cpp line 239: > 237: while (p < top()) { > 238: oop obj = cast_to_oop(p); > 239: if (!obj->is_forwarded()) { Maybe some comment along these lines "When promotion-failure occurs during Young GC, eden/from space is not cleared, so we can encounter objs with "forwarded" markword. They are essentially dead, so skipping them." src/hotspot/share/gc/parallel/mutableSpace.cpp line 242: > 240: cl->do_object(cast_to_oop(p)); > 241: } > 242: p += cast_to_oop(p)->size(); One can use `obj` instead of `cast_to_oop`. src/hotspot/share/memory/heapInspection.cpp line 216: > 214: // of running out of space required to create a new entry. > 215: bool KlassInfoTable::record_instance(const oop obj) { > 216: assert(!obj->is_forwarded(), "must not be forwarded"); It'd be good if someone more familiar with this method/class can comment whether this assert makes sense in this context. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385018851 PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385014921 PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385020892 From pchilanomate at openjdk.org Tue Nov 7 14:56:36 2023 From: pchilanomate at openjdk.org (Patricio Chilano Mateo) Date: Tue, 7 Nov 2023 14:56:36 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> Message-ID: On Mon, 6 Nov 2023 23:22:03 GMT, Serguei Spitsyn wrote: >> The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. >> At the low level, the JVMTI code supporting platform and virtual threads still can be different. >> This implementation is based on the `JvmtiVTMSTransitionDisabler` class. >> >> The internal API includes two new classes: >> - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` >> >> The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. >> >> The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: >> - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` >> >> To get the test results clean, the update also fixes the test issue: >> [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" >> >> Testing: >> - the mach5 tiers 1-6 are all passed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: get rid of the VM_HandshakeUnmountedVirtualThread Hi Serguei, Looks good to me, nice code consolidation. src/hotspot/share/prims/jvmtiEnvBase.cpp line 1974: > 1972: > 1973: if (java_lang_VirtualThread::is_instance(target_h())) { // virtual thread > 1974: if (!JvmtiEnvBase::is_vthread_alive(target_h())) { There is only one issue I see in how this check is implemented and the removal of the VM_op for unmounted vthreads. The change of state to TERMINATED happens after notifyJvmtiUnmount(), i.e we can see that this vthread is alive here but a check later can return is not. This might hit the assert in JvmtiEnvBase::get_vthread_jvf() (maybe this the issue you saw on your first prototype). We can either change that order at the Java level, or maybe better change this function to read the state and add a case where if the state is RUNNING check whether the continuation is done or not (jdk_internal_vm_Continuation::done(cont)). src/hotspot/share/prims/jvmtiEnvBase.cpp line 1978: > 1976: } > 1977: if (target_jt == nullptr) { // unmounted virtual thread > 1978: hs_cl->do_vthread(target_h); // execute handshake closure callback on current thread directly I think comment should be: s/current thread/unmounted vthread src/hotspot/share/prims/jvmtiEnvBase.cpp line 2416: > 2414: if (!JvmtiEnvBase::is_vthread_alive(_target_h())) { > 2415: return; // JVMTI_ERROR_THREAD_NOT_ALIVE (default) > 2416: } Don't we have this check already in JvmtiHandshake::execute()? Same with the other converted functions. src/hotspot/share/prims/jvmtiEnvBase.hpp line 490: > 488: class JvmtiHandshake : public Handshake { > 489: protected: > 490: static bool is_vthread_handshake_safe(JavaThread* thread, oop vt); Not defined, leftover? ------------- PR Review: https://git.openjdk.org/jdk/pull/16460#pullrequestreview-1717815943 PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1385033726 PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1384994419 PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1384999063 PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1385002852 From rkennke at openjdk.org Tue Nov 7 14:59:48 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 14:59:48 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v5] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Review comments; expose self-forwarded objects ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/78ffecb1..988b96e6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=03-04 Stats: 7 lines in 2 files changed: 3 ins; 2 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From ayang at openjdk.org Tue Nov 7 15:13:34 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 15:13:34 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v5] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 14:59:48 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Review comments; expose self-forwarded objects src/hotspot/share/gc/parallel/mutableSpace.cpp line 242: > 240: // so we can encounter objects with "forwarded" markword. > 241: // They are essentially dead, so skipping them > 242: if (!obj->is_forwarded() || obj->forwardee() == obj) { Self-forwarded ones should already be handled by `PSPromotionManager::restore_preserved_marks`, so the second condition will always be false. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16494#discussion_r1385076675 From jsjolen at openjdk.org Tue Nov 7 15:54:35 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Tue, 7 Nov 2023 15:54:35 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v6] In-Reply-To: References: <77doYU5p16r_I0mXIT9LoSCHYv1gPrbNTo_FdUd4UCs=.9d2cfc5a-1a84-4b2d-9624-badfc928f540@github.com> Message-ID: On Wed, 1 Nov 2023 10:04:30 GMT, Johan Sj?len wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> fix various builds > > OK, went through the cache. Will continue later. > @jdksjolen anything more needed from my side? I'll work through this tomorrow, just to make sure I didn't miss anything. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1799002931 From tschatzl at openjdk.org Tue Nov 7 18:11:52 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Nov 2023 18:11:52 GMT Subject: RFR: 8318706: Implementation of JDK-8276094: JEP 423: Region Pinning for G1 [v15] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 19 commits: - Merge branch 'master' into 8318706-implementation-of-region-pinning-in-g1 - "GCLocker Initiated GC" is not a valid GC cause for G1 any more - Fix tests after merge - Merge tag 'jdk-22+22' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+22 for changeset d354141a - Merge tag 'jdk-22+21' into 8318706-implementation-of-region-pinning-in-g1 Added tag jdk-22+21 for changeset d96f38b8 - iwalulya review - typos - ayang review - renamings + documentation - Add documentation about why and how we handle pinned regions in the young/old generation. - Renamings to (almost) consistently use the following nomenclature for evacuation failure and types of it: * evacuation failure is the general concept. It includes * pinned regions * allocation failure One region can both be pinned and experience an allocation failure. G1 GC messages use tags "(Pinned)" and "(Allocation Failure)" now instead of "(Evacuation Failure)" Did not rename the G1EvacFailureInjector since this adds a lot of noise. - ... and 9 more: https://git.openjdk.org/jdk/compare/45e68ae2...83eff9fe ------------- Changes: https://git.openjdk.org/jdk/pull/16342/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=14 Stats: 1820 lines in 59 files changed: 1147 ins; 430 del; 243 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From rkennke at openjdk.org Tue Nov 7 18:12:45 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Tue, 7 Nov 2023 18:12:45 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [ ] tier2 -XX:+UseParallelGC > - [ ] hotspot_gc Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Do *not* handle self-forwarded objects ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16494/files - new: https://git.openjdk.org/jdk/pull/16494/files/988b96e6..6ba7c713 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16494&range=04-05 Stats: 6 lines in 1 file changed: 5 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16494.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16494/head:pull/16494 PR: https://git.openjdk.org/jdk/pull/16494 From ayang at openjdk.org Tue Nov 7 18:17:31 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Tue, 7 Nov 2023 18:17:31 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 18:12:45 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [ ] tier2 -XX:+UseParallelGC >> - [ ] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Do *not* handle self-forwarded objects Marked as reviewed by ayang (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16494#pullrequestreview-1718395065 From mbaesken at openjdk.org Wed Nov 8 08:35:05 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Wed, 8 Nov 2023 08:35:05 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX Message-ID: On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) at java.base/java.lang.Thread.run(Thread.java:1570) Caused by: java.lang.OutOfMemoryError: Metaspace at java.base/java.time.Duration.(Duration.java:149) at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) at java.base/java.time.LocalDate.of(LocalDate.java:272) at java.base/java.time.LocalDate.(LocalDate.java:147) at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.pathsAndContainers(JavacFileManager.java:1078) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.indexPathsAndContainersByRelativeDirectory(JavacFileManager.java:1033) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$$Lambda/0x00000328001a3168.apply(Unknown Source) at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1228) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.pathsAndContainers(JavacFileManager.java:1021) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.list(JavacFileManager.java:777) at [java.compiler at 22-internal](mailto:java.compiler at 22-internal)/javax.tools.ForwardingJavaFileManager.list(ForwardingJavaFileManager.java:82) at jdk.compiler/com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager.list(ClientCodeWrapper.java:223) at jdk.compiler/com.sun.tools.javac.code.ClassFinder.list(ClassFinder.java:752) at jdk.compiler/com.sun.tools.javac.code.ClassFinder.scanUserPaths(ClassFinder.java:689) at jdk.compiler/com.sun.tools.javac.code.ClassFinder.fillIn(ClassFinder.java:570) at jdk.compiler/com.sun.tools.javac.code.ClassFinder.complete(ClassFinder.java:311) at jdk.compiler/com.sun.tools.javac.code.ClassFinder$$Lambda/0x00000328001287f0.complete(Unknown Source) at jdk.compiler/com.sun.tools.javac.code.Symtab.lambda$addRootPackageFor$8(Symtab.java:855) Seems the MetaSpace settings of the test need to be adjusted; 17m works on the other platforms but AIX needs 23m. I tested with higher iteration value in main of RedefineLeakThrowable so see if it is not a leak and with 23m it passes too on AIX with higher iteration value. For some reason, with product binaries 21m was sufficient too (but fastdebug needed a bit more). Should we keep the old MetaSpace value 17m for non - AIX ? ------------- Commit messages: - Adjust start section of the test - JDK-8319375 Changes: https://git.openjdk.org/jdk/pull/16553/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16553&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319375 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16553.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16553/head:pull/16553 PR: https://git.openjdk.org/jdk/pull/16553 From jsjolen at openjdk.org Wed Nov 8 09:54:06 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 8 Nov 2023 09:54:06 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 17:42:29 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > Fix another windows error As this adds a JCmd, doesn't this need both a CSR and a manual entry? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1801457657 From ayang at openjdk.org Wed Nov 8 10:59:34 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Wed, 8 Nov 2023 10:59:34 GMT Subject: RFR: 8319703: Serial: Remove generationSpec Message-ID: Simple cleanup of removing an unnecessary indirection. The real change is only 11 LOC, using the global variables directly for young/old gen. ------------- Commit messages: - s1-remove-gen-spec Changes: https://git.openjdk.org/jdk/pull/16554/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16554&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319703 Stats: 265 lines in 12 files changed: 1 ins; 254 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/16554.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16554/head:pull/16554 PR: https://git.openjdk.org/jdk/pull/16554 From stuefe at openjdk.org Wed Nov 8 11:27:03 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 8 Nov 2023 11:27:03 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: Message-ID: <4e6l0dEPEUZ3OXDZpQmbnxwgRupmQyYt66QTWW5PGRA=.8d004fe7-0821-4487-997e-57ec8dbea648@github.com> On Wed, 8 Nov 2023 09:51:12 GMT, Johan Sj?len wrote: > As this adds a JCmd, doesn't this need both a CSR and a manual entry? - CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. - Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1801695544 From jsjolen at openjdk.org Wed Nov 8 11:43:05 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 8 Nov 2023 11:43:05 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: <4e6l0dEPEUZ3OXDZpQmbnxwgRupmQyYt66QTWW5PGRA=.8d004fe7-0821-4487-997e-57ec8dbea648@github.com> References: <4e6l0dEPEUZ3OXDZpQmbnxwgRupmQyYt66QTWW5PGRA=.8d004fe7-0821-4487-997e-57ec8dbea648@github.com> Message-ID: On Wed, 8 Nov 2023 11:24:43 GMT, Thomas Stuefe wrote: > > As this adds a JCmd, doesn't this need both a CSR and a manual entry? > > * CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. > > * Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? @dholmes-ora, would you mind helping us out here with regards to the process? Would a new JCmd require a CSR or would it be acceptable to merge without one? Thank you. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1801718561 From tschatzl at openjdk.org Wed Nov 8 11:44:59 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Wed, 8 Nov 2023 11:44:59 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 18:12:45 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [x] tier2 -XX:+UseParallelGC >> - [x] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Do *not* handle self-forwarded objects Marked as reviewed by tschatzl (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16494#pullrequestreview-1720129165 From kevinw at openjdk.org Wed Nov 8 13:23:56 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Wed, 8 Nov 2023 13:23:56 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 08:29:52 GMT, Matthias Baesken wrote: > On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: > > java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) > at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) > at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) > at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) > at java.base/java.lang.Thread.run(Thread.java:1570) > Caused by: java.lang.OutOfMemoryError: Metaspace > at java.base/java.time.Duration.(Duration.java:149) > at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) > at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) > at java.base/java.time.LocalDate.of(LocalDate.java:272) > at java.base/java.time.LocalDate.(LocalDate.java:147) > at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) > at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.pathsA... Reasonable to increase for all platforms. JDK-8316658 has a note (comment - [2023-09-22 07:56) suggesting 24M but the actual change went with 17M. ------------- Marked as reviewed by kevinw (Committer). PR Review: https://git.openjdk.org/jdk/pull/16553#pullrequestreview-1720339742 From mbaesken at openjdk.org Wed Nov 8 14:43:21 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Wed, 8 Nov 2023 14:43:21 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case Message-ID: In parseAllowedMask (file socketTransport.c) , prefixLen of mask is compared with a maxValue (32 for IPv4, 128 otherwise). This fails on AIX if it is larger than 32, because getaddrinfo seems to often (always ?) detect IPv4 family, even for IPv6 addresses, so we take the wrong maxValue. Probably we have to adjust the allowed maxValue on AIX, or adjust the IPv6 check. Example: images/jdk/bin/java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:0,allow=0:0:0:0:0:0:10:0/106 Error in allow option: '106' ERROR: transport error 103: invalid netmask in allow option ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) ------------- Commit messages: - JDK-8319382 Changes: https://git.openjdk.org/jdk/pull/16561/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16561&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319382 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16561.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16561/head:pull/16561 PR: https://git.openjdk.org/jdk/pull/16561 From sjohanss at openjdk.org Wed Nov 8 14:49:00 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 8 Nov 2023 14:49:00 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v9] In-Reply-To: References: Message-ID: <6_MNWrkq_LnvB4EqrBFNo6e6kVHLgffcRbkxhRPzTpg=.1a05d8af-b181-4e80-9922-6aea9a517564@github.com> On Fri, 3 Nov 2023 14:14:36 GMT, Albert Mingkun Yang wrote: >> Parsing the separate components is easier :) Not sure if these tags in any way ever indicated some level of abstraction. >> >> I do not have a strong opinion here. The combinations >> >> (Pinned) >> (Allocation Failure) >> (Pinned + Allocation Failure) // or the other way around, or some other symbol for "+" or no symbol at all? >> >> are fine with me (and I thought about doing something more elaborate here), but my concern has been that any complicated string makes it less unique (e.g. `(Allocation Failure)` vs. "Allocation Failure") and adds code both to implement and parse the result. >> >> Much more disrupting is likely that there is no "Evacuation Failure" string any more. But log messages are not part of the external interface, and we should not want to change them just because. > > The example looks good to me. Have the final output looking something like this was agreed on during internal discussion: GC(6) Pause Young (Normal) (Evacuation Failure: Pinned) 1M->1M(22M) 36.16ms GC(6) Pause Young (Normal) (Evacuation Failure: Allocation) 1M->1M(22M) 36.16ms GC(6) Pause Young (Normal) (Evacuation Failure: Allocation / Pinned) 1M->1M(22M) 36.16ms ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1386736860 From rkennke at openjdk.org Wed Nov 8 15:21:06 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 8 Nov 2023 15:21:06 GMT Subject: RFR: 8319376: ParallelGC: Forwarded objects found during heap inspection [v6] In-Reply-To: References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Tue, 7 Nov 2023 18:12:45 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> Testing: >> - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC >> - [x] tier1 -XX:+UseParallelGC >> - [x] tier2 -XX:+UseParallelGC >> - [x] hotspot_gc > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Do *not* handle self-forwarded objects Thanks! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16494#issuecomment-1802098998 From rkennke at openjdk.org Wed Nov 8 15:21:08 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 8 Nov 2023 15:21:08 GMT Subject: Integrated: 8319376: ParallelGC: Forwarded objects found during heap inspection In-Reply-To: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> References: <5AqUR0kRw2-IhZobZQEN0URFpLkm6T4OG_21IiVOdec=.708b6631-afcb-4e1f-bc09-6f76f92bf2be@github.com> Message-ID: On Fri, 3 Nov 2023 12:42:46 GMT, Roman Kennke wrote: > See JBS issue for details. > > Testing: > - [x] gc/logging/TestUnifiedLoggingSwitchStress.java -XX:+UseParallelGC > - [x] tier1 -XX:+UseParallelGC > - [x] tier2 -XX:+UseParallelGC > - [x] hotspot_gc This pull request has now been integrated. Changeset: 59e9981e Author: Roman Kennke URL: https://git.openjdk.org/jdk/commit/59e9981ec21258b8aa5f93cb1fb9b0ccf9f846af Stats: 12 lines in 1 file changed: 11 ins; 0 del; 1 mod 8319376: ParallelGC: Forwarded objects found during heap inspection Co-authored-by: Albert Mingkun Yang Reviewed-by: ayang, tschatzl ------------- PR: https://git.openjdk.org/jdk/pull/16494 From sspitsyn at openjdk.org Wed Nov 8 16:02:03 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 8 Nov 2023 16:02:03 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> Message-ID: On Tue, 7 Nov 2023 14:54:05 GMT, Patricio Chilano Mateo wrote: > Hi Serguei, > Looks good to me, nice code consolidation. Hi Patricio, thank you a lot for reviewing this! > src/hotspot/share/prims/jvmtiEnvBase.cpp line 1978: > >> 1976: } >> 1977: if (target_jt == nullptr) { // unmounted virtual thread >> 1978: hs_cl->do_vthread(target_h); // execute handshake closure callback on current thread directly > > I think comment should be: s/current thread/unmounted vthread Thank you for the comment but I'm not sure what do you mean. If target virtual thread is unmounted we execute the hs_cl callback on current thread. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16460#issuecomment-1802188368 PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1386857320 From sspitsyn at openjdk.org Wed Nov 8 16:05:03 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 8 Nov 2023 16:05:03 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> Message-ID: On Tue, 7 Nov 2023 14:23:34 GMT, Patricio Chilano Mateo wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: get rid of the VM_HandshakeUnmountedVirtualThread > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 2416: > >> 2414: if (!JvmtiEnvBase::is_vthread_alive(_target_h())) { >> 2415: return; // JVMTI_ERROR_THREAD_NOT_ALIVE (default) >> 2416: } > > Don't we have this check already in JvmtiHandshake::execute()? Same with the other converted functions. Good suggestion, thanks. I'm a little bit paranoid about terminated vthreads. :) Will try to get rid and retest all tiers. > src/hotspot/share/prims/jvmtiEnvBase.hpp line 490: > >> 488: class JvmtiHandshake : public Handshake { >> 489: protected: >> 490: static bool is_vthread_handshake_safe(JavaThread* thread, oop vt); > > Not defined, leftover? Good catch, thanks! Will remove it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1386861657 PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1386862308 From sspitsyn at openjdk.org Wed Nov 8 16:14:01 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 8 Nov 2023 16:14:01 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> Message-ID: <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> On Tue, 7 Nov 2023 14:44:52 GMT, Patricio Chilano Mateo wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: get rid of the VM_HandshakeUnmountedVirtualThread > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 1974: > >> 1972: >> 1973: if (java_lang_VirtualThread::is_instance(target_h())) { // virtual thread >> 1974: if (!JvmtiEnvBase::is_vthread_alive(target_h())) { > > There is only one issue I see in how this check is implemented and the removal of the VM_op for unmounted vthreads. The change of state to TERMINATED happens after notifyJvmtiUnmount(), i.e we can see that this vthread is alive here but a check later can return is not. This might hit the assert in JvmtiEnvBase::get_vthread_jvf() (maybe this the issue you saw on your first prototype). We can either change that order at the Java level, or maybe better change this function to read the state and add a case where if the state is RUNNING check whether the continuation is done or not (jdk_internal_vm_Continuation::done(cont)). Thank you for the suggestion. Will check it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1386875290 From lucy at openjdk.org Wed Nov 8 16:15:58 2023 From: lucy at openjdk.org (Lutz Schmidt) Date: Wed, 8 Nov 2023 16:15:58 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 08:29:52 GMT, Matthias Baesken wrote: > On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: > > java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) > at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) > at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) > at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) > at java.base/java.lang.Thread.run(Thread.java:1570) > Caused by: java.lang.OutOfMemoryError: Metaspace > at java.base/java.time.Duration.(Duration.java:149) > at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) > at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) > at java.base/java.time.LocalDate.of(LocalDate.java:272) > at java.base/java.time.LocalDate.(LocalDate.java:147) > at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) > at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.pathsA... LGTM. Thanks for verifying we are not hiding a leak with the increase. ------------- Marked as reviewed by lucy (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16553#pullrequestreview-1720751570 From cjplummer at openjdk.org Wed Nov 8 17:33:57 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 8 Nov 2023 17:33:57 GMT Subject: RFR: 8319703: Serial: Remove generationSpec In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 10:51:19 GMT, Albert Mingkun Yang wrote: > Simple cleanup of removing an unnecessary indirection. The real change is only 11 LOC, using the global variables directly for young/old gen. SA changes look good. ------------- Marked as reviewed by cjplummer (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16554#pullrequestreview-1720915415 From cjplummer at openjdk.org Wed Nov 8 18:00:58 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 8 Nov 2023 18:00:58 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 14:37:29 GMT, Matthias Baesken wrote: > In parseAllowedMask (file socketTransport.c) , prefixLen of mask is compared with a maxValue (32 for IPv4, 128 otherwise). This fails on AIX if it is larger than 32, because getaddrinfo seems to often (always ?) detect IPv4 family, even for IPv6 addresses, so we take the wrong maxValue. > Probably we have to adjust the allowed maxValue on AIX, or adjust the IPv6 check. > > Example: > images/jdk/bin/java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:0,allow=0:0:0:0:0:0:10:0/106 > Error in allow option: '106' > ERROR: transport error 103: invalid netmask in allow option > ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c line 468: > 466: maxValue = 128; > 467: #endif > 468: IIUC, this code would work for any port, but is only required for AIX. And since it is a hack to work around an AIX issue, you didn't want to subject all ports to it. The downside of this change is that it will cause the option parsing code to not properly produce an error if an IPv4 mask prefix between 32 and 128 is given. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16561#discussion_r1387008497 From rkennke at openjdk.org Wed Nov 8 19:06:17 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 8 Nov 2023 19:06:17 GMT Subject: RFR: 8318895: Deoptimization results in incorrect lightweight locking stack Message-ID: See JBS issue for details. I basically: - took the test-modification and turned it into its own test-case - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. Testing: - [x] EATests.java - [ ] tier1 - [ ] tier2 ------------- Commit messages: - 8318895: Deoptimization results in incorrect lightweight locking stack Changes: https://git.openjdk.org/jdk/pull/16568/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16568&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8318895 Stats: 99 lines in 2 files changed: 96 ins; 0 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16568.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16568/head:pull/16568 PR: https://git.openjdk.org/jdk/pull/16568 From amenkov at openjdk.org Wed Nov 8 21:57:56 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 8 Nov 2023 21:57:56 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: References: Message-ID: <83tY8a_r4Yh4fni2jQoi0Gz3PxBfE-EGs7U5AImK6-Q=.2cc49085-75ad-4962-969f-9cceb0764ec6@github.com> On Wed, 8 Nov 2023 14:37:29 GMT, Matthias Baesken wrote: > In parseAllowedMask (file socketTransport.c) , prefixLen of mask is compared with a maxValue (32 for IPv4, 128 otherwise). This fails on AIX if it is larger than 32, because getaddrinfo seems to often (always ?) detect IPv4 family, even for IPv6 addresses, so we take the wrong maxValue. > Probably we have to adjust the allowed maxValue on AIX, or adjust the IPv6 check. > > Example: > images/jdk/bin/java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:0,allow=0:0:0:0:0:0:10:0/106 > Error in allow option: '106' > ERROR: transport error 103: invalid netmask in allow option > ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) The fix looks wrong. The problem is not with parsing of the prefix len, but with parsing of the address itself (parseAllowedAddr function). Note that parseAllowedAddr converts returned IPv4 addresses to mapped IPv6 and in the example provided it won't be expected ::10:0, so the "allow" logic will be broken. I think the issue need some research on the affected systems: - if getaddrinfo returns AF_INET, then what is returned in ai_addr field? it must be valid IPv4 address (which can be used for connect()); - does getaddrinfo return a single address? (parseAllowedAddr assumes so, but maybe that's wrong in the case) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16561#issuecomment-1802741222 From jjoo at openjdk.org Thu Nov 9 00:42:33 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 9 Nov 2023 00:42:33 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v39] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Refactor changes to counters, successful build ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/ac780c5e..22ccb909 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=38 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=37-38 Stats: 149 lines in 16 files changed: 26 ins; 112 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From dlong at openjdk.org Thu Nov 9 01:30:55 2023 From: dlong at openjdk.org (Dean Long) Date: Thu, 9 Nov 2023 01:30:55 GMT Subject: RFR: 8318895: Deoptimization results in incorrect lightweight locking stack In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 19:00:53 GMT, Roman Kennke wrote: > See JBS issue for details. > > I basically: > - took the test-modification and turned it into its own test-case > - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default > - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. > > Testing: > - [x] EATests.java > - [x] tier1 > - [ ] tier2 Marked as reviewed by dlong (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16568#pullrequestreview-1721539402 From jjoo at openjdk.org Thu Nov 9 05:27:40 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 9 Nov 2023 05:27:40 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Add missing cpuTimeCounters files ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/22ccb909..41771db6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=39 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=38-39 Stats: 220 lines in 2 files changed: 220 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From dholmes at openjdk.org Thu Nov 9 07:06:03 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 9 Nov 2023 07:06:03 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: <4e6l0dEPEUZ3OXDZpQmbnxwgRupmQyYt66QTWW5PGRA=.8d004fe7-0821-4487-997e-57ec8dbea648@github.com> Message-ID: On Wed, 8 Nov 2023 11:40:35 GMT, Johan Sj?len wrote: > > > As this adds a JCmd, doesn't this need both a CSR and a manual entry? > > > > > > ``` > > * CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. > > > > * Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? > > ``` > > @dholmes-ora, would you mind helping us out here with regards to the process? Would a new JCmd require a CSR or would it be acceptable to merge without one? Thank you. @jdksjolen No CSR needed: from another related PR - "We do not use CSR requests with jcmd changes as they are deemed diagnostic commands - ref JDK-8203682" But yes the `jcmd` manpage should be updated ref: https://docs.oracle.com/en/java/javase/19/docs/specs/man/jcmd.html though I'm worried we may not have kept it up to date! That requires an Oracle engineer to apply the changes to the jcmd.md markdown sources in our repo, and then regenerate the `jcmd.1` manpage file. The doc update can be split into a separate doc sub-task so that the main PR is not held up. (And we probably need an audit to see if any other updates are missing - which is painful.) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1803267397 From stuefe at openjdk.org Thu Nov 9 07:14:03 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 9 Nov 2023 07:14:03 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: <4e6l0dEPEUZ3OXDZpQmbnxwgRupmQyYt66QTWW5PGRA=.8d004fe7-0821-4487-997e-57ec8dbea648@github.com> Message-ID: On Thu, 9 Nov 2023 07:03:34 GMT, David Holmes wrote: > > > > As this adds a JCmd, doesn't this need both a CSR and a manual entry? > > > > > > > > > ``` > > > * CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. > > > > > > * Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? > > > ``` > > > > > > @dholmes-ora, would you mind helping us out here with regards to the process? Would a new JCmd require a CSR or would it be acceptable to merge without one? Thank you. > > @jdksjolen No CSR needed: from another related PR - "We do not use CSR requests with jcmd changes as they are deemed diagnostic commands - ref JDK-8203682" > > But yes the `jcmd` manpage should be updated ref: https://docs.oracle.com/en/java/javase/19/docs/specs/man/jcmd.html though I'm worried we may not have kept it up to date! That requires an Oracle engineer to apply the changes to the jcmd.md markdown sources in our repo, and then regenerate the `jcmd.1` manpage file. The doc update can be split into a separate doc sub-task so that the main PR is not held up. (And we probably need an audit to see if any other updates are missing - which is painful.) Would be nice if that process were open like the rest. It would take load from Oracle and most developers would be fine with maintaining the man page themselves. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1803274807 From dholmes at openjdk.org Thu Nov 9 07:35:08 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 9 Nov 2023 07:35:08 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: Message-ID: On Thu, 2 Nov 2023 17:42:29 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > Fix another windows error If the command is only usable on Linux, and only registered on Linux, do we really need the empty definitions in `memMapPrinter_.cpp`? At present it seems we only need it because all the implementation code is not isolated to `ifdef LINUX`. src/hotspot/share/runtime/thread.cpp line 474: > 472: osthread()->print_on(st); > 473: } > 474: st->print("stack=[" PTR_FORMAT ", " PTR_FORMAT ")", p2i(stack_end()), p2i(stack_base())); We already report the thread stack elsewhere, so is this going to duplicate the information? src/hotspot/share/runtime/vmOperation.hpp line 121: > 119: template(JvmtiPostObjectFree) \ > 120: template(RendezvousGCThreads) \ > 121: template(SystemMap) I can't spot the VMop for this. ?? src/hotspot/share/services/diagnosticCommand.cpp line 1159: > 1157: } > 1158: > 1159: SystemMapDCmd::SystemMapDCmd(outputStream* output, bool heap) : Shouldn't this all be inside `ifdef LINUX`? You don't need `os::realpath` if you do that. src/hotspot/share/services/diagnosticCommand.cpp line 1193: > 1191: output()->print_cr("Memory map dumped to \"%s\".", name); > 1192: } else { > 1193: output()->print_cr("Failed to open \"%s\" for writing.", name); Don't you want to report errno here? src/hotspot/share/services/diagnosticCommand.hpp line 988: > 986: static const char* name() { return "System.map"; } > 987: static const char* description() { > 988: return "Prints an annotated process memory map of the VM process."; Should this specify "Linux Only" ? src/hotspot/share/services/diagnosticCommand.hpp line 1007: > 1005: static const char* name() { return "System.dump_map"; } > 1006: static const char* description() { > 1007: return "Dumps an annotated process memory map to an output file."; Again "Linux Only"? ------------- Changes requested by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16301#pullrequestreview-1721842005 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1387576827 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1387580917 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1387584901 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1387585667 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1387572049 PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1387577576 From dholmes at openjdk.org Thu Nov 9 07:35:10 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 9 Nov 2023 07:35:10 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: Message-ID: <_FI6K3-iWZ3OXlYuNyUN2bSAdzXcZrVtL0iX5Oaxj_M=.d5b36bc8-a683-4c1b-ad5a-40f3ba4dd20b@github.com> On Thu, 9 Nov 2023 07:26:46 GMT, David Holmes wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix another windows error > > src/hotspot/share/services/diagnosticCommand.cpp line 1159: > >> 1157: } >> 1158: >> 1159: SystemMapDCmd::SystemMapDCmd(outputStream* output, bool heap) : > > Shouldn't this all be inside `ifdef LINUX`? You don't need `os::realpath` if you do that. Well to call `os::Posix::realpath` it would need Linux specific includes ... but as it is you have unused code in the Windows version. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1387589586 From dholmes at openjdk.org Thu Nov 9 08:00:57 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 9 Nov 2023 08:00:57 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 16:12:48 GMT, Lutz Schmidt wrote: > LGTM. Thanks for verifying we are not hiding a leak with the increase. Has that actually been verified? I couldn't quite understand what was being said in the description. This new value would need to be tested on all platforms, on an unpatched VM to see if the leak is still present. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16553#issuecomment-1803325038 From mbaesken at openjdk.org Thu Nov 9 08:10:58 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 9 Nov 2023 08:10:58 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX In-Reply-To: References: Message-ID: On Thu, 9 Nov 2023 07:58:29 GMT, David Holmes wrote: > > LGTM. Thanks for verifying we are not hiding a leak with the increase. > > Has that actually been verified? I couldn't quite understand what was being said in the description. This new value would need to be tested on all platforms, on an unpatched VM to see if the leak is still present. The test has been executed with 1700 instead of 700 iterations a number of times to see if a leak occurs but it did not show up. If you think that the other platforms (!= AIX) should continue with the previous memory settings that's fine. I already asked this above . We could set up two jtreg test blocks one for AIX one for the other platforms. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16553#issuecomment-1803338202 From tschatzl at openjdk.org Thu Nov 9 10:44:29 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 9 Nov 2023 10:44:29 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v16] In-Reply-To: References: Message-ID: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Modify evacuation failure log message as suggested by sjohanss: Use "Evacuation Failure" with a cause description (either "Allocation" or "Pinned") ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/83eff9fe..6395696a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=15 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=14-15 Stats: 16 lines in 3 files changed: 11 ins; 1 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From tschatzl at openjdk.org Thu Nov 9 10:44:30 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Thu, 9 Nov 2023 10:44:30 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v9] In-Reply-To: <6_MNWrkq_LnvB4EqrBFNo6e6kVHLgffcRbkxhRPzTpg=.1a05d8af-b181-4e80-9922-6aea9a517564@github.com> References: <6_MNWrkq_LnvB4EqrBFNo6e6kVHLgffcRbkxhRPzTpg=.1a05d8af-b181-4e80-9922-6aea9a517564@github.com> Message-ID: On Wed, 8 Nov 2023 14:46:16 GMT, Stefan Johansson wrote: >> The example looks good to me. > > Have the final output looking something like this was agreed on during internal discussion: > GC(6) Pause Young (Normal) (Evacuation Failure: Pinned) 1M->1M(22M) 36.16ms > GC(6) Pause Young (Normal) (Evacuation Failure: Allocation) 1M->1M(22M) 36.16ms > GC(6) Pause Young (Normal) (Evacuation Failure: Allocation / Pinned) 1M->1M(22M) 36.16ms Done. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1387817157 From sjohanss at openjdk.org Thu Nov 9 10:48:08 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 9 Nov 2023 10:48:08 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: <4EVNbC2fI1AQGHMkRMfI6SDJrw98KEX0xuRJR1s361o=.d1ff2f1a-ffc9-4454-9755-e6e9e14d9110@github.com> On Thu, 9 Nov 2023 05:27:40 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing cpuTimeCounters files A few more comments. src/hotspot/share/gc/g1/g1ConcurrentRefineThread.cpp line 82: > 80: _vtime_accum = 0.0; > 81: } > 82: maybe_update_threads_cpu_time(); I think the lines above here: if (os::supports_vtime()) { _vtime_accum = (os::elapsedVTime() - _vtime_start); } else { _vtime_accum = 0.0; } Should be extracted out into the new method and instead of calling it `maybe_update_threads_cpu_time()` just call `track_usage()` or `track_cpu_time()`. The the implementation in the primary thread can then call this and do the extra tracking. src/hotspot/share/gc/g1/g1ConcurrentRefineThread.cpp line 189: > 187: void G1PrimaryConcurrentRefineThread::maybe_update_threads_cpu_time() { > 188: if (UsePerfData && os::is_thread_cpu_time_supported()) { > 189: cr()->update_concurrent_refine_threads_cpu_time(); I think we should pull the tracking closure in here and that way leave the concurrent refine class untouched. Suggestion: // The primary thread is responsible for updating the CPU time for all workers. CPUTimeCounters* counters = G1CollectedHeap::heap()->cpu_time_counters(); ThreadTotalCPUTimeClosure tttc(counters, CPUTimeGroups::gc_conc_refine); cr()->threads_do(&tttc); This is more or less a copy from `G1ConcurrentRefineThreadControl::update_threads_cpu_time()` which if we go with this solution can be removed. The above needs some new includes though. I change the comment a because I could not fully understand it, the primary thread is the one always checking and starting more threads so it is not stopped first. Also not sure when a terminated thread could be read. Even the stopped threads are still present so should be fine. If I'm missing something feel free to add back the comment. src/hotspot/share/gc/g1/g1ServiceThread.cpp line 138: > 136: ThreadTotalCPUTimeClosure tttc(counters, CPUTimeGroups::gc_service); > 137: tttc.do_thread(task->_service_thread); > 138: } Please extract this to a function, similar to the other cases something like `track_cpu_time()`. ------------- Changes requested by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1722194318 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387787912 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387803508 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387805665 From sjohanss at openjdk.org Thu Nov 9 11:03:11 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 9 Nov 2023 11:03:11 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: <9yVx0YyUxW40p2wV4ee2VB5m6WLOL9CxVcEqUE8W81Q=.a68ded01-b31f-4b09-91dc-04e893ff15eb@github.com> On Thu, 9 Nov 2023 05:27:40 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing cpuTimeCounters files src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 2093: > 2091: tttc.do_thread(cm_thread()); > 2092: threads_do(&tttc); > 2093: } Any particular reason for having this in `G1ConcurrentMark` instead of keeping it where it is called in `G1ConcurrentMarkThread`. The implementation would be more or less the same apart from the two last lines: tttc.do_thread(this); cm()->threads_do(&tttc); ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1387839879 From sjohanss at openjdk.org Thu Nov 9 11:13:06 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 9 Nov 2023 11:13:06 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: References: Message-ID: On Thu, 9 Nov 2023 05:27:40 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Add missing cpuTimeCounters files One more thing I noticed is that the parallel worker time from the Remark pause is not updated after the remark pause. So I guess we should add a call to update that. It will still be account the next time we do a normal GC, but if we want the counter to be up to date as much as possible we should update it after the remark pause. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1803629858 From rrich at openjdk.org Thu Nov 9 14:58:03 2023 From: rrich at openjdk.org (Richard Reingruber) Date: Thu, 9 Nov 2023 14:58:03 GMT Subject: RFR: 8318895: Deoptimization results in incorrect lightweight locking stack In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 19:00:53 GMT, Roman Kennke wrote: > See JBS issue for details. > > I basically: > - took the test-modification and turned it into its own test-case > - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default > - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. > > Testing: > - [x] EATests.java > - [x] tier1 > - [ ] tier2 Hi Roman, thanks for opening the pr. I've implemented [another test case](https://github.com/reinrich/jdk/commit/b72b3b3d7d1b5927811ae49e3ddea01d298dcb85) that demonstrates why relocking should be done before an object reference with eliminated locking is passed to a JVMTI agent. Would it be ok to include it in your pr? ([This is a version](https://github.com/reinrich/jdk/commit/f7a90c13e27e9fe38c892f069bd8d58484f59445) where relocking is delayed until the compiled frame is deoptimized. The new test fails with -XX:+UseNewCode). I will put the change through our CI testing. Cheers, Richard. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16568#issuecomment-1803985944 From rkennke at openjdk.org Thu Nov 9 15:54:13 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Thu, 9 Nov 2023 15:54:13 GMT Subject: RFR: 8318895: Deoptimization results in incorrect lightweight locking stack [v2] In-Reply-To: References: Message-ID: <90buwH_81LCEUj7bv7Ug4fDC8IbyMDCFcmNfmyd1Hxk=.8747fb2c-265b-41cd-8d74-a576e58adf85@github.com> > See JBS issue for details. > > I basically: > - took the test-modification and turned it into its own test-case > - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default > - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. > > Testing: > - [x] EATests.java > - [x] tier1 > - [ ] tier2 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Add @reinrich's test-case ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16568/files - new: https://git.openjdk.org/jdk/pull/16568/files/d9370df6..966d0a3e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16568&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16568&range=00-01 Stats: 58 lines in 1 file changed: 58 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16568.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16568/head:pull/16568 PR: https://git.openjdk.org/jdk/pull/16568 From mbaesken at openjdk.org Thu Nov 9 16:16:57 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 9 Nov 2023 16:16:57 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: <83tY8a_r4Yh4fni2jQoi0Gz3PxBfE-EGs7U5AImK6-Q=.2cc49085-75ad-4962-969f-9cceb0764ec6@github.com> References: <83tY8a_r4Yh4fni2jQoi0Gz3PxBfE-EGs7U5AImK6-Q=.2cc49085-75ad-4962-969f-9cceb0764ec6@github.com> Message-ID: <3z3fv0FbQvV_2x3tOZXXN80ZHTPjMWywDdZ3lVtwHaA=.062f4e04-514e-4603-9132-0a710b525e3e@github.com> On Wed, 8 Nov 2023 21:55:44 GMT, Alex Menkov wrote: > does getaddrinfo return a single address? (parseAllowedAddr assumes so, but maybe that's wrong in the case) >From what I see on AIX, only one entry is returned in the results struct of the getaddrinfo call. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16561#issuecomment-1804127644 From mbaesken at openjdk.org Thu Nov 9 16:17:00 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Thu, 9 Nov 2023 16:17:00 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: References: Message-ID: <9ZZyvNfhowm9r7ffFg7wJmw8mJPLXsKwCzg2qq1gsgQ=.333589b3-1cb5-4d3b-bef5-78464540173e@github.com> On Wed, 8 Nov 2023 17:57:42 GMT, Chris Plummer wrote: > The downside of this change is that it will cause the option parsing code to not properly produce an error if an IPv4 mask prefix between 32 and 128 is given. Yes this is true; maybe we should not unconditionally increase the maxValue on AIX . ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16561#discussion_r1388236604 From dholmes at openjdk.org Fri Nov 10 05:47:55 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 10 Nov 2023 05:47:55 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 08:29:52 GMT, Matthias Baesken wrote: > On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: > > java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) > at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) > at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) > at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) > at java.base/java.lang.Thread.run(Thread.java:1570) > Caused by: java.lang.OutOfMemoryError: Metaspace > at java.base/java.time.Duration.(Duration.java:149) > at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) > at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) > at java.base/java.time.LocalDate.of(LocalDate.java:272) > at java.base/java.time.LocalDate.(LocalDate.java:147) > at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) > at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.pathsA... I just want it confirmed that the proposed test changes will exhibit a leak on all platforms on an unpatched VM. If that can't be ascertained then it may be better to only change for AIX. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16553#issuecomment-1805134932 From stuefe at openjdk.org Fri Nov 10 07:10:35 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 10 Nov 2023 07:10:35 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v10] In-Reply-To: References: Message-ID: > Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. > > Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. > > That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. > > Example output (from a spring petstore run): > > [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) > > This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: Feedback David; move common header printing to shared code ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16301/files - new: https://git.openjdk.org/jdk/pull/16301/files/7dd5a911..050656a0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=08-09 Stats: 167 lines in 11 files changed: 32 ins; 131 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16301.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16301/head:pull/16301 PR: https://git.openjdk.org/jdk/pull/16301 From stuefe at openjdk.org Fri Nov 10 07:16:37 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 10 Nov 2023 07:16:37 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v11] In-Reply-To: References: Message-ID: > Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. > > Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. > > That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. > > Example output (from a spring petstore run): > > [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) > > This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: remove remnants of os::realpath ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16301/files - new: https://git.openjdk.org/jdk/pull/16301/files/050656a0..e181298b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=09-10 Stats: 10 lines in 2 files changed: 0 ins; 10 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16301.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16301/head:pull/16301 PR: https://git.openjdk.org/jdk/pull/16301 From stuefe at openjdk.org Fri Nov 10 07:25:08 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 10 Nov 2023 07:25:08 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: Message-ID: <3cFwXdRv6HTGJzM_Nw8TIDx6V6Xf__9g1ytQMG-LmA8=.dd2b8d76-8c37-47de-ad63-a2558cb932cf@github.com> On Thu, 9 Nov 2023 07:27:37 GMT, David Holmes wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix another windows error > > src/hotspot/share/services/diagnosticCommand.cpp line 1193: > >> 1191: output()->print_cr("Memory map dumped to \"%s\".", name); >> 1192: } else { >> 1193: output()->print_cr("Failed to open \"%s\" for writing.", name); > > Don't you want to report errno here? I'm always leery about relying on errno from wrapper functions or objects like this, unless they have an explicit channel to report the relevant errno. I looked at fileStream, I guess its okay here. Okay, I'll add it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16301#discussion_r1389011079 From rrich at openjdk.org Fri Nov 10 07:25:58 2023 From: rrich at openjdk.org (Richard Reingruber) Date: Fri, 10 Nov 2023 07:25:58 GMT Subject: RFR: 8318895: Deoptimization results in incorrect lightweight locking stack [v2] In-Reply-To: <90buwH_81LCEUj7bv7Ug4fDC8IbyMDCFcmNfmyd1Hxk=.8747fb2c-265b-41cd-8d74-a576e58adf85@github.com> References: <90buwH_81LCEUj7bv7Ug4fDC8IbyMDCFcmNfmyd1Hxk=.8747fb2c-265b-41cd-8d74-a576e58adf85@github.com> Message-ID: On Thu, 9 Nov 2023 15:54:13 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> I basically: >> - took the test-modification and turned it into its own test-case >> - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default >> - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. >> >> Testing: >> - [x] EATests.java >> - [x] tier1 >> - [ ] tier2 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Add @reinrich's test-case Fix and new test case look good to me. Local testing was clean. Thanks, Richard. test/jdk/com/sun/jdi/EATests.java line 1755: > 1753: ///////////////////////////////////////////////////////////////////////////// > 1754: > 1755: // The debugger reads and publishes an object with eliminated locking to a static variable. Suggestion: // The debugger reads and publishes an object with eliminated locking to an instance field. ------------- Marked as reviewed by rrich (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16568#pullrequestreview-1724167589 PR Review Comment: https://git.openjdk.org/jdk/pull/16568#discussion_r1389009395 From stuefe at openjdk.org Fri Nov 10 07:38:24 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 10 Nov 2023 07:38:24 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v12] In-Reply-To: References: Message-ID: <8rWMhr_ziMgLN99McN6EDTMxMDtOYU7KMGjaTRBAz98=.75748a9c-6ce5-444e-995e-d75f17e2862d@github.com> > Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. > > Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. > > That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. > > Example output (from a spring petstore run): > > [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) > > This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. Thomas Stuefe has updated the pull request incrementally with three additional commits since the last revision: - add test to test dumping to specific location - change filename arg from -f to -F to prevent conflicts with jcmd -f - print errno if dump file cannot be opened ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16301/files - new: https://git.openjdk.org/jdk/pull/16301/files/e181298b..24a9916a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16301&range=10-11 Stats: 19 lines in 2 files changed: 14 ins; 0 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16301.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16301/head:pull/16301 PR: https://git.openjdk.org/jdk/pull/16301 From stuefe at openjdk.org Fri Nov 10 07:41:05 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 10 Nov 2023 07:41:05 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: <4e6l0dEPEUZ3OXDZpQmbnxwgRupmQyYt66QTWW5PGRA=.8d004fe7-0821-4487-997e-57ec8dbea648@github.com> Message-ID: On Thu, 9 Nov 2023 07:03:34 GMT, David Holmes wrote: >>> > As this adds a JCmd, doesn't this need both a CSR and a manual entry? >>> >>> * CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. >>> >>> * Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? >> >> @dholmes-ora, would you mind helping us out here with regards to the process? Would a new JCmd require a CSR or would it be acceptable to merge without one? Thank you. > >> > > As this adds a JCmd, doesn't this need both a CSR and a manual entry? >> > >> > >> > ``` >> > * CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. >> > >> > * Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? >> > ``` >> >> @dholmes-ora, would you mind helping us out here with regards to the process? Would a new JCmd require a CSR or would it be acceptable to merge without one? Thank you. > > @jdksjolen No CSR needed: from another related PR - "We do not use CSR requests with jcmd changes as they are deemed diagnostic commands - ref JDK-8203682" > > But yes the `jcmd` manpage should be updated ref: > https://docs.oracle.com/en/java/javase/19/docs/specs/man/jcmd.html > though I'm worried we may not have kept it up to date! That requires an Oracle engineer to apply the changes to the jcmd.md markdown sources in our repo, and then regenerate the `jcmd.1` manpage file. The doc update can be split into a separate doc sub-task so that the main PR is not held up. (And we probably need an audit to see if any other updates are missing - which is painful.) @dholmes-ora thank you for your review. I hope I have addressed all concerns. I made this all Linux only, removed os::realpath, print errno. I am currently working on a follow-up RFE that provides this command for Windows, and it works, but I will do it in a separate RFE (rolling back the ifdef LINUX should be easy) and possibly not for JDK 22. https://github.com/openjdk/jdk/pull/16593 ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1805238529 From jsjolen at openjdk.org Fri Nov 10 09:48:11 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Fri, 10 Nov 2023 09:48:11 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v12] In-Reply-To: <8rWMhr_ziMgLN99McN6EDTMxMDtOYU7KMGjaTRBAz98=.75748a9c-6ce5-444e-995e-d75f17e2862d@github.com> References: <8rWMhr_ziMgLN99McN6EDTMxMDtOYU7KMGjaTRBAz98=.75748a9c-6ce5-444e-995e-d75f17e2862d@github.com> Message-ID: <_83ucpj3Hfc7p6cQbxqxd9Z1WP6QtNrIa6j22RzBVD4=.7c757fa9-a741-40f0-8d00-c7695167bb0d@github.com> On Fri, 10 Nov 2023 07:38:24 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with three additional commits since the last revision: > > - add test to test dumping to specific location > - change filename arg from -f to -F to prevent conflicts with jcmd -f > - print errno if dump file cannot be opened Hi Thomas, Thanks for your hard work on this! LGTM. ------------- Marked as reviewed by jsjolen (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16301#pullrequestreview-1724406291 From rkennke at openjdk.org Fri Nov 10 10:41:16 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Fri, 10 Nov 2023 10:41:16 GMT Subject: RFR: 8318895: Deoptimization results in incorrect lightweight locking stack [v3] In-Reply-To: References: Message-ID: > See JBS issue for details. > > I basically: > - took the test-modification and turned it into its own test-case > - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default > - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. > > Testing: > - [x] EATests.java > - [x] tier1 > - [ ] tier2 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Update test/jdk/com/sun/jdi/EATests.java Co-authored-by: Richard Reingruber ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16568/files - new: https://git.openjdk.org/jdk/pull/16568/files/966d0a3e..b27def47 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16568&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16568&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16568.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16568/head:pull/16568 PR: https://git.openjdk.org/jdk/pull/16568 From mbaesken at openjdk.org Fri Nov 10 10:42:58 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 10 Nov 2023 10:42:58 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 05:44:55 GMT, David Holmes wrote: > I just want it confirmed that the proposed test changes will exhibit a leak on all platforms on an unpatched VM. If that can't be ascertained then it may be better to only change for AIX. On my SUSE Linux x86_64 test box, with the leak (from 8308762: Metaspace leak with Instrumentation.retransform) back in the source we have this behavior -XX:MetaspaceSize=22m -XX:MaxMetaspaceSize=22m FAILS -XX:MetaspaceSize=23m -XX:MaxMetaspaceSize=23m PASSES So I think we better go for separate settings for AIX and the rest of platforms to be on the safe side, because 23m is not really a good choice for Linux x86_64, it should still fail with the leak back in. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16553#issuecomment-1805483102 From stuefe at openjdk.org Fri Nov 10 10:46:10 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 10 Nov 2023 10:46:10 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v12] In-Reply-To: <_83ucpj3Hfc7p6cQbxqxd9Z1WP6QtNrIa6j22RzBVD4=.7c757fa9-a741-40f0-8d00-c7695167bb0d@github.com> References: <8rWMhr_ziMgLN99McN6EDTMxMDtOYU7KMGjaTRBAz98=.75748a9c-6ce5-444e-995e-d75f17e2862d@github.com> <_83ucpj3Hfc7p6cQbxqxd9Z1WP6QtNrIa6j22RzBVD4=.7c757fa9-a741-40f0-8d00-c7695167bb0d@github.com> Message-ID: On Fri, 10 Nov 2023 09:45:11 GMT, Johan Sj?len wrote: > Hi Thomas, > > Thanks for your hard work on this! LGTM. Thank you for your Review, Johan! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1805489022 From sjohanss at openjdk.org Fri Nov 10 11:02:06 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Fri, 10 Nov 2023 11:02:06 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v16] In-Reply-To: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> References: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> Message-ID: On Thu, 9 Nov 2023 10:44:29 GMT, Thomas Schatzl wrote: >> The JEP covers the idea very well, so I'm only covering some implementation details here: >> >> * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. >> >> * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: >> >> * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. >> >> * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). >> >> * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. >> >> * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) >> >> The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. >> >> I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. >> >> * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in a... > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Modify evacuation failure log message as suggested by sjohanss: Use "Evacuation Failure" with a cause description (either "Allocation" or "Pinned") Looks good. Just a few small things. src/hotspot/share/gc/g1/g1ParScanThreadState.cpp line 466: > 464: if (region_attr.is_pinned() && klass->is_typeArray_klass()) { > 465: return handle_evacuation_failure_par(old, old_mark, word_sz, true /* cause_pinned */); > 466: } I think it would make sense to add a comment here that this is an optimization, that we know that only type array can be pinned so we only care if the region is pinned or not for type arrays. src/hotspot/share/gc/g1/g1YoungGCPostEvacuateTasks.cpp line 925: > 923: _evac_failure_regions->num_regions_alloc_failed(), > 924: G1GCPhaseTimes::RestoreEvacFailureRegionsAllocFailedNum); > 925: The counts used here are the totals, but they are added to thread work item, so total count will be wrong. Discussed this a bit with Thomas and we should probably count those things in this task instead and could that way make `G1EvacFailureRegions` store a bit less information. src/hotspot/share/gc/shared/collectedHeap.hpp line 169: > 167: static inline size_t filler_array_hdr_size(); > 168: public: > 169: static size_t filler_array_min_size(); Is this needed or some leftover from earlier iterations? ------------- Marked as reviewed by sjohanss (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16342#pullrequestreview-1722530941 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1389227404 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1389211990 PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1387998596 From tschatzl at openjdk.org Fri Nov 10 11:25:06 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 10 Nov 2023 11:25:06 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v16] In-Reply-To: References: <1kvCiT9zD5ZqoLH_HFRtPsh8M78WaFwE6R5ODemjUMs=.2e5b019c-4a3b-4184-8171-49ff5a84c841@github.com> Message-ID: On Thu, 9 Nov 2023 13:26:06 GMT, Stefan Johansson wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> Modify evacuation failure log message as suggested by sjohanss: Use "Evacuation Failure" with a cause description (either "Allocation" or "Pinned") > > src/hotspot/share/gc/shared/collectedHeap.hpp line 169: > >> 167: static inline size_t filler_array_hdr_size(); >> 168: public: >> 169: static size_t filler_array_min_size(); > > Is this needed or some leftover from earlier iterations? Removed. I started doing an implementation that properly formats smaller filler arrays into the dead areas taking potential `typeArray`s into account, but ultimately opted for just allowing oversized filler arrays. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16342#discussion_r1389274174 From tschatzl at openjdk.org Fri Nov 10 12:14:19 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 10 Nov 2023 12:14:19 GMT Subject: RFR: 8318706: Implement JEP 423: Region Pinning for G1 [v17] In-Reply-To: References: Message-ID: > The JEP covers the idea very well, so I'm only covering some implementation details here: > > * regions get a "pin count" (reference count). As long as it is non-zero, we conservatively never reclaim that region even if there is no reference in there. JNI code might have references to it. > > * the JNI spec only requires us to provide pinning support for typeArrays, nothing else. This implementation uses this in various ways: > > * when evacuating from a pinned region, we evacuate everything live but the typeArrays to get more empty regions to clean up later. > > * when formatting dead space within pinned regions we use filler objects. Pinned regions may be referenced by JNI code only, so we can't overwrite contents of any dead typeArray either. These dead but referenced typeArrays luckily have the same header size of our filler objects, so we can use their headers for our fillers. The problem is that previously there has been that restriction that filler objects are half a region size at most, so we can end up with the need for placing a filler object header inside a typeArray. The code could be clever and handle this situation by splitting the to be filled area so that this can't happen, but the solution taken here is allowing filler arrays to cover a whole region. They are not referenced by Java code anyway, so there is no harm in doing so (i.e. gc code never touches them anyway). > > * G1 currently only ever actually evacuates young pinned regions. Old pinned regions of any kind are never put into the collection set and automatically skipped. However assuming that the pinning is of short length, we put them into the candidates when we can. > > * there is the problem that if an applications pins a region for a long time g1 will skip evacuating that region over and over. that may lead to issues with the current policy in marking regions (only exit mixed phase when there are no marking candidates) and just waste of processing time (when the candidate stays in the retained candidates) > > The cop-out chosen here is to "age out" the regions from the candidates and wait until the next marking happens. > > I.e. pinned marking candidates are immediately moved to retained candidates, and if in total the region has been pinned for `G1NumCollectionsKeepUnreclaimable` collections it is dropped from the candidates. Its current value is fairly random. > > * G1 pauses got a new tag if there were pinned regions in the collection set. I.e. in addition to something like: > > `GC(6) P... Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: stefanj review - fix counting of pinned/allocation failed regions in log - some cleanup of evacuation failure code, removing unnecessary members - comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16342/files - new: https://git.openjdk.org/jdk/pull/16342/files/6395696a..d6df3b06 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=16 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16342&range=15-16 Stats: 54 lines in 6 files changed: 16 ins; 28 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/16342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16342/head:pull/16342 PR: https://git.openjdk.org/jdk/pull/16342 From mbaesken at openjdk.org Fri Nov 10 12:23:13 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Fri, 10 Nov 2023 12:23:13 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX [v2] In-Reply-To: References: Message-ID: > On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: > > java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) > at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) > at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) > at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) > at java.base/java.lang.Thread.run(Thread.java:1570) > Caused by: java.lang.OutOfMemoryError: Metaspace > at java.base/java.time.Duration.(Duration.java:149) > at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) > at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) > at java.base/java.time.LocalDate.of(LocalDate.java:272) > at java.base/java.time.LocalDate.(LocalDate.java:147) > at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) > at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.pathsA... Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: Use separate test memory settings for AIX and non-AIX ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16553/files - new: https://git.openjdk.org/jdk/pull/16553/files/5d61cda6..56822dce Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16553&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16553&range=00-01 Stats: 16 lines in 1 file changed: 16 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16553.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16553/head:pull/16553 PR: https://git.openjdk.org/jdk/pull/16553 From rkennke at openjdk.org Fri Nov 10 15:31:12 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Fri, 10 Nov 2023 15:31:12 GMT Subject: RFR: 8318895: Deoptimization results in incorrect lightweight locking stack [v3] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 10:41:16 GMT, Roman Kennke wrote: >> See JBS issue for details. >> >> I basically: >> - took the test-modification and turned it into its own test-case >> - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default >> - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. >> >> Testing: >> - [x] EATests.java >> - [x] tier1 >> - [x] tier2 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Update test/jdk/com/sun/jdi/EATests.java > > Co-authored-by: Richard Reingruber Thanks, all! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16568#issuecomment-1805939306 From rkennke at openjdk.org Fri Nov 10 15:31:13 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Fri, 10 Nov 2023 15:31:13 GMT Subject: Integrated: 8318895: Deoptimization results in incorrect lightweight locking stack In-Reply-To: References: Message-ID: <2t-1klBg1-GAN4ss9bqW8KpK98y5-8yYTjYgM61R8B8=.29af1d78-cbfa-44c4-8a3b-1ff237f6a0ec@github.com> On Wed, 8 Nov 2023 19:00:53 GMT, Roman Kennke wrote: > See JBS issue for details. > > I basically: > - took the test-modification and turned it into its own test-case > - added test runners for lightweight- and legacy-locking, so that we keep testing both, no matter what is the default > - added Axels fix (mentioned in the JBS issue) with the modification to only inflate when exec_mode == Unpack_none, as explained by Richard. > > Testing: > - [x] EATests.java > - [x] tier1 > - [x] tier2 This pull request has now been integrated. Changeset: ea1ffa34 Author: Roman Kennke URL: https://git.openjdk.org/jdk/commit/ea1ffa34192448317ce9a61a3588b0dee3a2ef44 Stats: 157 lines in 2 files changed: 154 ins; 0 del; 3 mod 8318895: Deoptimization results in incorrect lightweight locking stack Co-authored-by: Axel Boldt-Christmas Co-authored-by: Richard Reingruber Reviewed-by: dlong, rrich ------------- PR: https://git.openjdk.org/jdk/pull/16568 From jjoo at openjdk.org Sat Nov 11 00:23:28 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 11 Nov 2023 00:23:28 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v41] In-Reply-To: References: Message-ID: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Refactor ConcurrentRefine logic - Make CPUTimeCounters a singleton class ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/41771db6..533af850 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=40 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=39-40 Stats: 151 lines in 17 files changed: 62 ins; 62 del; 27 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Sat Nov 11 00:23:29 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 11 Nov 2023 00:23:29 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v40] In-Reply-To: <4EVNbC2fI1AQGHMkRMfI6SDJrw98KEX0xuRJR1s361o=.d1ff2f1a-ffc9-4454-9755-e6e9e14d9110@github.com> References: <4EVNbC2fI1AQGHMkRMfI6SDJrw98KEX0xuRJR1s361o=.d1ff2f1a-ffc9-4454-9755-e6e9e14d9110@github.com> Message-ID: <1NfEWhHC6ZHOtFiDpwcpZwuxDzbf-sfY7IenNn3nv9M=.d189f48e-8772-4ec2-bb17-bd912a28c5ab@github.com> On Thu, 9 Nov 2023 10:29:29 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Add missing cpuTimeCounters files > > src/hotspot/share/gc/g1/g1ConcurrentRefineThread.cpp line 189: > >> 187: void G1PrimaryConcurrentRefineThread::maybe_update_threads_cpu_time() { >> 188: if (UsePerfData && os::is_thread_cpu_time_supported()) { >> 189: cr()->update_concurrent_refine_threads_cpu_time(); > > I think we should pull the tracking closure in here and that way leave the concurrent refine class untouched. > > Suggestion: > > // The primary thread is responsible for updating the CPU time for all workers. > CPUTimeCounters* counters = G1CollectedHeap::heap()->cpu_time_counters(); > ThreadTotalCPUTimeClosure tttc(counters, CPUTimeGroups::gc_conc_refine); > cr()->threads_do(&tttc); > > > This is more or less a copy from `G1ConcurrentRefineThreadControl::update_threads_cpu_time()` which if we go with this solution can be removed. The above needs some new includes though. > > I change the comment a because I could not fully understand it, the primary thread is the one always checking and starting more threads so it is not stopped first. Also not sure when a terminated thread could be read. Even the stopped threads are still present so should be fine. If I'm missing something feel free to add back the comment. Thank you for the review! Do you think you could take a look at the newest update and see if that aligns with what you were thinking? I wanted to make sure I understood your comments correctly. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1390054625 From jjoo at openjdk.org Sat Nov 11 00:30:07 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Sat, 11 Nov 2023 00:30:07 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v41] In-Reply-To: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> References: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> Message-ID: On Sat, 11 Nov 2023 00:23:28 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Refactor ConcurrentRefine logic > - Make CPUTimeCounters a singleton class Will address the Remark pause update in my next commit! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1806593319 From dholmes at openjdk.org Mon Nov 13 02:49:05 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 13 Nov 2023 02:49:05 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v12] In-Reply-To: <8rWMhr_ziMgLN99McN6EDTMxMDtOYU7KMGjaTRBAz98=.75748a9c-6ce5-444e-995e-d75f17e2862d@github.com> References: <8rWMhr_ziMgLN99McN6EDTMxMDtOYU7KMGjaTRBAz98=.75748a9c-6ce5-444e-995e-d75f17e2862d@github.com> Message-ID: <48rN0Q1g1pIScO7tjWk5LVZp0GKGh7l0WefQ5sLSQqo=.9941353a-e86e-4f55-b0a8-c68a7af446f4@github.com> On Fri, 10 Nov 2023 07:38:24 GMT, Thomas Stuefe wrote: >> Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. >> >> Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. >> >> That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. >> >> Example output (from a spring petstore run): >> >> [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) >> >> This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. > > Thomas Stuefe has updated the pull request incrementally with three additional commits since the last revision: > > - add test to test dumping to specific location > - change filename arg from -f to -F to prevent conflicts with jcmd -f > - print errno if dump file cannot be opened Thanks for the updates @tstuefe - nothing further from me (with the Windows variant in the pipeline I will wait and see how things look in terms of shared versus platform-specific code. Please create a JBS issue to get this new command documented in the jcmd manpage, with details on what that documentation should look like. Thanks ------------- PR Review: https://git.openjdk.org/jdk/pull/16301#pullrequestreview-1726475921 From dholmes at openjdk.org Mon Nov 13 03:00:57 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 13 Nov 2023 03:00:57 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX [v2] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 12:23:13 GMT, Matthias Baesken wrote: >> On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: >> >> java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace >> at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) >> at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) >> at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) >> at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) >> at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) >> at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) >> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) >> at java.base/java.lang.reflect.Method.invoke(Method.java:580) >> at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) >> at java.base/java.lang.Thread.run(Thread.java:1570) >> Caused by: java.lang.OutOfMemoryError: Metaspace >> at java.base/java.time.Duration.(Duration.java:149) >> at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) >> at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) >> at java.base/java.time.LocalDate.of(LocalDate.java:272) >> at java.base/java.time.LocalDate.(LocalDate.java:147) >> at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) >> at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) >> at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) >> at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) > ... > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > Use separate test memory settings for AIX and non-AIX Looks good (though screams for the ability to use conditionals in the test definition :) ). Thanks for checking. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16553#pullrequestreview-1726482744 From mbaesken at openjdk.org Mon Nov 13 07:56:05 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 13 Nov 2023 07:56:05 GMT Subject: RFR: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX [v2] In-Reply-To: References: Message-ID: On Fri, 10 Nov 2023 12:23:13 GMT, Matthias Baesken wrote: >> On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: >> >> java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace >> at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) >> at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) >> at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) >> at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) >> at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) >> at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) >> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) >> at java.base/java.lang.reflect.Method.invoke(Method.java:580) >> at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) >> at java.base/java.lang.Thread.run(Thread.java:1570) >> Caused by: java.lang.OutOfMemoryError: Metaspace >> at java.base/java.time.Duration.(Duration.java:149) >> at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) >> at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) >> at java.base/java.time.LocalDate.of(LocalDate.java:272) >> at java.base/java.time.LocalDate.(LocalDate.java:147) >> at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) >> at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) >> at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) >> at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) >> at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) > ... > > Matthias Baesken has updated the pull request incrementally with one additional commit since the last revision: > > Use separate test memory settings for AIX and non-AIX Hi David, Kevin, Lutz , thanks for the reviews ! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16553#issuecomment-1807620044 From mbaesken at openjdk.org Mon Nov 13 07:56:06 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 13 Nov 2023 07:56:06 GMT Subject: Integrated: JDK-8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 08:29:52 GMT, Matthias Baesken wrote: > On AIX the test test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into this error: > > java.lang.RuntimeException: java.lang.OutOfMemoryError: Metaspace > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:168) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100) > at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94) > at jdk.test.lib.compiler.InMemoryJavaCompiler.compile(InMemoryJavaCompiler.java:188) > at RedefineClassHelper.redefineClass(RedefineClassHelper.java:50) > at RedefineLeakThrowable.main(RedefineLeakThrowable.java:64) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) > at java.base/java.lang.Thread.run(Thread.java:1570) > Caused by: java.lang.OutOfMemoryError: Metaspace > at java.base/java.time.Duration.(Duration.java:149) > at java.base/java.time.temporal.ChronoUnit.(ChronoUnit.java:83) > at java.base/java.time.temporal.ChronoField.(ChronoField.java:124) > at java.base/java.time.LocalDate.of(LocalDate.java:272) > at java.base/java.time.LocalDate.(LocalDate.java:147) > at java.base/java.time.LocalDateTime.(LocalDateTime.java:145) > at jdk.zipfs/jdk.nio.zipfs.ZipUtils.dosToJavaTime(ZipUtils.java:216) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.readCEN(ZipFileSystem.java:2816) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem$Entry.(ZipFileSystem.java:2778) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.getEntry(ZipFileSystem.java:1941) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:859) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.isMultiReleaseJar(ZipFileSystem.java:1444) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.initializeReleaseVersion(ZipFileSystem.java:1416) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.(ZipFileSystem.java:191) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) > at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager$ArchiveContainer.(JavacFileManager.java:566) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.getContainer(JavacFileManager.java:329) > at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.pathsA... This pull request has now been integrated. Changeset: e035637a Author: Matthias Baesken URL: https://git.openjdk.org/jdk/commit/e035637a4cab7a28ba46be7d4000d3b2815b5e58 Stats: 18 lines in 1 file changed: 16 ins; 0 del; 2 mod 8319375: test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineLeakThrowable.java runs into OutOfMemoryError: Metaspace on AIX Reviewed-by: kevinw, lucy, dholmes ------------- PR: https://git.openjdk.org/jdk/pull/16553 From stuefe at openjdk.org Mon Nov 13 08:30:19 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 13 Nov 2023 08:30:19 GMT Subject: RFR: JDK-8318636: Add jcmd to print annotated process memory map [v9] In-Reply-To: References: <4e6l0dEPEUZ3OXDZpQmbnxwgRupmQyYt66QTWW5PGRA=.8d004fe7-0821-4487-997e-57ec8dbea648@github.com> Message-ID: On Thu, 9 Nov 2023 07:03:34 GMT, David Holmes wrote: >>> > As this adds a JCmd, doesn't this need both a CSR and a manual entry? >>> >>> * CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. >>> >>> * Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? >> >> @dholmes-ora, would you mind helping us out here with regards to the process? Would a new JCmd require a CSR or would it be acceptable to merge without one? Thank you. > >> > > As this adds a JCmd, doesn't this need both a CSR and a manual entry? >> > >> > >> > ``` >> > * CSR: not sure; there are precedences for going with CSR and without CSR. Since we get awfully close to JDK22 freeze, I would prefer for a CSR not to be needed. Also would make backporting a lot easier. >> > >> > * Manpage: not sure either; IIRC last time I tried, I was told that Oracle maintains these internally, because there is internal documentation that needs updating too? >> > ``` >> >> @dholmes-ora, would you mind helping us out here with regards to the process? Would a new JCmd require a CSR or would it be acceptable to merge without one? Thank you. > > @jdksjolen No CSR needed: from another related PR - "We do not use CSR requests with jcmd changes as they are deemed diagnostic commands - ref JDK-8203682" > > But yes the `jcmd` manpage should be updated ref: > https://docs.oracle.com/en/java/javase/19/docs/specs/man/jcmd.html > though I'm worried we may not have kept it up to date! That requires an Oracle engineer to apply the changes to the jcmd.md markdown sources in our repo, and then regenerate the `jcmd.1` manpage file. The doc update can be split into a separate doc sub-task so that the main PR is not held up. (And we probably need an audit to see if any other updates are missing - which is painful.) Thanks @dholmes-ora ! > Thanks for the updates @tstuefe - nothing further from me (with the Windows variant in the pipeline I will wait and see how things look in terms of shared versus platform-specific code. > > Please create a JBS issue to get this new command documented in the jcmd manpage, with details on what that documentation should look like. There are a number of inconsistencies apart from this new dcmd; I think the jcmd man page needs to be updated. I collected all inconsistencies I could spot in this issue: https://bugs.openjdk.org/browse/JDK-8319948 and put it onto your name. > > Thanks Thank you, David. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16301#issuecomment-1807663916 From stuefe at openjdk.org Mon Nov 13 08:30:23 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 13 Nov 2023 08:30:23 GMT Subject: Integrated: JDK-8318636: Add jcmd to print annotated process memory map In-Reply-To: References: Message-ID: <7Q50ICMvb0Z_17z9mONQTmPpoEDIto01of1dBk2aK3w=.e833f04f-48fa-42c8-ae6d-b5c9cdf3b091@github.com> On Sun, 22 Oct 2023 10:08:49 GMT, Thomas Stuefe wrote: > Analysts and supporters often use /proc/xx/maps to make sense of the memory footprint of a process. > > Interpreting the memory map correctly can help when used as a complement to other tools (e.g. NMT). There even exist tools out there that attempt to annotate the process memory map with JVM information. > > That, however, can be more accurately done from within the JVM, at least for mappings originating from hotspot. After all, we can correlate the mapping information in NMT with the VMA map. > > Example output (from a spring petstore run): > > [example_system_map.txt](https://github.com/openjdk/jdk/files/13179054/example_system_map.txt) > > This patch adds the VM annotations for VMAs that are *mmaped*. I also have an experimental patch that works with malloc'ed memory, but it is not ready yet for consumption and I leave that part of for now. This pull request has now been integrated. Changeset: 6f863b2a Author: Thomas Stuefe URL: https://git.openjdk.org/jdk/commit/6f863b2a1baa67deb2a7b33fcd93d272aea01812 Stats: 793 lines in 9 files changed: 792 ins; 1 del; 0 mod 8318636: Add jcmd to print annotated process memory map Reviewed-by: jsjolen, gziemski ------------- PR: https://git.openjdk.org/jdk/pull/16301 From mbaesken at openjdk.org Mon Nov 13 11:45:57 2023 From: mbaesken at openjdk.org (Matthias Baesken) Date: Mon, 13 Nov 2023 11:45:57 GMT Subject: RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case In-Reply-To: References: Message-ID: On Wed, 8 Nov 2023 14:37:29 GMT, Matthias Baesken wrote: > In parseAllowedMask (file socketTransport.c) , prefixLen of mask is compared with a maxValue (32 for IPv4, 128 otherwise). This fails on AIX if it is larger than 32, because getaddrinfo seems to often (always ?) detect IPv4 family, even for IPv6 addresses, so we take the wrong maxValue. > Probably we have to adjust the allowed maxValue on AIX, or adjust the IPv6 check. > > Example: > images/jdk/bin/java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:0,allow=0:0:0:0:0:0:10:0/106 > Error in allow option: '106' > ERROR: transport error 103: invalid netmask in allow option > ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510) A colleague contacted IBM about the different behavior of getaddrinfo on AIX (compared to Linux/macOS); maybe we have to adjust the result of the getaddrinfo call on AIX. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16561#issuecomment-1808002445 From thartmann at openjdk.org Mon Nov 13 11:54:03 2023 From: thartmann at openjdk.org (Tobias Hartmann) Date: Mon, 13 Nov 2023 11:54:03 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v4] In-Reply-To: References: Message-ID: On Thu, 26 Oct 2023 16:17:14 GMT, Thomas Stuefe wrote: >> When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > Fix Windows build I think this is ready for integration given that both @dholmes-ora and @jdksjolen are okay with it. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16335#issuecomment-1808014143 From stuefe at openjdk.org Mon Nov 13 13:33:27 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 13 Nov 2023 13:33:27 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v4] In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 11:51:30 GMT, Tobias Hartmann wrote: > I think this is ready for integration given that both @dholmes-ora and @jdksjolen are okay with it. Well, they did not approve yet; @jdksjolen or @dholmes-ora, if you are happy with this, could you hit the big green button please? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16335#issuecomment-1808169293 From stuefe at openjdk.org Mon Nov 13 13:33:26 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Mon, 13 Nov 2023 13:33:26 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v5] In-Reply-To: References: Message-ID: > When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: - Merge branch 'master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 - Fix Windows build - remove tab - Make patch more palatable - Merge branch 'openjdk:master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 - JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 ------------- Changes: https://git.openjdk.org/jdk/pull/16335/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16335&range=04 Stats: 110 lines in 3 files changed: 79 ins; 21 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/16335.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16335/head:pull/16335 PR: https://git.openjdk.org/jdk/pull/16335 From jiangli at openjdk.org Mon Nov 13 22:01:10 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 13 Nov 2023 22:01:10 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread Message-ID: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. ------------- Commit messages: - 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread Changes: https://git.openjdk.org/jdk/pull/16642/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319935 Stats: 14 lines in 1 file changed: 9 ins; 4 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16642.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16642/head:pull/16642 PR: https://git.openjdk.org/jdk/pull/16642 From dcubed at openjdk.org Mon Nov 13 22:18:27 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Mon, 13 Nov 2023 22:18:27 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread In-Reply-To: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 13 Nov 2023 21:52:22 GMT, Jiangli Zhou wrote: > Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. @jianglizhou - I fixed a typo in the bug's synopsis line. Change this PR's title: s/is create/is created/ ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1809221878 From jiangli at openjdk.org Mon Nov 13 23:12:27 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 13 Nov 2023 23:12:27 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 13 Nov 2023 22:15:18 GMT, Daniel D. Daugherty wrote: > @jianglizhou - I fixed a typo in the bug's synopsis line. Change this PR's title: s/is create/is created/ Thanks, @dcubed-ojdk! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1809279446 From manc at openjdk.org Mon Nov 13 23:39:29 2023 From: manc at openjdk.org (Man Cao) Date: Mon, 13 Nov 2023 23:39:29 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread In-Reply-To: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 13 Nov 2023 21:52:22 GMT, Jiangli Zhou wrote: > Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. Changes requested by manc (Committer). src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 94: > 92: // The state->get_thread_oop() may be null if the state is created during > 93: // the allocation of the thread oop when a native thread is attaching. Make > 94: // sure we don't create a new state for the JavaThread. I think it is important to maintain `JvmtiThreadState::_thread_oop_h` correctly for the attached native thread. In the existing logic, with and without the proposed change, `JvmtiThreadState::_thread_oop_h` could stay null for an attached native thread, which seems wrong. It may be OK since `JvmtiThreadState::_thread_oop_h` is only used by support for virtual threads. It is unlikely that an attached native thread becomes a carrier for a virtual thread. However, it is probably still desirable to update `JvmtiThreadState::_thread_oop_h` to the correct java.lang.Thread oop. src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 95: > 93: // the allocation of the thread oop when a native thread is attaching. Make > 94: // sure we don't create a new state for the JavaThread. > 95: if (state == nullptr || (state->get_thread_oop() != nullptr && In my limited testing with and without virtual threads, when `state` is not null, `state->get_thread_oop()` is always either null or equal to `thread_oop`. So I suspect this whole if is equivalent to `if (state == nullptr)` in practice. I think it is better to split this if into two conditions. Here is my proposed change: https://github.com/openjdk/jdk/commit/00ace66c36243671a0fb1b673b3f9845460c6d22. It adds a `JvmtiThreadState::set_thread_oop()` method, and checks the above observation with `state->get_thread_oop()`, and addresses the problem of `JvmtiThreadState::_thread_oop_h` incorrectly staying null for attached native thread. I tested it with the 10000 virtual threads running Thread.sleep() example from https://openjdk.org/jeps/425, and `make test TEST=jdk_loom`. src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 98: > 96: state->get_thread_oop() != thread_oop)) { > 97: // Check if java_lang_Thread already has a link to the JvmtiThreadState. > 98: if (thread_oop != nullptr) { // thread_oop can be null during early VMStart. This comment is another case of `state->get_thread_oop()` being null. We should merge this comment with the new comment about attaching native thread. ------------- PR Review: https://git.openjdk.org/jdk/pull/16642#pullrequestreview-1728423451 PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1391795730 PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1391805673 PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1391813214 From jiangli at openjdk.org Tue Nov 14 02:13:33 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Tue, 14 Nov 2023 02:13:33 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 13 Nov 2023 23:21:56 GMT, Man Cao wrote: >> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. > > src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 95: > >> 93: // the allocation of the thread oop when a native thread is attaching. Make >> 94: // sure we don't create a new state for the JavaThread. >> 95: if (state == nullptr || (state->get_thread_oop() != nullptr && > > In my limited testing with and without virtual threads, when `state` is not null, `state->get_thread_oop()` is always either null or equal to `thread_oop`. So I suspect this whole if is equivalent to `if (state == nullptr)` in practice. > > I think it is better to split this if into two conditions. Here is my proposed change: https://github.com/openjdk/jdk/commit/00ace66c36243671a0fb1b673b3f9845460c6d22. It adds a `JvmtiThreadState::set_thread_oop()` method, and checks the above observation with `state->get_thread_oop()`, and addresses the problem of `JvmtiThreadState::_thread_oop_h` incorrectly staying null for attached native thread. > > I tested it with the 10000 virtual threads running Thread.sleep() example from https://openjdk.org/jeps/425, and `make test TEST=jdk_loom`. If my understand of [JavaThread::rebind_to_jvmti_thread_state_of](https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/javaThread.cpp#L1819) is correct, state->get_thread_oop() could be either `JavaThread::_threadObj` or `JavaThread::_jvmti_vthread`, ignoring the null case here. The `state` could be null if the virtual thread is unmounted and the thread is null, or if a JvmtiThreadState has not been created for the thread. Currently the `JvmtiThreadState::_thread_oop_h` is 'sealed' once a JvmtiThreadState is created. It can never be altered before a JvmtiThreadState is destroyed. Adding a `JvmtiThreadState:set_thread_oop()` changes that assumption. Let's wait for others who are more involved in this area to comment as well. I just took a look of your https://github.com/openjdk/jdk/commit/00ace66c36243671a0fb1b673b3f9845460c6d22 change. I think it does not address the specific issue that we were seeing in our tests. At the time when `JvmtiThreadState::state_for_while_locked` is called for a JavaThread associated with a native thread being attached, the thread oop is being allocated. There is no non-null thread oop to be set into `JvmtiThreadState::_thread_oop_h`. I'll address other comments later. Thanks, @caoman. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1391895645 From dholmes at openjdk.org Tue Nov 14 03:13:30 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 14 Nov 2023 03:13:30 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread In-Reply-To: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 13 Nov 2023 21:52:22 GMT, Jiangli Zhou wrote: > Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. Is this a case where the code should be checking for `is_attaching_via_jni()`? ------------- PR Review: https://git.openjdk.org/jdk/pull/16642#pullrequestreview-1728674210 From dholmes at openjdk.org Tue Nov 14 05:07:29 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 14 Nov 2023 05:07:29 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v4] In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 11:51:30 GMT, Tobias Hartmann wrote: >> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix Windows build > > I think this is ready for integration given that both @dholmes-ora and @jdksjolen are okay with it. @TobiHartmann both @jdksjolen and myself indicated we would defer to the compiler team to review/approve this. I only had some drive-by comments. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16335#issuecomment-1809554541 From rmarchenko at openjdk.org Tue Nov 14 06:56:49 2023 From: rmarchenko at openjdk.org (Roman Marchenko) Date: Tue, 14 Nov 2023 06:56:49 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks Message-ID: Zero'ing memory of extension event callbacks ------------- Commit messages: - JDK-8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks Changes: https://git.openjdk.org/jdk/pull/16647/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16647&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8319961 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16647.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16647/head:pull/16647 PR: https://git.openjdk.org/jdk/pull/16647 From dholmes at openjdk.org Tue Nov 14 07:45:29 2023 From: dholmes at openjdk.org (David Holmes) Date: Tue, 14 Nov 2023 07:45:29 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks In-Reply-To: References: Message-ID: <5maZLsqVb1mNYUvUJLwSiSF9xAww0GryVFtdKMBOmHk=.c8ff2d36-7337-4965-acc9-4a08750a258b@github.com> On Tue, 14 Nov 2023 06:50:58 GMT, Roman Marchenko wrote: > Zero'ing memory of extension event callbacks Marked as reviewed by dholmes (Reviewer). src/hotspot/share/prims/jvmtiEnvBase.cpp line 217: > 215: // all callbacks initially null > 216: memset(&_event_callbacks,0,sizeof(jvmtiEventCallbacks)); > 217: memset(&_ext_event_callbacks, 0, sizeof(jvmtiExtEventCallbacks)); Looks good. While you are here could you adjust the existing memset line and add spaces after the commas please. Thanks. ------------- PR Review: https://git.openjdk.org/jdk/pull/16647#pullrequestreview-1729084003 PR Review Comment: https://git.openjdk.org/jdk/pull/16647#discussion_r1392118874 From stuefe at openjdk.org Tue Nov 14 08:13:30 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Tue, 14 Nov 2023 08:13:30 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v5] In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 13:33:26 GMT, Thomas Stuefe wrote: >> When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). > > Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Merge branch 'master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 > - Fix Windows build > - remove tab > - Make patch more palatable > - Merge branch 'openjdk:master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 > - JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 @shipilev maybe, as bug owner? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16335#issuecomment-1809719133 From rmarchenko at openjdk.org Tue Nov 14 08:18:41 2023 From: rmarchenko at openjdk.org (Roman Marchenko) Date: Tue, 14 Nov 2023 08:18:41 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks [v2] In-Reply-To: References: Message-ID: > Zero'ing memory of extension event callbacks Roman Marchenko has updated the pull request incrementally with one additional commit since the last revision: Fixing review comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16647/files - new: https://git.openjdk.org/jdk/pull/16647/files/aa3bfaec..2ef1e341 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16647&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16647&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16647.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16647/head:pull/16647 PR: https://git.openjdk.org/jdk/pull/16647 From rmarchenko at openjdk.org Tue Nov 14 08:18:43 2023 From: rmarchenko at openjdk.org (Roman Marchenko) Date: Tue, 14 Nov 2023 08:18:43 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks [v2] In-Reply-To: <5maZLsqVb1mNYUvUJLwSiSF9xAww0GryVFtdKMBOmHk=.c8ff2d36-7337-4965-acc9-4a08750a258b@github.com> References: <5maZLsqVb1mNYUvUJLwSiSF9xAww0GryVFtdKMBOmHk=.c8ff2d36-7337-4965-acc9-4a08750a258b@github.com> Message-ID: On Tue, 14 Nov 2023 07:42:08 GMT, David Holmes wrote: >> Roman Marchenko has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixing review comments > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 217: > >> 215: // all callbacks initially null >> 216: memset(&_event_callbacks,0,sizeof(jvmtiEventCallbacks)); >> 217: memset(&_ext_event_callbacks, 0, sizeof(jvmtiExtEventCallbacks)); > > Looks good. > > While you are here could you adjust the existing memset line and add spaces after the commas please. Thanks. Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16647#discussion_r1392157095 From rmarchenko at openjdk.org Tue Nov 14 11:04:31 2023 From: rmarchenko at openjdk.org (Roman Marchenko) Date: Tue, 14 Nov 2023 11:04:31 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks [v2] In-Reply-To: References: Message-ID: <_6upH3UTrhu5j3PkSfF3na6dRbYKMCf-_8YQAWbctG8=.527eceac-8009-4b8a-92df-773d2c5717bd@github.com> On Tue, 14 Nov 2023 08:18:41 GMT, Roman Marchenko wrote: >> Zero'ing memory of extension event callbacks > > Roman Marchenko has updated the pull request incrementally with one additional commit since the last revision: > > Fixing review comments The previous test run was OK https://github.com/wkia/jdk/actions/runs/6852317395 Now it fails on MacOS: "hotspot/jtreg/gc/TestAllocHumongousFragment.java: java.lang.OutOfMemoryError: Java heap space", so I guess it is caused by infrastructure issues. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16647#issuecomment-1809990842 From rmarchenko at openjdk.org Tue Nov 14 14:39:43 2023 From: rmarchenko at openjdk.org (Roman Marchenko) Date: Tue, 14 Nov 2023 14:39:43 GMT Subject: Integrated: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 06:50:58 GMT, Roman Marchenko wrote: > Zero'ing memory of extension event callbacks This pull request has now been integrated. Changeset: 97ea5bf0 Author: Roman Marchenko Committer: Yuri Nesterenko URL: https://git.openjdk.org/jdk/commit/97ea5bf0ffafaf8009c19483b9a9b1c30401cf9a Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks Reviewed-by: dholmes ------------- PR: https://git.openjdk.org/jdk/pull/16647 From asmehra at openjdk.org Tue Nov 14 14:51:47 2023 From: asmehra at openjdk.org (Ashutosh Mehra) Date: Tue, 14 Nov 2023 14:51:47 GMT Subject: RFR: 8316342: CLHSDB "dumpclass" command produces invalid classes In-Reply-To: <-EIhfH1Toohh84d_Mza7fZSCDk21-bN0TII3kWYirO8=.dd880a1e-0731-456c-829e-98f40de6b613@github.com> References: <-EIhfH1Toohh84d_Mza7fZSCDk21-bN0TII3kWYirO8=.dd880a1e-0731-456c-829e-98f40de6b613@github.com> Message-ID: On Thu, 28 Sep 2023 23:40:58 GMT, Chris Plummer wrote: >> Please review this change to fix the operands of the bytecodes that operate on fields when dumping a class using SA. >> >> Testing: hotspot_serviceability >> >> I have also verified that `test/hotspot/jtreg/serviceability/sa/ClhsdbDumpclass.javaClhsdbDumpclass.java`, which is in the problem list because of this issue, passes with this change. >> I have also verified it by dumping the class that has getstatic/putstatic bytecodes and comparing the bytecodes manually with the original classfile. > > Ok. The changes look good to me, but I don't have much of understanding of the hotspot code involved here, so I'll defer to a hotspot expert for the 2nd review. I am returning from a break. Thanks @plummercj @sspitsyn for reviewing the changes. It looks like this doesn't have any merge conflicts, so can still be integrated. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15973#issuecomment-1810366426 From asmehra at openjdk.org Tue Nov 14 14:51:47 2023 From: asmehra at openjdk.org (Ashutosh Mehra) Date: Tue, 14 Nov 2023 14:51:47 GMT Subject: Integrated: 8316342: CLHSDB "dumpclass" command produces invalid classes In-Reply-To: References: Message-ID: On Thu, 28 Sep 2023 21:23:25 GMT, Ashutosh Mehra wrote: > Please review this change to fix the operands of the bytecodes that operate on fields when dumping a class using SA. > > Testing: hotspot_serviceability > > I have also verified that `test/hotspot/jtreg/serviceability/sa/ClhsdbDumpclass.javaClhsdbDumpclass.java`, which is in the problem list because of this issue, passes with this change. > I have also verified it by dumping the class that has getstatic/putstatic bytecodes and comparing the bytecodes manually with the original classfile. This pull request has now been integrated. Changeset: 7bb1999c Author: Ashutosh Mehra URL: https://git.openjdk.org/jdk/commit/7bb1999c51cdfeb020047e1094229fda1ec5a99d Stats: 49 lines in 2 files changed: 7 ins; 37 del; 5 mod 8316342: CLHSDB "dumpclass" command produces invalid classes Reviewed-by: cjplummer, sspitsyn ------------- PR: https://git.openjdk.org/jdk/pull/15973 From shade at openjdk.org Tue Nov 14 15:06:43 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 14 Nov 2023 15:06:43 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v5] In-Reply-To: References: Message-ID: On Mon, 13 Nov 2023 13:33:26 GMT, Thomas Stuefe wrote: >> When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). > > Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Merge branch 'master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 > - Fix Windows build > - remove tab > - Make patch more palatable > - Merge branch 'openjdk:master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 > - JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 src/hotspot/share/compiler/compilerOracle.cpp line 678: > 676: // Parse an uintx-based option value. Also takes care of parsing enum values for options that are enums. > 677: // Returns true if ok, false if the value could not be parsed. > 678: static bool parseUintxValue(enum CompileCommand option, const char* line, uintx& value, int& bytes_read) { It is honestly weird to see `parse***Uintx***Value` dealing with enums, and be specialized for `MemStat`. Can you reflow this to match how `MemLimit` does it? https://github.com/openjdk/jdk/blob/7bb1999c51cdfeb020047e1094229fda1ec5a99d/src/hotspot/share/compiler/compilerOracle.cpp#L702 src/hotspot/share/compiler/compilerOracle.cpp line 727: > 725: line += bytes_read; > 726: register_command(matcher, option, value); > 727: return; Why this `return` removed? All other cases in this file have the `return` after `register_command`, which I assume the style here: once any command is properly matched, return. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16335#discussion_r1392741630 PR Review Comment: https://git.openjdk.org/jdk/pull/16335#discussion_r1392718662 From jiangli at openjdk.org Tue Nov 14 20:24:27 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Tue, 14 Nov 2023 20:24:27 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Tue, 14 Nov 2023 03:10:20 GMT, David Holmes wrote: > Is this a case where the code should be checking for `is_attaching_via_jni()`? That's a good question. I think maybe we should try to completely avoid the situation where a 'partial' `JvmtiThreadState` is created when a native thread is attaching and is in the middle of allocating the thread oop. Looking at `JvmtiSampledObjectAllocEventCollector::start()`, I think we can check if the current JavaThread `is_attaching_via_jni()` and the `threadObj()` is null. If that's the case, don't try `setup_jvmti_thread_state()` as things are not ready. In `JvmtiThreadState::state_for_while_locked` we probably want to assert that thread->threadObj() is not null if thread->jvmti_thread_state() not null, to make sure that we don't see a incomplete `JvmtiThreadState`. @caoman, I think this can also address your input on keeping `JvmtiThreadState::_thread_oop_h` always properly initialized for the attaching native thread. I tested it and it seems to work well. I'll update this PR. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1811187153 From dcubed at openjdk.org Tue Nov 14 20:54:40 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Tue, 14 Nov 2023 20:54:40 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks [v2] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 08:18:41 GMT, Roman Marchenko wrote: >> Zero'ing memory of extension event callbacks > > Roman Marchenko has updated the pull request incrementally with one additional commit since the last revision: > > Fixing review comments Doing a post integration review. This is a trivial fix and does not need a second review nor wait 24 hours. Just a heads up that HotSpot code normally requires two reviews (1 from a (R)eviewer) and 24 hours unless it is called trivial AND agreed to be trivial by your reviewers. ------------- PR Review: https://git.openjdk.org/jdk/pull/16647#pullrequestreview-1730761590 PR Comment: https://git.openjdk.org/jdk/pull/16647#issuecomment-1811258657 From jiangli at openjdk.org Tue Nov 14 20:57:09 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Tue, 14 Nov 2023 20:57:09 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v2] In-Reply-To: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: > Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16642/files - new: https://git.openjdk.org/jdk/pull/16642/files/959305be..c2f83e8a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=00-01 Stats: 24 lines in 2 files changed: 19 ins; 4 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16642.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16642/head:pull/16642 PR: https://git.openjdk.org/jdk/pull/16642 From jjoo at openjdk.org Tue Nov 14 21:33:37 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 14 Nov 2023 21:33:37 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Update parallel workers time after Remark ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/533af850..189d1852 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=41 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=40-41 Stats: 10 lines in 4 files changed: 5 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Tue Nov 14 21:33:38 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 14 Nov 2023 21:33:38 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v41] In-Reply-To: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> References: <3kqtWcZA2lY3fPjUgyo5aO_-4SicTOPzF6AnKGyRCBA=.53d5b175-7e0f-4f6b-91a7-e4cfea62cb7a@github.com> Message-ID: On Sat, 11 Nov 2023 00:23:28 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Refactor ConcurrentRefine logic > - Make CPUTimeCounters a singleton class I believe all comments have been addressed and this PR is once again RFR! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1811353654 From jiangli at openjdk.org Tue Nov 14 21:45:33 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Tue, 14 Nov 2023 21:45:33 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v2] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 13 Nov 2023 23:04:19 GMT, Man Cao wrote: >> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. > > src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 94: > >> 92: // The state->get_thread_oop() may be null if the state is created during >> 93: // the allocation of the thread oop when a native thread is attaching. Make >> 94: // sure we don't create a new state for the JavaThread. > > I think it is important to maintain `JvmtiThreadState::_thread_oop_h` correctly for the attached native thread. In the existing logic, with and without the proposed change, `JvmtiThreadState::_thread_oop_h` could stay null for an attached native thread, which seems wrong. > > It may be OK since `JvmtiThreadState::_thread_oop_h` is only used by support for virtual threads. It is unlikely that an attached native thread becomes a carrier for a virtual thread. However, it is probably still desirable to update `JvmtiThreadState::_thread_oop_h` to the correct java.lang.Thread oop. Thanks for the input @caoman. I updated the PR to avoid creating a JvmtiThreadState during attaching and allocating thread oop. I think that avoids a incomplete JvmtiThreadState being created, which is seems to be cleaner. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1393334558 From amenkov at openjdk.org Tue Nov 14 23:26:11 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 14 Nov 2023 23:26:11 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references Message-ID: The change impelements dumping of unmounted virtual threads data (stack traces and stack references). Unmounted vthreads can be detected only by iterating over the heap, but hprof stack trace records (HPROF_FRAME/HPROF_TRACE) should be written before HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT. HeapDumper supports segment dump (parallel dump to separate files with subsequent file merge outside of safepoint), the fix switches HeapDumper to always use segment dump: 1st segment contains only non-heap data, other segments are used for dumping heap objects. For serial dumping single-threaded dumping is performed, but 2 segments are created anyway. When HeapObjectDumper detects unmounted virtual thread, it writes HPROF_FRAME/HPROF_TRACE records to the 1st segment ("global writer"), and writes thread object (HPROF_GC_ROOT_JAVA_FRAME) and stack references (HPROF_GC_ROOT_JAVA_FRAME/HPROF_GC_ROOT_JNI_LOCAL) to the HeapObjectDumper segment. As parallel dumpers may write HPROF_FRAME/HPROF_TRACE concurrently and VMDumper needs to write non-heap data before heap object dumpers can write virtual threads data, writing to global writer is protected with DumperController::_global_writer_lock. ------------- Commit messages: - vthreads in heapdump Changes: https://git.openjdk.org/jdk/pull/16665/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16665&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8299426 Stats: 308 lines in 3 files changed: 145 ins; 76 del; 87 mod Patch: https://git.openjdk.org/jdk/pull/16665.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16665/head:pull/16665 PR: https://git.openjdk.org/jdk/pull/16665 From amenkov at openjdk.org Wed Nov 15 02:35:01 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 15 Nov 2023 02:35:01 GMT Subject: RFR: 8320130: Problemlist 2 vmTestbase/nsk/jdi/StepRequest/addClassFilter_rt tests with Xcomp Message-ID: ProblemList 2 nsk jdi tests with Xcomp on all platforms ------------- Commit messages: - _cur_stack_depth_pl Changes: https://git.openjdk.org/jdk/pull/16667/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16667&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320130 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16667.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16667/head:pull/16667 PR: https://git.openjdk.org/jdk/pull/16667 From cjplummer at openjdk.org Wed Nov 15 04:48:28 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 15 Nov 2023 04:48:28 GMT Subject: RFR: 8320130: Problemlist 2 vmTestbase/nsk/jdi/StepRequest/addClassFilter_rt tests with Xcomp In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 02:24:17 GMT, Alex Menkov wrote: > ProblemList 2 nsk jdi tests with Xcomp on all platforms Marked as reviewed by cjplummer (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16667#pullrequestreview-1731230983 From stuefe at openjdk.org Wed Nov 15 06:22:58 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 15 Nov 2023 06:22:58 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v6] In-Reply-To: References: Message-ID: > When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: feedback shade ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16335/files - new: https://git.openjdk.org/jdk/pull/16335/files/7abf7cc4..591ce6e1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16335&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16335&range=04-05 Stats: 27 lines in 1 file changed: 9 ins; 8 del; 10 mod Patch: https://git.openjdk.org/jdk/pull/16335.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16335/head:pull/16335 PR: https://git.openjdk.org/jdk/pull/16335 From stuefe at openjdk.org Wed Nov 15 06:23:01 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 15 Nov 2023 06:23:01 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v5] In-Reply-To: References: Message-ID: <118aGIXApnvVIv016zUAUzrZgKyOa_duBzckMgqrTMM=.a0d755b2-a860-4032-94b5-78920c81804a@github.com> On Tue, 14 Nov 2023 15:03:42 GMT, Aleksey Shipilev wrote: >> Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: >> >> - Merge branch 'master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 >> - Fix Windows build >> - remove tab >> - Make patch more palatable >> - Merge branch 'openjdk:master' into JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 >> - JDK-8318671-Potential-uninitialized-uintx-value-after-JDK-8317683 > > src/hotspot/share/compiler/compilerOracle.cpp line 678: > >> 676: // Parse an uintx-based option value. Also takes care of parsing enum values for options that are enums. >> 677: // Returns true if ok, false if the value could not be parsed. >> 678: static bool parseUintxValue(enum CompileCommand option, const char* line, uintx& value, int& bytes_read) { > > It is honestly weird to see `parse***Uintx***Value` dealing with enums, and be specialized for `MemStat`. Can you reflow this to match how `MemLimit` does it? https://github.com/openjdk/jdk/blob/7bb1999c51cdfeb020047e1094229fda1ec5a99d/src/hotspot/share/compiler/compilerOracle.cpp#L702 Okay, rewritten in the style of https://github.com/openjdk/jdk/pull/16631. Retested too. Let's get this out of the door, please. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16335#discussion_r1393696933 From rmarchenko at openjdk.org Wed Nov 15 07:38:39 2023 From: rmarchenko at openjdk.org (Roman Marchenko) Date: Wed, 15 Nov 2023 07:38:39 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks [v2] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 20:51:22 GMT, Daniel D. Daugherty wrote: >> Roman Marchenko has updated the pull request incrementally with one additional commit since the last revision: >> >> Fixing review comments > > Just a heads up that HotSpot code normally requires two reviews (1 from a (R)eviewer) > and 24 hours unless it is called trivial AND agreed to be trivial by your reviewers. @dcubed-ojdk Sorry, I didn't know that. Could you point the discussion about +24 hours waiting please? BTW I seems like both requirements may be automated in github. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16647#issuecomment-1811943616 From sjohanss at openjdk.org Wed Nov 15 09:39:46 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 15 Nov 2023 09:39:46 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 21:33:37 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Update parallel workers time after Remark Thanks for addressing my comments. I have a few more things: - I think all changes to `test_g1ServiceThread.cpp` can be reverted. Should not be needed now - Please fix all whitespace issues - Should we move the VMThread and StringDedup counters into `CPUTimeCounters` as well? Any problem with this? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1812105608 From shade at openjdk.org Wed Nov 15 09:40:39 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 15 Nov 2023 09:40:39 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v6] In-Reply-To: References: Message-ID: <68GSCm9MVgGUvscnzMeeOIIn3wQDHVmKl6-UPDb5Iho=.58423501-ee27-49da-8dec-40e2fdc5438c@github.com> On Wed, 15 Nov 2023 06:22:58 GMT, Thomas Stuefe wrote: >> When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > feedback shade All right, new version looks significantly better, thanks! ------------- Marked as reviewed by shade (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16335#pullrequestreview-1731613654 From azafari at openjdk.org Wed Nov 15 09:41:38 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Wed, 15 Nov 2023 09:41:38 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: References: Message-ID: On Sun, 29 Oct 2023 08:07:55 GMT, Kim Barrett wrote: >> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). > >> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). > > Not touched since 2008 suggests to me there might not be a rush to make the change as proposed, and instead take > the (I think small) additional time to do the better thing, e.g. the unary-predicate suggestion made by several folks. @kimbarrett , @dholmes-ora , @merykitty Is there any comment on this PR? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1812108032 From stuefe at openjdk.org Wed Nov 15 09:58:55 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 15 Nov 2023 09:58:55 GMT Subject: RFR: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 [v6] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 06:22:58 GMT, Thomas Stuefe wrote: >> When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). > > Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision: > > feedback shade MacOS error unrelated. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16335#issuecomment-1812132637 From stuefe at openjdk.org Wed Nov 15 09:58:57 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 15 Nov 2023 09:58:57 GMT Subject: Integrated: JDK-8318671: Potential uninitialized uintx value after JDK-8317683 In-Reply-To: References: Message-ID: On Tue, 24 Oct 2023 07:08:07 GMT, Thomas Stuefe wrote: > When using 'MemStat' CompileCommand, we accidentally register the command if an invalid suboption had been specified. Fixed, added regression test (verified). This pull request has now been integrated. Changeset: 2e34a2eb Author: Thomas Stuefe URL: https://git.openjdk.org/jdk/commit/2e34a2ebf0f14043b129461b0397495e7e75a38b Stats: 105 lines in 3 files changed: 76 ins; 17 del; 12 mod 8318671: Potential uninitialized uintx value after JDK-8317683 Reviewed-by: thartmann, shade ------------- PR: https://git.openjdk.org/jdk/pull/16335 From dchuyko at openjdk.org Wed Nov 15 10:38:44 2023 From: dchuyko at openjdk.org (Dmitry Chuyko) Date: Wed, 15 Nov 2023 10:38:44 GMT Subject: RFR: 8309271: A way to align already compiled methods with compiler directives [v10] In-Reply-To: References: Message-ID: > Compiler Control (https://openjdk.org/jeps/165) provides method-context dependent control of the JVM compilers (C1 and C2). The active directive stack is built from the directive files passed with the `-XX:CompilerDirectivesFile` diagnostic command-line option and the Compiler.add_directives diagnostic command. It is also possible to clear all directives or remove the top from the stack. > > A matching directive will be applied at method compilation time when such compilation is started. If directives are added or changed, but compilation does not start, then the state of compiled methods doesn't correspond to the rules. This is not an error, and it happens in long running applications when directives are added or removed after compilation of methods that could be matched. For example, the user decides that C2 compilation needs to be disabled for some method due to a compiler bug, issues such a directive but this does not affect the application behavior. In such case, the target application needs to be restarted, and such an operation can have high costs and risks. Another goal is testing/debugging compilers. > > It would be convenient to optionally reconcile at least existing matching nmethods to the current stack of compiler directives (so bypass inlined methods). > > Natural way to eliminate the discrepancy between the result of compilation and the broken rule is to discard the compilation result, i.e. deoptimization. Prior to that we can try to re-compile the method letting compile broker to perform it taking new directives stack into account. Re-compilation helps to prevent hot methods from execution in the interpreter. > > A new flag `-r` has beed introduced for some directives related to compile commands: `Compiler.add_directives`, `Compiler.remove_directives`, `Compiler.clear_directives`. The default behavior has not changed (no flag). If the new flag is present, the command scans already compiled methods and puts methods that have any active non-default matching compiler directives to re-compilation if possible, otherwise marks them for deoptimization. There is currently no distinction which directives are found. In particular, this means that if there are rules for inlining into some method, it will be refreshed. On the other hand, if there are rules for a method and it was inlined, top-level methods won't be refreshed, but this can be achieved by having rules for them. > > In addition, a new diagnostic command `Compiler.replace_directives`, has been added for ... Dmitry Chuyko has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 28 commits: - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - Merge branch 'openjdk:master' into compiler-directives-force-update - jcheck - Unnecessary import - force_update->refresh - Merge branch 'openjdk:master' into compiler-directives-force-update - Use only top directive for add/remove; better mutex rank definition; texts - ... and 18 more: https://git.openjdk.org/jdk/compare/2e34a2eb...37c50c74 ------------- Changes: https://git.openjdk.org/jdk/pull/14111/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=14111&range=09 Stats: 372 lines in 15 files changed: 339 ins; 3 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/14111.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/14111/head:pull/14111 PR: https://git.openjdk.org/jdk/pull/14111 From dcubed at openjdk.org Wed Nov 15 17:07:31 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 15 Nov 2023 17:07:31 GMT Subject: RFR: 8320130: Problemlist 2 vmTestbase/nsk/jdi/StepRequest/addClassFilter_rt tests with Xcomp In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 02:24:17 GMT, Alex Menkov wrote: > ProblemList 2 nsk jdi tests with Xcomp on all platforms Thumbs up. This is a trivial fix. ------------- Marked as reviewed by dcubed (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16667#pullrequestreview-1732508536 From amenkov at openjdk.org Wed Nov 15 18:51:45 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Wed, 15 Nov 2023 18:51:45 GMT Subject: Integrated: 8320130: Problemlist 2 vmTestbase/nsk/jdi/StepRequest/addClassFilter_rt tests with Xcomp In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 02:24:17 GMT, Alex Menkov wrote: > ProblemList 2 nsk jdi tests with Xcomp on all platforms This pull request has now been integrated. Changeset: 536b1cee Author: Alex Menkov URL: https://git.openjdk.org/jdk/commit/536b1cee249ec50d6270ced3ba52ed3848c80a97 Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod 8320130: Problemlist 2 vmTestbase/nsk/jdi/StepRequest/addClassFilter_rt tests with Xcomp Reviewed-by: cjplummer, dcubed ------------- PR: https://git.openjdk.org/jdk/pull/16667 From dcubed at openjdk.org Wed Nov 15 19:03:52 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Wed, 15 Nov 2023 19:03:52 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks [v2] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 08:18:41 GMT, Roman Marchenko wrote: >> Zero'ing memory of extension event callbacks > > Roman Marchenko has updated the pull request incrementally with one additional commit since the last revision: > > Fixing review comments Sure. Here's the relevant sub-section from the "OpenJDK Developers? Guide": https://openjdk.org/guide/index.html#life-of-a-pr Get the required reviews At least one Reviewer knowledgeable in each area being changed must approve every change. Some areas (e.g. Client and HotSpot) require two reviewers in most cases, so be sure to read the relevant OpenJDK group pages for advice or ask your sponsor. Be open to comments and polite in replies. Remember that the reviewer wants to improve the world just as much as you do, only in a slightly different way. If you don?t understand some comment, ask the reviewer to clarify. Accept authority when applicable. If you?re making changes in an area where you?re not the area expert, acknowledge that your reviewers may be. Take their advice seriously, even if it is to not make the change. There are many reasons [why a change may get rejected](https://openjdk.org/guide/index.html#why-is-my-change-rejected). And you did read the section [Things to consider before changing OpenJDK code], right? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16647#issuecomment-1813094911 From jjoo at openjdk.org Wed Nov 15 23:43:57 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 15 Nov 2023 23:43:57 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v43] In-Reply-To: References: Message-ID: <0PG8RL0P8m4dbQe-gmltDKvcQYPJvUvUshV-anYvFFM=.41402e8d-c895-43b9-9567-036723afcabc@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Move vm and conc_dedup counters to cpuTimeCounters class - Revert test changes and fix whitespace issues ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/189d1852..4db8f09f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=42 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=41-42 Stats: 30 lines in 9 files changed: 11 ins; 5 del; 14 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Wed Nov 15 23:50:02 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 15 Nov 2023 23:50:02 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v44] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Fix whitespace ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/4db8f09f..ce7dbfcf Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=43 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=42-43 Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Thu Nov 16 00:17:42 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Thu, 16 Nov 2023 00:17:42 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 09:36:47 GMT, Stefan Johansson wrote: >> Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: >> >> Update parallel workers time after Remark > > Thanks for addressing my comments. I have a few more things: > > - I think all changes to `test_g1ServiceThread.cpp` can be reverted. Should not be needed now > - Please fix all whitespace issues > - Should we move the VMThread and StringDedup counters into `CPUTimeCounters` as well? Any problem with this? @kstefanj Good points - done! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1813500906 From manc at openjdk.org Thu Nov 16 00:27:44 2023 From: manc at openjdk.org (Man Cao) Date: Thu, 16 Nov 2023 00:27:44 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v44] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 23:50:02 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace Overall looks good, a few details could be improved. src/hotspot/share/gc/g1/g1CollectedHeap.hpp line 59: > 57: #include "memory/iterator.hpp" > 58: #include "memory/memRegion.hpp" > 59: #include "runtime/cpuTimeCounters.hpp" Probably move this include to the .cpp file? src/hotspot/share/gc/g1/g1ServiceThread.cpp line 161: > 159: if (UsePerfData && os::is_thread_cpu_time_supported()) { > 160: ThreadTotalCPUTimeClosure tttc(CPUTimeCounters::get_instance(), CPUTimeGroups::gc_service); > 161: tttc.do_thread(task->_service_thread); It could just use `do_thread(this)`, then it can remove the `task` parameter. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.cpp line 72: > 70: _processor = new Processor(); > 71: if (UsePerfData && os::is_thread_cpu_time_supported()) { > 72: EXCEPTION_MARK; This whole `if` block could be updated to `CPUTimeCounters::get_instance()->create_counter(CPUTimeGroups::conc_dedup)`. We can also remove the `_concurrent_dedup_thread_cpu_time` field and the `ThreadTotalCPUTimeClosure(PerfCounter* counter)` constructor. In `StringDedup::Processor::run()`, it can call if (UsePerfData && os::is_thread_cpu_time_supported()) { ThreadTotalCPUTimeClosure tttc(CPUTimeCounters::get_instance(), CPUTimeGroups::conc_dedup); tttc.do_thread(thread); } Similar, this can be applied to vmThread. src/hotspot/share/runtime/init.cpp line 124: > 122: codeCache_init(); > 123: VM_Version_init(); // depends on codeCache_init for emitting code > 124: // Initialize CPUTimeCounters object, which must be done before creation of the heap. Would it be possible to move this inside `universe_init()` in universe.cpp, somewhere above `Universe::initialize_heap()`? There's a similar `MetaspaceCounters::initialize_performance_counters()` in `universe_init()`. ------------- Changes requested by manc (Committer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1731128372 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395001799 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395007511 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395009733 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1395000735 From manc at openjdk.org Thu Nov 16 00:27:54 2023 From: manc at openjdk.org (Man Cao) Date: Thu, 16 Nov 2023 00:27:54 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v42] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 21:33:37 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Update parallel workers time after Remark src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 1519: > 1517: > 1518: CPUTimeCounters* instance = CPUTimeCounters::get_instance(); > 1519: assert(instance != nullptr, "no instance found"); It's probably better to move this `assert` inside the `CPUTimeCounters::get_instance()` body. src/hotspot/share/runtime/cpuTimeCounters.cpp line 43: > 41: case gc_service: > 42: return "gc_service"; > 43: case COUNT: "default" seems more appropriate than COUNT. This seems better? default: ShouldNotReachHere(); src/hotspot/share/runtime/cpuTimeCounters.cpp line 65: > 63: } > 64: > 65: CPUTimeCounters* CPUTimeCounters::_instance = nullptr; No need for extra whitespaces. Single space should be fine. src/hotspot/share/runtime/cpuTimeCounters.hpp line 2: > 1: /* > 2: * Copyright (c) 2002, 2019, Oracle and/or its affiliates. All rights reserved. Year should be "2023". src/hotspot/share/runtime/cpuTimeCounters.hpp line 32: > 30: #include "runtime/perfData.hpp" > 31: #include "runtime/perfDataTypes.hpp" > 32: Include "memory/iterator.hpp" and remove this include from perfData.hpp? src/hotspot/share/runtime/cpuTimeCounters.hpp line 35: > 33: class CPUTimeGroups : public AllStatic { > 34: public: > 35: enum CPUTimeType { I think new code should prefer `enum class` over plain `enum`. https://stackoverflow.com/q/18335861 src/hotspot/share/runtime/cpuTimeCounters.hpp line 36: > 34: public: > 35: enum CPUTimeType { > 36: total, Probably `gc_total` instead of `total`, since we will include non-GC counters in this class. Also for naming style, I find it better to be `GcTotal` or `GC_TOTAL` for public enum constants. But HotSpot has mixed styles for enums, so changing the names is optional. src/hotspot/share/runtime/cpuTimeCounters.hpp line 48: > 46: }; > 47: > 48: class CPUTimeCounters: public CHeapObj { Perhaps `mtServiceability` as hsperf counters are part of serviceability, and we will include non-GC hsperf counters. src/hotspot/share/runtime/cpuTimeCounters.hpp line 50: > 48: class CPUTimeCounters: public CHeapObj { > 49: private: > 50: // We want CPUTimeCounters to be a singleton instance accessed by the vm thread. Suggest remove "accessed by the vm thread". It is already access by non-VM threads like G1 concurrent mark thread and concurrent refine thread. src/hotspot/share/runtime/cpuTimeCounters.hpp line 61: > 59: // since the last time we called `publish_total_cpu_time()`. > 60: // It is incremented using Atomic::add() to prevent race conditions, and > 61: // is added to the `total` CPUTimeGroup at the end of GC. Ditto, better to have "gc" substring in these names: _total_cpu_time_diff inc_total_cpu_time() publish_total_cpu_time() src/hotspot/share/runtime/cpuTimeCounters.hpp line 80: > 78: void operator=(const CPUTimeCounters& copy) = delete; > 79: > 80: ~CPUTimeCounters(); No need to declare destructor since it is not overwritten. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1393555723 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394989485 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394992509 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394981679 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394997677 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394972807 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394976545 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394981147 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394969526 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394977769 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1394983409 From duke at openjdk.org Thu Nov 16 02:08:35 2023 From: duke at openjdk.org (Long Yang) Date: Thu, 16 Nov 2023 02:08:35 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v3] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Mon, 13 Nov 2023 10:11:26 GMT, Long Yang wrote: >> I would like to fix this. >> >> Create 4096 threads, and the stack depth of each thread is 256. >> After running jmx.dumpAllThreads(true, true), the RSS reaches 5.3GiB. >> After optimization, the RSS is 250MiB. >> >> I would appreciate it if anyone could review this. >> >> --------- >> update >> >> If the number of `threads` and `stack depth` are relatively large, we need to apply for more space in `ResourceArea` during the execution of `jmx.dumpAllThreads(true, true)`. >> >> The reason is that `VM_ThreadDump::doit` creates `vframe` for each `frame` of each `thread`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/services/threadService.cpp#L704 >> sizeof `vframe` is 4808 (bytes), and sizeof `compiledVFrame` is 4824 (bytes), mainly because the `xmm registers` in `RegisterMap` are relatively large. Assuming there are 4096 `threads` and each `thread` has 256 `frames`, the memory required is 4096 * 256 * 4824 = 4.7GiB? >> >> These memories of all threads are released once by the the initial `ResourceMark` of `VM_ThreadDump::doit`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/vmOperations.cpp#L269 >> My solution is to add a `ResourceMark` for each thread. > > Long Yang has updated the pull request incrementally with one additional commit since the last revision: > > Use VMThread::vm_thread() to avoid the need to call Thread::current() Hi I ran `tier1`, `tier2`, `tier3`, `tier4` on my host machine. `tier1`, `tier2`, and `tier3` all passed. Because my host does not have a display device, I added `export JTREG_KEYWORDS="!headful"` before running `tier4`. Finally, some tests in `tier4` that depend on the printing device failed, and the rest were successful. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16598#issuecomment-1813661550 From yyang at openjdk.org Thu Nov 16 02:43:32 2023 From: yyang at openjdk.org (Yi Yang) Date: Thu, 16 Nov 2023 02:43:32 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references In-Reply-To: References: Message-ID: <5mCTsa0npqQr9CQM4n9k5pgSzBCocyPJjjG47gYQC2Q=.ae34ca29-0bd7-42ab-9716-d70931551961@github.com> On Tue, 14 Nov 2023 21:54:06 GMT, Alex Menkov wrote: > The change impelements dumping of unmounted virtual threads data (stack traces and stack references). > Unmounted vthreads can be detected only by iterating over the heap, but hprof stack trace records (HPROF_FRAME/HPROF_TRACE) should be written before HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT. > HeapDumper supports segment dump (parallel dump to separate files with subsequent file merge outside of safepoint), the fix switches HeapDumper to always use segment dump: 1st segment contains only non-heap data, other segments are used for dumping heap objects. For serial dumping single-threaded dumping is performed, but 2 segments are created anyway. > When HeapObjectDumper detects unmounted virtual thread, it writes HPROF_FRAME/HPROF_TRACE records to the 1st segment ("global writer"), and writes thread object (HPROF_GC_ROOT_JAVA_FRAME) and stack references (HPROF_GC_ROOT_JAVA_FRAME/HPROF_GC_ROOT_JNI_LOCAL) to the HeapObjectDumper segment. > As parallel dumpers may write HPROF_FRAME/HPROF_TRACE concurrently and VMDumper needs to write non-heap data before heap object dumpers can write virtual threads data, writing to global writer is protected with DumperController::_global_writer_lock. > > Testing: run tests which perform heap dump (in different scenarios): > - test/hotspot/jtreg/serviceability > - test/hotspot/jtreg/runtime/ErrorHandling > - test/hotspot/jtreg/gc/epsilon > - test/jdk/sun/tools/jhsdb src/hotspot/share/services/heapDumper.cpp line 1896: > 1894: > 1895: public: > 1896: HeapObjectDumper(AbstractDumpWriter* writer, UnmountedVThreadDumper* vthread_dumper) Maybe we can pass global writer to HeapObjectDumper and thus simplify the implementation? We can remove `class VM_HeapDumper : public VM_GC_Operation, public WorkerTask, public UnmountedVThreadDumper {` and `UnmountedVThreadDumper` then. src/hotspot/share/services/heapDumper.cpp line 2024: > 2022: char* DumpMerger::get_writer_path(const char* base_path, int seq) { > 2023: // calculate required buffer size > 2024: size_t buf_size = strlen(base_path) Format src/hotspot/share/services/heapDumper.cpp line 2411: > 2409: > 2410: WorkerThreads* workers = ch->safepoint_workers(); > 2411: update_parallel_dump_count(workers); Maybe consolidate them together update_parallel_dump_count(workers); => prepare_parallel_dump(workers){ ... // copy from update_parallel_dump_count _dumper_controller = new (std::nothrow) DumperController(_num_dumper_threads); } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16665#discussion_r1395079535 PR Review Comment: https://git.openjdk.org/jdk/pull/16665#discussion_r1395071539 PR Review Comment: https://git.openjdk.org/jdk/pull/16665#discussion_r1395084194 From dholmes at openjdk.org Thu Nov 16 06:00:39 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 16 Nov 2023 06:00:39 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v8] In-Reply-To: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> References: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> Message-ID: On Tue, 7 Nov 2023 11:40:02 GMT, Afshin Zafari wrote: >> The `find` method now is >> ```C++ >> template >> int find(T* token, bool f(T*, E)) const { >> ... >> >> Any other functions which use this are also changed. >> Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. > > Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: > > function pointer is replaced with template Functor. src/hotspot/share/gc/parallel/mutableNUMASpace.cpp line 163: > 161: } > 162: // That's the normal case, where we know the locality group of the thread. > 163: int i = lgrp_spaces()->find((uint*)&lgrp_id, LGRPSpace::equals); I guess it is somewhat outside the scope of this PR but I wish this code would make its mind up about whether the NUMA group ids are `int` or `uint`! ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15418#discussion_r1395182714 From dholmes at openjdk.org Thu Nov 16 06:07:33 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 16 Nov 2023 06:07:33 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 09:38:25 GMT, Afshin Zafari wrote: >>> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). >> >> Not touched since 2008 suggests to me there might not be a rush to make the change as proposed, and instead take >> the (I think small) additional time to do the better thing, e.g. the unary-predicate suggestion made by several folks. > > @kimbarrett , @dholmes-ora , @merykitty > Is there any comment on this PR? @afshin-zafari I will leave it to other to (re-) review the latest changes. I don't grok this template stuff enough to pass judgement. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1813839612 From sspitsyn at openjdk.org Thu Nov 16 06:41:44 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 06:41:44 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v5] In-Reply-To: References: Message-ID: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes two new classes: > - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: addressed review: added check for jdk_internal_vm_Continuation::done(cont) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/ca2fbb98..13e05ab1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=03-04 Stats: 12 lines in 2 files changed: 4 ins; 5 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From qamai at openjdk.org Thu Nov 16 06:49:33 2023 From: qamai at openjdk.org (Quan Anh Mai) Date: Thu, 16 Nov 2023 06:49:33 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v8] In-Reply-To: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> References: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> Message-ID: <648SrHxCX6_kRX7cmxyGurxOWecLTWUw0_C79J_okbo=.473eaa8c-7dfc-4aa0-839d-bb580cc9d312@github.com> On Tue, 7 Nov 2023 11:40:02 GMT, Afshin Zafari wrote: >> The `find` method now is >> ```C++ >> template >> int find(T* token, bool f(T*, E)) const { >> ... >> >> Any other functions which use this are also changed. >> Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. > > Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: > > function pointer is replaced with template Functor. src/hotspot/share/utilities/growableArray.hpp line 213: > 211: > 212: template > 213: int find(T* token, F f) const { Should be template int find(F f) const { for (int i = 0; i < _len; i++) { if (f(_data[i]) { return i; } } return -1; } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15418#discussion_r1395219935 From sspitsyn at openjdk.org Thu Nov 16 06:59:49 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 06:59:49 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v6] In-Reply-To: References: Message-ID: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes two new classes: > - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: addressed review: (1) removed is_vthread_alive checks; (2) reverted one update ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/13e05ab1..a526bd60 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=04-05 Stats: 17 lines in 1 file changed: 0 ins; 15 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From sspitsyn at openjdk.org Thu Nov 16 07:13:36 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 07:13:36 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Wed, 8 Nov 2023 16:11:44 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiEnvBase.cpp line 1974: >> >>> 1972: >>> 1973: if (java_lang_VirtualThread::is_instance(target_h())) { // virtual thread >>> 1974: if (!JvmtiEnvBase::is_vthread_alive(target_h())) { >> >> There is only one issue I see in how this check is implemented and the removal of the VM_op for unmounted vthreads. The change of state to TERMINATED happens after notifyJvmtiUnmount(), i.e we can see that this vthread is alive here but a check later can return is not. This might hit the assert in JvmtiEnvBase::get_vthread_jvf() (maybe this the issue you saw on your first prototype). We can either change that order at the Java level, or maybe better change this function to read the state and add a case where if the state is RUNNING check whether the continuation is done or not (jdk_internal_vm_Continuation::done(cont)). > > Thank you for the suggestion. Will check it. I've added the check for `!jdk_internal_vm_Continuation::done(cont)` into ` JvmtiEnvBase::is_vthread_alive(oop vt)` but then decided to remove it again. This is racy for `JvmtiHandshake` execution. As you correctly stated, the change of state to `TERMINATED` happens after `notifyJvmtiUnmount()`. The target virtual thread will be blocked in the `notifyJvmtiUnmount()` because the `JvmtiVTMSTransitionDisabler` is set. This gives us a guaranty that the target virtual thread won't change its state to `TERMINATED` while a handshake is executed. But it becomes not true if we add the `!jdk_internal_vm_Continuation::done(cont)` check. Form the other hand, absence of this check allows for target virtual thread stack to become empty (with no frames). This is a known problem but I'd prefer to attack it separately. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1395238429 From dholmes at openjdk.org Thu Nov 16 09:32:36 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 16 Nov 2023 09:32:36 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v3] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Mon, 13 Nov 2023 10:11:26 GMT, Long Yang wrote: >> I would like to fix this. >> >> Create 4096 threads, and the stack depth of each thread is 256. >> After running jmx.dumpAllThreads(true, true), the RSS reaches 5.3GiB. >> After optimization, the RSS is 250MiB. >> >> I would appreciate it if anyone could review this. >> >> --------- >> update >> >> If the number of `threads` and `stack depth` are relatively large, we need to apply for more space in `ResourceArea` during the execution of `jmx.dumpAllThreads(true, true)`. >> >> The reason is that `VM_ThreadDump::doit` creates `vframe` for each `frame` of each `thread`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/services/threadService.cpp#L704 >> sizeof `vframe` is 4808 (bytes), and sizeof `compiledVFrame` is 4824 (bytes), mainly because the `xmm registers` in `RegisterMap` are relatively large. Assuming there are 4096 `threads` and each `thread` has 256 `frames`, the memory required is 4096 * 256 * 4824 = 4.7GiB? >> >> These memories of all threads are released once by the the initial `ResourceMark` of `VM_ThreadDump::doit`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/vmOperations.cpp#L269 >> My solution is to add a `ResourceMark` for each thread. > > Long Yang has updated the pull request incrementally with one additional commit since the last revision: > > Use VMThread::vm_thread() to avoid the need to call Thread::current() Okay I'm running this through our CI testing as well. I can't see any easy way to check for escaping resource objects other than testing. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16598#issuecomment-1814079577 From dholmes at openjdk.org Thu Nov 16 09:51:34 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 16 Nov 2023 09:51:34 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v2] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: <1sURlXfpTlEu9U30aZAojUIhDgtzeyA8MYrJ_q3xDUs=.bda6a027-ebc8-4107-a1f7-be3edf737e5f@github.com> On Tue, 14 Nov 2023 20:57:09 GMT, Jiangli Zhou wrote: >> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. > > Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: > > Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 87: > 85: // Don't add a JvmtiThreadState to a thread that is exiting. > 86: return nullptr; > 87: } I'm wondering if there should also be an `is_jni_attaching` check here? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1395431239 From dholmes at openjdk.org Thu Nov 16 10:22:47 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 16 Nov 2023 10:22:47 GMT Subject: RFR: 8319961: JvmtiEnvBase doesn't zero _ext_event_callbacks [v2] In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 08:18:41 GMT, Roman Marchenko wrote: >> Zero'ing memory of extension event callbacks > > Roman Marchenko has updated the pull request incrementally with one additional commit since the last revision: > > Fixing review comments Also see: https://openjdk.org/guide/index.html#hotspot-development and the definition of "trivial" in the Glossary. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16647#issuecomment-1814158850 From sspitsyn at openjdk.org Thu Nov 16 10:40:52 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 10:40:52 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v7] In-Reply-To: References: Message-ID: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes two new classes: > - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: - Merge - addressed review: (1) removed is_vthread_alive checks; (2) reverted one update - addressed review: added check for jdk_internal_vm_Continuation::done(cont) - review: get rid of the VM_HandshakeUnmountedVirtualThread - remove unneeded ResourceMark from JVMTI GetStackTrace - address review: remove fix in libGetStackTraceSuspendedStress.cpp - addressed initial minor review comments - 8319244: implement JVMTI handshakes support for virtual threads ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/a526bd60..2df63547 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=05-06 Stats: 634349 lines in 1310 files changed: 89706 ins; 480412 del; 64231 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From sspitsyn at openjdk.org Thu Nov 16 11:20:42 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 11:20:42 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 Message-ID: This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. Additionally, the fix includes the following related changes: - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: JvmtiThreadState* state = thread->jvmti_thread_state(); if (state != nullptr && state->is_pending_interp_only_mode()) { MutexLocker mu(JvmtiThreadState_lock); state = thread->jvmti_thread_state(); if (state != nullptr && state->is_pending_interp_only_mode()) { JvmtiEventController::enter_interp_only_mode(); } } - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. Performance mesurements: - Without this fix the test provided by the bug submitter gives execution numbers: - no ClassLoad events enabled: 3251 ms - ClassLoad events are enabled: 40534 ms - With the fix: - no ClassLoad events enabled: 3270 ms - ClassLoad events are enabled: 3385 ms Testing: - Ran mach5 tiers 1-6, no regressions are noticed ------------- Commit messages: - 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 Changes: https://git.openjdk.org/jdk/pull/16686/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16686&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8308614 Stats: 37 lines in 3 files changed: 24 ins; 4 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/16686.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16686/head:pull/16686 PR: https://git.openjdk.org/jdk/pull/16686 From duke at openjdk.org Thu Nov 16 12:18:50 2023 From: duke at openjdk.org (duke) Date: Thu, 16 Nov 2023 12:18:50 GMT Subject: Withdrawn: 8305895: Implementation: JEP 450: Compact Object Headers (Experimental) In-Reply-To: References: Message-ID: On Fri, 12 May 2023 17:27:25 GMT, Roman Kennke wrote: > This is the main body of the JEP 450: Compact Object Headers (Experimental). > > Main changes: > - Introduction of the (experimental) flag UseCompactObjectHeaders. All changes in this PR are protected by this flag. > - The compressed Klass* can now be stored in the mark-word of objects. In order to be able to do this, we are building on #10907, #13582 and #13779 to protect the relevant (upper 32) bits of the mark-word. Significant parts of this PR deal with loading the compressed Klass* from the mark-word, and dealing with (monitor-)locked objects. When the object is monitor-locked, we load the displaced mark-word from the monitor, and load the compressed Klass* from there. This PR also changes some code paths (mostly in GCs) to be more careful when accessing Klass* (or mark-word or size) to be able to fetch it from the forwardee in case the object is forwarded, and/or reach through to the monitor when the object is locked by a monitor. > - The identity hash-code is narrowed to 25 bits. > - Instances can now have their base-offset (the offset where the field layouter starts to place fields) at offset 8 (instead of 12 or 16). > - Arrays will can now store their length at offset 8. Due to alignment restrictions, array elements will still start at offset 16. #11044 will resolve that restriction and allow array elements to start at offset 12 (except for long, double and uncompressed oops, which are still required to start at an element-aligned offset). > - CDS can now write and read archives with the compressed header. However, it is not possible to read an archive that has been written with an opposite setting of UseCompactObjectHeaders. > > Testing: > (+UseCompactObjectHeaders tests are run with the flag hard-patched into the build, to also catch @flagless tests, and to avoid mismatches with CDS - see above.) > - [x] tier1 (x86_64) > - [x] tier2 (x86_64) > - [x] tier3 (x86_64) > - [ ] tier4 (x86_64) > - [x] tier1 (aarch64) > - [x] tier2 (aarch64) > - [x] tier3 (aarch64) > - [ ] tier4 (aarch64) > - [ ] tier1 (x86_64) +UseCompactObjectHeaders > - [ ] tier2 (x86_64) +UseCompactObjectHeaders > - [ ] tier3 (x86_64) +UseCompactObjectHeaders > - [ ] tier4 (x86_64) +UseCompactObjectHeaders > - [ ] tier1 (aarch64) +UseCompactObjectHeaders > - [ ] tier2 (aarch64) +UseCompactObjectHeaders > - [ ] tier3 (aarch64) +UseCompactObjectHeaders > - [ ] tier4 (aarch64) +UseCompactObjectHeaders This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/13961 From sspitsyn at openjdk.org Thu Nov 16 12:41:38 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 12:41:38 GMT Subject: RFR: 8320239: add dynamic switch for JvmtiVTMSTransitionDisabler sync protocol Message-ID: This is an update for a performance/scalability enhancement. The `JvmtiVTMSTransitionDisabler`sync protocol is on a performance critical path of the virtual threads mount state transitions (VTMS transitions). It has a noticeable performance overhead (about 10%) which contributes to the combined JVMTI performance overhead when Java apps are executed with loaded JVMTI agents. Please, also see another/related performance issue which contributes around 70% of total performance overhead: [8308614](https://bugs.openjdk.org/browse/JDK-8308614): Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 Testing: - Ran mach5 tiers 1-6 with no regressions noticed. ------------- Commit messages: - 8320239: add dynamic switch for JvmtiVTMSTransitionDisabler sync protocol Changes: https://git.openjdk.org/jdk/pull/16688/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16688&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320239 Stats: 41 lines in 2 files changed: 38 ins; 2 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16688.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16688/head:pull/16688 PR: https://git.openjdk.org/jdk/pull/16688 From sspitsyn at openjdk.org Thu Nov 16 12:45:48 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 12:45:48 GMT Subject: RFR: 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) [v2] In-Reply-To: References: Message-ID: > It is a fix of a minor test issue. > The test should not fail when the JVMTI function `SetEventNotificationMode()` returns errors codes: > - `JVMTI_ERROR_THREAD_NOT_ALIVE` > - `JVMTI_ERROR_WRONG_PHASE` > > Tested the fix locally and with mach5 test runs. Serguei Spitsyn has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge - 8318631: GetStackTraceSuspendedStressTest.java failed with: check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16488/files - new: https://git.openjdk.org/jdk/pull/16488/files/58609b44..4f9d30c0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16488&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16488&range=00-01 Stats: 634349 lines in 1310 files changed: 89706 ins; 480412 del; 64231 mod Patch: https://git.openjdk.org/jdk/pull/16488.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16488/head:pull/16488 PR: https://git.openjdk.org/jdk/pull/16488 From pchilanomate at openjdk.org Thu Nov 16 15:24:30 2023 From: pchilanomate at openjdk.org (Patricio Chilano Mateo) Date: Thu, 16 Nov 2023 15:24:30 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Thu, 16 Nov 2023 07:11:03 GMT, Serguei Spitsyn wrote: >> Thank you for the suggestion. Will check it. > > I've added the check for `!jdk_internal_vm_Continuation::done(cont)` into ` JvmtiEnvBase::is_vthread_alive(oop vt)` but then decided to remove it again. This is racy for `JvmtiHandshake` execution. As you correctly stated, the change of state to `TERMINATED` happens after `notifyJvmtiUnmount()`. The target virtual thread will be blocked in the `notifyJvmtiUnmount()` because the `JvmtiVTMSTransitionDisabler` is set. This gives us a guaranty that the target virtual thread won't change its state to `TERMINATED` while a handshake is executed. But it becomes not true if we add the `!jdk_internal_vm_Continuation::done(cont)` check. > Form the other hand, absence of this check allows for target virtual thread stack to become empty (with no frames). This is a known problem but I'd prefer to attack it separately. So the problematic case I'm thinking is when the JvmtiVTMSTransitionDisabler starts after the vthread executed notifyJvmtiUnmount(), i.e the vthread is already outside the transition, but before changing the state to TERMINATED. JvmtiVTMSTransitionDisabler will proceed, and since the carrierThread field has already been cleared we will treat it as an unmounted vthread. Then we can see first that is alive in JvmtiHandshake::execute() but then we could hit the assert that is already TERMINATED in JvmtiEnvBase::get_vthread_jvf(). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1395892738 From pchilanomate at openjdk.org Thu Nov 16 15:24:34 2023 From: pchilanomate at openjdk.org (Patricio Chilano Mateo) Date: Thu, 16 Nov 2023 15:24:34 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> Message-ID: On Wed, 8 Nov 2023 15:59:16 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiEnvBase.cpp line 1978: >> >>> 1976: } >>> 1977: if (target_jt == nullptr) { // unmounted virtual thread >>> 1978: hs_cl->do_vthread(target_h); // execute handshake closure callback on current thread directly >> >> I think comment should be: s/current thread/unmounted vthread > > Thank you for the comment but I'm not sure what do you mean. > If target virtual thread is unmounted we execute the hs_cl callback on current thread. Ok, I see. When I read "closure executed on" I think of the intended target thread of the closure rather than the current thread used. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1395894618 From sspitsyn at openjdk.org Thu Nov 16 16:30:29 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 16:30:29 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Thu, 16 Nov 2023 15:20:31 GMT, Patricio Chilano Mateo wrote: >> I've added the check for `!jdk_internal_vm_Continuation::done(cont)` into ` JvmtiEnvBase::is_vthread_alive(oop vt)` but then decided to remove it again. This is racy for `JvmtiHandshake` execution. As you correctly stated, the change of state to `TERMINATED` happens after `notifyJvmtiUnmount()`. The target virtual thread will be blocked in the `notifyJvmtiUnmount()` because the `JvmtiVTMSTransitionDisabler` is set. This gives us a guaranty that the target virtual thread won't change its state to `TERMINATED` while a handshake is executed. But it becomes not true if we add the `!jdk_internal_vm_Continuation::done(cont)` check. >> Form the other hand, absence of this check allows for target virtual thread stack to become empty (with no frames). This is a known problem but I'd prefer to attack it separately. > > So the problematic case I'm thinking is when the JvmtiVTMSTransitionDisabler starts after the vthread executed notifyJvmtiUnmount(), i.e the vthread is already outside the transition, but before changing the state to TERMINATED. JvmtiVTMSTransitionDisabler will proceed, and since the carrierThread field has already been cleared we will treat it as an unmounted vthread. Then we can see first that is alive in JvmtiHandshake::execute() but then we could hit the assert that is already TERMINATED in JvmtiEnvBase::get_vthread_jvf(). Thanks! This is a valid concern. Will try to fix this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1396000487 From lmesnik at openjdk.org Thu Nov 16 17:16:32 2023 From: lmesnik at openjdk.org (Leonid Mesnik) Date: Thu, 16 Nov 2023 17:16:32 GMT Subject: RFR: 8320239: add dynamic switch for JvmtiVTMSTransitionDisabler sync protocol In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 12:35:08 GMT, Serguei Spitsyn wrote: > This is an update for a performance/scalability enhancement. > > The `JvmtiVTMSTransitionDisabler`sync protocol is on a performance critical path of the virtual threads mount state transitions (VTMS transitions). It has a noticeable performance overhead (about 10%) which contributes to the combined JVMTI performance overhead when Java apps are executed with loaded JVMTI agents. > > Please, also see another/related performance issue which contributes around 70% of total performance overhead: > [8308614](https://bugs.openjdk.org/browse/JDK-8308614): Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 > > Testing: > - Ran mach5 tiers 1-6 with no regressions noticed. src/hotspot/share/prims/jvmtiThreadState.cpp line 430: > 428: assert(!thread->is_in_VTMS_transition(), "VTMS_transition sanity check"); > 429: thread->set_is_in_VTMS_transition(true); > 430: java_lang_Thread::set_is_in_VTMS_transition(vt, true); indentation is incorrect. src/hotspot/share/prims/jvmtiThreadState.hpp line 86: > 84: static volatile bool _SR_mode; // there is an active suspender or resumer > 85: static volatile int _VTMS_transition_count; // current number of VTMS transitions > 86: static int _sync_protocol_enabled_count; // current number of JvmtiVTMSTransitionDisablers enabled sync protocol The _sync_protocol_enabled_count and _sync_protocol_enabled_permanently are read/updated in different threads. How access to them is protected from racing? Might be make sense to add this info in comment? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16688#discussion_r1396061383 PR Review Comment: https://git.openjdk.org/jdk/pull/16688#discussion_r1396071674 From sspitsyn at openjdk.org Thu Nov 16 17:27:31 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 17:27:31 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Thu, 16 Nov 2023 16:27:23 GMT, Serguei Spitsyn wrote: >> So the problematic case I'm thinking is when the JvmtiVTMSTransitionDisabler starts after the vthread executed notifyJvmtiUnmount(), i.e the vthread is already outside the transition, but before changing the state to TERMINATED. JvmtiVTMSTransitionDisabler will proceed, and since the carrierThread field has already been cleared we will treat it as an unmounted vthread. Then we can see first that is alive in JvmtiHandshake::execute() but then we could hit the assert that is already TERMINATED in JvmtiEnvBase::get_vthread_jvf(). > > Thanks! This is a valid concern. Will try to fix this. I'm suggesting to fix it this way for the unmounted case only: @@ -1976,6 +1976,13 @@ JvmtiHandshake::execute(JvmtiUnitedHandshakeClosure* hs_cl, ThreadsListHandle* t return; } if (target_jt == nullptr) { // unmounted virtual thread + // JvmtiVTMSTransitionDisabler can start after the vthread executed notifyJvmtiUnmount(), i.e. + // the vthread is already outside the transition, but before changing the state to TERMINATED. + // Changing the state to TERMINATED is racy, so we check if the continuation is done in advance. + oop cont = java_lang_VirtualThread::continuation(target_h()); + if (jdk_internal_vm_Continuation::done(cont)) { + return; + } hs_cl->do_vthread(target_h); // execute handshake closure callback on current thread directly } } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1396084391 From sspitsyn at openjdk.org Thu Nov 16 17:45:36 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 17:45:36 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> Message-ID: On Wed, 8 Nov 2023 16:02:08 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiEnvBase.cpp line 2416: >> >>> 2414: if (!JvmtiEnvBase::is_vthread_alive(_target_h())) { >>> 2415: return; // JVMTI_ERROR_THREAD_NOT_ALIVE (default) >>> 2416: } >> >> Don't we have this check already in JvmtiHandshake::execute()? Same with the other converted functions. > > Good suggestion, thanks. > I'm a little bit paranoid about terminated vthreads. :) > Will try to get rid of it and retest all tiers. I've removed the extra checks for `JvmtiEnvBase::is_vthread_alive()`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1396115170 From sspitsyn at openjdk.org Thu Nov 16 18:57:32 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 18:57:32 GMT Subject: RFR: 8320239: add dynamic switch for JvmtiVTMSTransitionDisabler sync protocol In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 17:05:17 GMT, Leonid Mesnik wrote: >> This is an update for a performance/scalability enhancement. >> >> The `JvmtiVTMSTransitionDisabler`sync protocol is on a performance critical path of the virtual threads mount state transitions (VTMS transitions). It has a noticeable performance overhead (about 10%) which contributes to the combined JVMTI performance overhead when Java apps are executed with loaded JVMTI agents. >> >> Please, also see another/related performance issue which contributes around 70% of total performance overhead: >> [8308614](https://bugs.openjdk.org/browse/JDK-8308614): Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 >> >> Testing: >> - Ran mach5 tiers 1-6 with no regressions noticed. > > src/hotspot/share/prims/jvmtiThreadState.cpp line 430: > >> 428: assert(!thread->is_in_VTMS_transition(), "VTMS_transition sanity check"); >> 429: thread->set_is_in_VTMS_transition(true); >> 430: java_lang_Thread::set_is_in_VTMS_transition(vt, true); > > indentation is incorrect. Thank you. Fixed now. > src/hotspot/share/prims/jvmtiThreadState.hpp line 86: > >> 84: static volatile bool _SR_mode; // there is an active suspender or resumer >> 85: static volatile int _VTMS_transition_count; // current number of VTMS transitions >> 86: static int _sync_protocol_enabled_count; // current number of JvmtiVTMSTransitionDisablers enabled sync protocol > > The _sync_protocol_enabled_count and _sync_protocol_enabled_permanently are read/updated in different threads. How access to them is protected from racing? Might be make sense to add this info in comment? Good catch, thanks. My initial intention was to make them volatile with Atomic load/store/update. Fixed now. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16688#discussion_r1396191582 PR Review Comment: https://git.openjdk.org/jdk/pull/16688#discussion_r1396194416 From pchilanomate at openjdk.org Thu Nov 16 19:05:30 2023 From: pchilanomate at openjdk.org (Patricio Chilano Mateo) Date: Thu, 16 Nov 2023 19:05:30 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Thu, 16 Nov 2023 17:22:35 GMT, Serguei Spitsyn wrote: >> Thanks! This is a valid concern. Will try to fix this. > > I'm suggesting to fix it this way for the unmounted case only: > > @@ -1976,6 +1976,13 @@ JvmtiHandshake::execute(JvmtiUnitedHandshakeClosure* hs_cl, ThreadsListHandle* t > return; > } > if (target_jt == nullptr) { // unmounted virtual thread > + // JvmtiVTMSTransitionDisabler can start after the vthread executed notifyJvmtiUnmount(), i.e. > + // the vthread is already outside the transition, but before changing the state to TERMINATED. > + // Changing the state to TERMINATED is racy, so we check if the continuation is done in advance. > + oop cont = java_lang_VirtualThread::continuation(target_h()); > + if (jdk_internal_vm_Continuation::done(cont)) { > + return; > + } > hs_cl->do_vthread(target_h); // execute handshake closure callback on current thread directly > } > } Sounds good. Is there a reason why not have the check inside JvmtiEnvBase::is_vthread_alive()? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1396202630 From jbachorik at openjdk.org Thu Nov 16 20:33:50 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 16 Nov 2023 20:33:50 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes Message-ID: Please, review this fix for a corner case handling of `jmethodID` values. The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ ------------- Commit messages: - 8313816: Accessing jmethodID might lead to spurious crashes Changes: https://git.openjdk.org/jdk/pull/16662/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8313816 Stats: 282 lines in 9 files changed: 278 ins; 0 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From dholmes at openjdk.org Thu Nov 16 20:33:54 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 16 Nov 2023 20:33:54 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 17:56:09 GMT, Jaroslav Bachorik wrote: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ src/hotspot/share/oops/instanceKlass.cpp line 531: > 529: > 530: void InstanceKlass::deallocate_methods(ClassLoaderData* loader_data, > 531: Array* methods, InstanceKlass* klass) { An explicit boolean parameter would be cleaner/clearer. src/hotspot/share/oops/instanceKlass.cpp line 542: > 540: if (klass) { > 541: jmethodID jmid = method->find_jmethod_id_or_null(); > 542: // Do the pointer maintenance before releasing the metadata, just in case I assume there should be a period after 'case`. But just in case of what? src/hotspot/share/oops/instanceKlass.cpp line 549: > 547: if (jmid != nullptr && *((Method**)jmid) == method) { > 548: *((Method**)jmid) = nullptr; > 549: } This should be abstracted behind a utility function e.g. `method->clear_jmethod_id()`. src/hotspot/share/oops/method.cpp line 2277: > 2275: } > 2276: } > 2277: Can this race with redefinition? src/hotspot/share/oops/method.hpp line 730: > 728: // so handles are not used to avoid deadlock. > 729: jmethodID find_jmethod_id_or_null() { > 730: return method_holder() != nullptr ? method_holder()->jmethod_id_or_null(this) : nullptr; If `method_holder()` is null at this point what does that mean for the lifecycle of the Method? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1393663791 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1393664277 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1393672300 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1395072721 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1395071297 From jbachorik at openjdk.org Thu Nov 16 20:33:54 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 16 Nov 2023 20:33:54 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 05:35:49 GMT, David Holmes wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > src/hotspot/share/oops/instanceKlass.cpp line 531: > >> 529: >> 530: void InstanceKlass::deallocate_methods(ClassLoaderData* loader_data, >> 531: Array* methods, InstanceKlass* klass) { > > An explicit boolean parameter would be cleaner/clearer. I just removed the `klass` argument. It is not really used anyway. > src/hotspot/share/oops/instanceKlass.cpp line 542: > >> 540: if (klass) { >> 541: jmethodID jmid = method->find_jmethod_id_or_null(); >> 542: // Do the pointer maintenance before releasing the metadata, just in case > > I assume there should be a period after 'case`. But just in case of what? The code was moved to`method.cpp` and this particular comment line became obsolete > src/hotspot/share/oops/instanceKlass.cpp line 549: > >> 547: if (jmid != nullptr && *((Method**)jmid) == method) { >> 548: *((Method**)jmid) = nullptr; >> 549: } > > This should be abstracted behind a utility function e.g. `method->clear_jmethod_id()`. Done > src/hotspot/share/oops/method.cpp line 2277: > >> 2275: } >> 2276: } >> 2277: > > Can this race with redefinition? The cleanup of previous versions is executed in VM_Operation at a safepoint - therefore we should be safe against races with class redefinitions. I am adding an assert to `clear_jmethod_id()` to check for being at a safepoint. > src/hotspot/share/oops/method.hpp line 730: > >> 728: // so handles are not used to avoid deadlock. >> 729: jmethodID find_jmethod_id_or_null() { >> 730: return method_holder() != nullptr ? method_holder()->jmethod_id_or_null(this) : nullptr; > > If `method_holder()` is null at this point what does that mean for the lifecycle of the Method? Please, ignore this part of code for the time being. I had a crash in CI which was pointing vaguely to this code - unfortunately, the hs_err.log files are not preserved in the test archives and I am not able to reproduce the failure locally. I need to debug the crash and make sure I understand the root cause. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1394642247 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1394647034 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1394647173 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1395100685 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1395102362 From jbachorik at openjdk.org Thu Nov 16 20:33:55 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 16 Nov 2023 20:33:55 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 18:47:19 GMT, Jaroslav Bachorik wrote: >> src/hotspot/share/oops/instanceKlass.cpp line 531: >> >>> 529: >>> 530: void InstanceKlass::deallocate_methods(ClassLoaderData* loader_data, >>> 531: Array* methods, InstanceKlass* klass) { >> >> An explicit boolean parameter would be cleaner/clearer. > > I just removed the `klass` argument. It is not really used anyway. I actually ended up with a boolean parameter here. >> src/hotspot/share/oops/method.hpp line 730: >> >>> 728: // so handles are not used to avoid deadlock. >>> 729: jmethodID find_jmethod_id_or_null() { >>> 730: return method_holder() != nullptr ? method_holder()->jmethod_id_or_null(this) : nullptr; >> >> If `method_holder()` is null at this point what does that mean for the lifecycle of the Method? > > Please, ignore this part of code for the time being. I had a crash in CI which was pointing vaguely to this code - unfortunately, the hs_err.log files are not preserved in the test archives and I am not able to reproduce the failure locally. I need to debug the crash and make sure I understand the root cause. _Update:_ I was able to get to the bottom of the methods not having method holder associated with them. The `ClassFileParser` does not finalize initialization of the `InstanceKlass` it has created if `_klass != nullptr` (https://github.com/openjdk/jdk/blob/9727f4bdddc071e6f59806087339f345405ab004/src/hotspot/share/classfile/classFileParser.cpp#L5161). This also means, that the `Method` instances are not wired to their method holders via 'constant method'->'constant pool'->'pool holder' chain. However, they need to be deallocated and as such I really need a distinguishing argument for `InstanceKlass::deallocate_methods` call such that I don't attempt to resolve `jmethodid` values in that case. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1396296501 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1396175846 From cjplummer at openjdk.org Thu Nov 16 21:40:36 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Thu, 16 Nov 2023 21:40:36 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 11:15:27 GMT, Serguei Spitsyn wrote: > This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. > However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. > It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. > It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. > > Additionally, the fix includes the following related changes: > - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: > > JvmtiThreadState* state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > MutexLocker mu(JvmtiThreadState_lock); > state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > JvmtiEventController::enter_interp_only_mode(); > } > } > > > - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. > - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. > > Performance mesurements: > - Without this fix the test provided by the bug submitter gives execution numbers: > - no ClassLoad events enabled: 3251 ms > - ClassLoad events are enabled: 40534 ms > > - With the fix: > - no ClassLoad events enabled: 3270 ms > - ClassLoad events are enabled: 3385 ms > > Testing: > - Ran mach5 tiers 1-6, no regressions are noticed src/hotspot/share/prims/jvmtiEventController.cpp line 372: > 370: return; // EnterInterpOnlyModeClosure will be executed right after mount. > 371: } > 372: ThreadsListHandle tlh(current); Why was this added? src/hotspot/share/prims/jvmtiThreadState.cpp line 531: > 529: > 530: // JvmtiThreadState objects for virtual thread filtered events enabled globally > 531: // must be created eagerly if the interp_only_mode is enabled. Otherwise, This sentence is hard to read. How about: "If interp_only_mode is enabled then we must eagerly create JvmtiThreadState objects for globally enabled virtual thread filtered events." src/hotspot/share/prims/jvmtiThreadState.cpp line 579: > 577: VTMS_mount_end(vthread); > 578: if (JvmtiExport::can_support_virtual_threads() && > 579: JvmtiExport::should_post_vthread_mount()) { It seems odd that "can_support" can be false when "should_post" is true. I would think that "should_post" would always be false when "can_support" is false, and therefore there would be no need to check "can_support". src/hotspot/share/prims/jvmtiThreadState.hpp line 234: > 232: inline void set_head_env_thread_state(JvmtiEnvThreadState* ets); > 233: > 234: static bool _seen_interp_only_mode; // needed for optimization Say what the flag represents, not why we have it. src/hotspot/share/prims/jvmtiThreadState.hpp line 257: > 255: // JvmtiThreadState objects for virtual thread filtered events enabled globally > 256: // must be created eagerly if the interp_only_mode is enabled. Otherwise, > 257: // it is an important optimization to create JvmtiThreadState objects lazily. No need for this comment here. It is already at the call site, which is where it belongs. Instead the comment here should say what this API does (return true if any thread has entered interp_only_mode at any point during the JVMs execution). ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396375451 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396372511 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396371590 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396362847 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396374577 From amenkov at openjdk.org Thu Nov 16 21:48:32 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Thu, 16 Nov 2023 21:48:32 GMT Subject: RFR: 8299426: Heap dump does not contain virtual Thread stack references In-Reply-To: <5mCTsa0npqQr9CQM4n9k5pgSzBCocyPJjjG47gYQC2Q=.ae34ca29-0bd7-42ab-9716-d70931551961@github.com> References: <5mCTsa0npqQr9CQM4n9k5pgSzBCocyPJjjG47gYQC2Q=.ae34ca29-0bd7-42ab-9716-d70931551961@github.com> Message-ID: On Thu, 16 Nov 2023 02:30:23 GMT, Yi Yang wrote: >> The change impelements dumping of unmounted virtual threads data (stack traces and stack references). >> Unmounted vthreads can be detected only by iterating over the heap, but hprof stack trace records (HPROF_FRAME/HPROF_TRACE) should be written before HPROF_HEAP_DUMP/HPROF_HEAP_DUMP_SEGMENT. >> HeapDumper supports segment dump (parallel dump to separate files with subsequent file merge outside of safepoint), the fix switches HeapDumper to always use segment dump: 1st segment contains only non-heap data, other segments are used for dumping heap objects. For serial dumping single-threaded dumping is performed, but 2 segments are created anyway. >> When HeapObjectDumper detects unmounted virtual thread, it writes HPROF_FRAME/HPROF_TRACE records to the 1st segment ("global writer"), and writes thread object (HPROF_GC_ROOT_JAVA_FRAME) and stack references (HPROF_GC_ROOT_JAVA_FRAME/HPROF_GC_ROOT_JNI_LOCAL) to the HeapObjectDumper segment. >> As parallel dumpers may write HPROF_FRAME/HPROF_TRACE concurrently and VMDumper needs to write non-heap data before heap object dumpers can write virtual threads data, writing to global writer is protected with DumperController::_global_writer_lock. >> >> Testing: run tests which perform heap dump (in different scenarios): >> - test/hotspot/jtreg/serviceability >> - test/hotspot/jtreg/runtime/ErrorHandling >> - test/hotspot/jtreg/gc/epsilon >> - test/jdk/sun/tools/jhsdb > > src/hotspot/share/services/heapDumper.cpp line 1896: > >> 1894: >> 1895: public: >> 1896: HeapObjectDumper(AbstractDumpWriter* writer, UnmountedVThreadDumper* vthread_dumper) > > Maybe we can pass global writer to HeapObjectDumper and thus simplify the implementation? We can remove `class VM_HeapDumper : public VM_GC_Operation, public WorkerTask, public UnmountedVThreadDumper {` and `UnmountedVThreadDumper` then. This was an initial plan, but it would require more refactoring (dumper for unmounted vthreads needs a way to lock global writer, so need to move DumperController declaration and definition or add new 'class GlobalDumpWriter: public DumpWriter' and implement the lock there). I prefer to make functional changes here with minimum refactoring to simplify review and do refactoring in follow-up fixes > src/hotspot/share/services/heapDumper.cpp line 2024: > >> 2022: char* DumpMerger::get_writer_path(const char* base_path, int seq) { >> 2023: // calculate required buffer size >> 2024: size_t buf_size = strlen(base_path) > > Format fixed > src/hotspot/share/services/heapDumper.cpp line 2411: > >> 2409: >> 2410: WorkerThreads* workers = ch->safepoint_workers(); >> 2411: update_parallel_dump_count(workers); > > Maybe consolidate them together > > update_parallel_dump_count(workers); > > > => > > prepare_parallel_dump(workers){ > ... // copy from update_parallel_dump_count > _dumper_controller = new (std::nothrow) DumperController(_num_dumper_threads); > } Done ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16665#discussion_r1396386291 PR Review Comment: https://git.openjdk.org/jdk/pull/16665#discussion_r1396359118 PR Review Comment: https://git.openjdk.org/jdk/pull/16665#discussion_r1396359593 From manc at openjdk.org Thu Nov 16 22:00:31 2023 From: manc at openjdk.org (Man Cao) Date: Thu, 16 Nov 2023 22:00:31 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v2] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Tue, 14 Nov 2023 20:57:09 GMT, Jiangli Zhou wrote: >> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. > > Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: > > Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. Thanks. The latest change to `JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample()` looks OK to me. Skipping a few allocations for JVMTI allocation sampler is better than resulting in a problematic `JvmtiThreadState` instance. My main question is if we can now change `if (state == nullptr || state->get_thread_oop() != thread_oop) ` to `if (state == nullptr)` in `JvmtiThreadState::state_for_while_locked()`. I suspect we would never run into a case of `state != nullptr && state->get_thread_oop() != thread_oop` with the latest change, even with virtual threads. This is backed up by testing with https://github.com/openjdk/jdk/commit/00ace66c36243671a0fb1b673b3f9845460c6d22 not triggering any failure. If we run into such as a case, it could still be problematic as `JvmtiThreadState::state_for_while_locked()` would allocate a new `JvmtiThreadState` instance pointing to the same JavaThread, and it does not delete the existing instance. Could anyone with deep knowledge on JvmtiThreadState and virtual threads provide some feedback on this change and https://bugs.openjdk.org/browse/JDK-8319935? @AlanBateman, do you know who would be the best reviewer for this? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1815379890 From sspitsyn at openjdk.org Thu Nov 16 22:46:31 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Thu, 16 Nov 2023 22:46:31 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Thu, 16 Nov 2023 19:02:39 GMT, Patricio Chilano Mateo wrote: >> I'm suggesting to fix it this way for the unmounted case only: >> >> @@ -1976,6 +1976,13 @@ JvmtiHandshake::execute(JvmtiUnitedHandshakeClosure* hs_cl, ThreadsListHandle* t >> return; >> } >> if (target_jt == nullptr) { // unmounted virtual thread >> + // JvmtiVTMSTransitionDisabler can start after the vthread executed notifyJvmtiUnmount(), i.e. >> + // the vthread is already outside the transition, but before changing the state to TERMINATED. >> + // Changing the state to TERMINATED is racy, so we check if the continuation is done in advance. >> + oop cont = java_lang_VirtualThread::continuation(target_h()); >> + if (jdk_internal_vm_Continuation::done(cont)) { >> + return; >> + } >> hs_cl->do_vthread(target_h); // execute handshake closure callback on current thread directly >> } >> } > > Sounds good. Is there a reason why not have the check inside JvmtiEnvBase::is_vthread_alive()? If it is a part of the `JvmtiEnvBase::is_vthread_alive()` then it is racy for mounted virtual threads. It is not racy for unmounted virtual threads. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1396456207 From pchilanomate at openjdk.org Fri Nov 17 02:13:30 2023 From: pchilanomate at openjdk.org (Patricio Chilano Mateo) Date: Fri, 17 Nov 2023 02:13:30 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Thu, 16 Nov 2023 22:44:12 GMT, Serguei Spitsyn wrote: >> Sounds good. Is there a reason why not have the check inside JvmtiEnvBase::is_vthread_alive()? > > If it is a part of the `JvmtiEnvBase::is_vthread_alive()` then it is racy for mounted virtual threads. > It is not racy for unmounted virtual threads. So we should only see that a continuation is done for an unmounted vthread. The last place where we could see a mounted vthread is at notifyJvmtiEnd(), blocked in start_VTMS_transition(), but the continuation is not marked done yet. Also I realize the window for the problematic case I mentioned starts even earlier at notifyJvmtiEnd(), not notifyJvmtiUnmount(), because blocking due to JvmtiVTMSTransitionDisabler happens in start_VTMS_transition() not finish_VTMS_transition(). Once the vthread executed notifyJvmtiEnd() any JvmtiVTMSTransitionDisabler that happens afterwards will fall into this case. So maybe the first sentence of the comment should instead be: "The vthread could have already executed the last unmount but might not have changed state to TERMINATED yet." ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1396592975 From jiangli at openjdk.org Fri Nov 17 02:53:36 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Fri, 17 Nov 2023 02:53:36 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v2] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Tue, 14 Nov 2023 20:57:09 GMT, Jiangli Zhou wrote: >> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. > > Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: > > Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. > Thanks. The latest change to `JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample()` looks OK to me. Skipping a few allocations for JVMTI allocation sampler is better than resulting in a problematic `JvmtiThreadState` instance. > > My main question is if we can now change `if (state == nullptr || state->get_thread_oop() != thread_oop) ` to `if (state == nullptr)` in `JvmtiThreadState::state_for_while_locked()`. I suspect we would never run into a case of `state != nullptr && state->get_thread_oop() != thread_oop` with the latest change, even with virtual threads. This is backed up by testing with [00ace66](https://github.com/openjdk/jdk/commit/00ace66c36243671a0fb1b673b3f9845460c6d22) not triggering any failure. > > If we run into such as a case, it could still be problematic as `JvmtiThreadState::state_for_while_locked()` would allocate a new `JvmtiThreadState` instance pointing to the same JavaThread, and it does not delete the existing instance. > > Could anyone with deep knowledge on JvmtiThreadState and virtual threads provide some feedback on this change and https://bugs.openjdk.org/browse/JDK-8319935? @AlanBateman, do you know who would be the best reviewer for this? @caoman and I discussed about his suggestion on changing `if (state == nullptr || state->get_thread_oop() != thread_oop)` check in person today. Since it may affect vthread, my main concern is that our current testing may not cover that sufficiently. The suggestion could be worked by a separate enhancement bug. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1815667615 From sspitsyn at openjdk.org Fri Nov 17 04:38:27 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 04:38:27 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v4] In-Reply-To: References: <6vZHqAagtrtDw-c9xNZNpaBCa1HrpYw22uhYtDTHwAw=.3a59dde0-ab33-43b8-a08f-d481e7acffef@github.com> <2F8ze1cLKzvTMEqwY8JJMZ9QbZUxqrMCv7nl6uFJLMI=.7c4dba0a-8c8d-4825-9670-75b76b9bf184@github.com> Message-ID: On Fri, 17 Nov 2023 02:10:50 GMT, Patricio Chilano Mateo wrote: >> If it is a part of the `JvmtiEnvBase::is_vthread_alive()` then it is racy for mounted virtual threads. >> It is not racy for unmounted virtual threads. > > So we should only see that a continuation is done for an unmounted vthread. The last place where we could see a mounted vthread is at notifyJvmtiEnd(), blocked in start_VTMS_transition(), but the continuation is not marked done yet. > Also I realize the window for the problematic case I mentioned starts even earlier at notifyJvmtiEnd(), not notifyJvmtiUnmount(), because blocking due to JvmtiVTMSTransitionDisabler happens in start_VTMS_transition() not finish_VTMS_transition(). Once the vthread executed notifyJvmtiEnd() any JvmtiVTMSTransitionDisabler that happens afterwards will fall into this case. So maybe the first sentence of the comment should instead be: "The vthread could have already executed the last unmount but might not have changed state to TERMINATED yet." Thank you. Now I see that `done` is set to `true` during an unmount transition. So, I'm convinced to move the check for `jdk_internal_vm_Continuation::done(cont)` to `JvmtiEnvBase::is_vthread_alive()`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1396695782 From sspitsyn at openjdk.org Fri Nov 17 05:37:46 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 05:37:46 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v8] In-Reply-To: References: Message-ID: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes two new classes: > - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: add jdk_internal_vm_Continuation::done(cont) check to JvmtiEnvBase::is_vthread_alive ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/2df63547..e61d0703 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=06-07 Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From sspitsyn at openjdk.org Fri Nov 17 05:46:37 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 05:46:37 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 In-Reply-To: References: Message-ID: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> On Thu, 16 Nov 2023 21:36:28 GMT, Chris Plummer wrote: >> This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. >> However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. >> It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. >> It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. >> >> Additionally, the fix includes the following related changes: >> - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: >> >> JvmtiThreadState* state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> MutexLocker mu(JvmtiThreadState_lock); >> state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> JvmtiEventController::enter_interp_only_mode(); >> } >> } >> >> >> - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. >> - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. >> >> Performance mesurements: >> - Without this fix the test provided by the bug submitter gives execution numbers: >> - no ClassLoad events enabled: 3251 ms >> - ClassLoad events are enabled: 40534 ms >> >> - With the fix: >> - no ClassLoad events enabled: 3270 ms >> - ClassLoad events are enabled: 3385 ms >> >> Testing: >> - Ran mach5 tiers 1-6, no regressions are noticed > > src/hotspot/share/prims/jvmtiEventController.cpp line 372: > >> 370: return; // EnterInterpOnlyModeClosure will be executed right after mount. >> 371: } >> 372: ThreadsListHandle tlh(current); > > Why was this added? This is explained in the PR description. Do you think, a just comment is needed or this has to be separated from this fix? > src/hotspot/share/prims/jvmtiThreadState.cpp line 531: > >> 529: >> 530: // JvmtiThreadState objects for virtual thread filtered events enabled globally >> 531: // must be created eagerly if the interp_only_mode is enabled. Otherwise, > > This sentence is hard to read. How about: > > "If interp_only_mode is enabled then we must eagerly create JvmtiThreadState objects for globally enabled virtual thread filtered events." Okay, thanks. The suggestion is taken. > src/hotspot/share/prims/jvmtiThreadState.cpp line 579: > >> 577: VTMS_mount_end(vthread); >> 578: if (JvmtiExport::can_support_virtual_threads() && >> 579: JvmtiExport::should_post_vthread_mount()) { > > It seems odd that "can_support" can be false when "should_post" is true. I would think that "should_post" would always be false when "can_support" is false, and therefore there would be no need to check "can_support". Right, thanks. It is why this check was missed in the first place. Will undo this change. > src/hotspot/share/prims/jvmtiThreadState.hpp line 234: > >> 232: inline void set_head_env_thread_state(JvmtiEnvThreadState* ets); >> 233: >> 234: static bool _seen_interp_only_mode; // needed for optimization > > Say what the flag represents, not why we have it. Thank you for looking at this PR! Okay, thanks. Will do. > src/hotspot/share/prims/jvmtiThreadState.hpp line 257: > >> 255: // JvmtiThreadState objects for virtual thread filtered events enabled globally >> 256: // must be created eagerly if the interp_only_mode is enabled. Otherwise, >> 257: // it is an important optimization to create JvmtiThreadState objects lazily. > > No need for this comment here. It is already at the call site, which is where it belongs. Instead the comment here should say what this API does (return true if any thread has entered interp_only_mode at any point during the JVMs execution). Thanks, good suggestion. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396725938 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396728778 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396728297 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396727211 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396729089 From cjplummer at openjdk.org Fri Nov 17 05:58:31 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 17 Nov 2023 05:58:31 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 In-Reply-To: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> References: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> Message-ID: On Fri, 17 Nov 2023 05:38:28 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiEventController.cpp line 372: >> >>> 370: return; // EnterInterpOnlyModeClosure will be executed right after mount. >>> 371: } >>> 372: ThreadsListHandle tlh(current); >> >> Why was this added? > > This is explained in the PR description. > Do you think, a just comment is needed or this has to be separated from this fix? I see the PR comment, but I don't really understand it. Is this to force some sort of early initialization to avoid a race later on? It just seems odd to create the tlh, but never use it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396735115 From dholmes at openjdk.org Fri Nov 17 06:00:34 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 17 Nov 2023 06:00:34 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v3] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Mon, 13 Nov 2023 10:11:26 GMT, Long Yang wrote: >> I would like to fix this. >> >> Create 4096 threads, and the stack depth of each thread is 256. >> After running jmx.dumpAllThreads(true, true), the RSS reaches 5.3GiB. >> After optimization, the RSS is 250MiB. >> >> I would appreciate it if anyone could review this. >> >> --------- >> update >> >> If the number of `threads` and `stack depth` are relatively large, we need to apply for more space in `ResourceArea` during the execution of `jmx.dumpAllThreads(true, true)`. >> >> The reason is that `VM_ThreadDump::doit` creates `vframe` for each `frame` of each `thread`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/services/threadService.cpp#L704 >> sizeof `vframe` is 4808 (bytes), and sizeof `compiledVFrame` is 4824 (bytes), mainly because the `xmm registers` in `RegisterMap` are relatively large. Assuming there are 4096 `threads` and each `thread` has 256 `frames`, the memory required is 4096 * 256 * 4824 = 4.7GiB? >> >> These memories of all threads are released once by the the initial `ResourceMark` of `VM_ThreadDump::doit`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/vmOperations.cpp#L269 >> My solution is to add a `ResourceMark` for each thread. > > Long Yang has updated the pull request incrementally with one additional commit since the last revision: > > Use VMThread::vm_thread() to avoid the need to call Thread::current() Marked as reviewed by dholmes (Reviewer). Our testing passed tiers 1-5. Thanks ------------- PR Review: https://git.openjdk.org/jdk/pull/16598#pullrequestreview-1736091329 PR Comment: https://git.openjdk.org/jdk/pull/16598#issuecomment-1815790163 From duke at openjdk.org Fri Nov 17 07:09:36 2023 From: duke at openjdk.org (Long Yang) Date: Fri, 17 Nov 2023 07:09:36 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v3] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Mon, 13 Nov 2023 10:11:26 GMT, Long Yang wrote: >> I would like to fix this. >> >> Create 4096 threads, and the stack depth of each thread is 256. >> After running jmx.dumpAllThreads(true, true), the RSS reaches 5.3GiB. >> After optimization, the RSS is 250MiB. >> >> I would appreciate it if anyone could review this. >> >> --------- >> update >> >> If the number of `threads` and `stack depth` are relatively large, we need to apply for more space in `ResourceArea` during the execution of `jmx.dumpAllThreads(true, true)`. >> >> The reason is that `VM_ThreadDump::doit` creates `vframe` for each `frame` of each `thread`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/services/threadService.cpp#L704 >> sizeof `vframe` is 4808 (bytes), and sizeof `compiledVFrame` is 4824 (bytes), mainly because the `xmm registers` in `RegisterMap` are relatively large. Assuming there are 4096 `threads` and each `thread` has 256 `frames`, the memory required is 4096 * 256 * 4824 = 4.7GiB? >> >> These memories of all threads are released once by the the initial `ResourceMark` of `VM_ThreadDump::doit`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/vmOperations.cpp#L269 >> My solution is to add a `ResourceMark` for each thread. > > Long Yang has updated the pull request incrementally with one additional commit since the last revision: > > Use VMThread::vm_thread() to avoid the need to call Thread::current() Could I get another review ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16598#issuecomment-1815842813 From duke at openjdk.org Fri Nov 17 07:09:34 2023 From: duke at openjdk.org (Long Yang) Date: Fri, 17 Nov 2023 07:09:34 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v3] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Fri, 17 Nov 2023 05:57:37 GMT, David Holmes wrote: >> Long Yang has updated the pull request incrementally with one additional commit since the last revision: >> >> Use VMThread::vm_thread() to avoid the need to call Thread::current() > > Our testing passed tiers 1-5. Thanks @dholmes-ora David Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16598#issuecomment-1815842644 From sspitsyn at openjdk.org Fri Nov 17 07:26:30 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 07:26:30 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 In-Reply-To: References: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> Message-ID: On Fri, 17 Nov 2023 05:55:44 GMT, Chris Plummer wrote: >> This is explained in the PR description. >> Do you think, a just comment is needed or this has to be separated from this fix? > > I see the PR comment, but I don't really understand it. Is this to force some sort of early initialization to avoid a race later on? It just seems odd to create the tlh, but never use it. The `tlh` is used to protect any existing at this point JavaThread's from being terminated while it is set. My understanding is that there is no need to iterate over all threads in the list to get this protection. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396795605 From sspitsyn at openjdk.org Fri Nov 17 07:31:33 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 07:31:33 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 In-Reply-To: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> References: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> Message-ID: On Fri, 17 Nov 2023 05:43:21 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiThreadState.cpp line 531: >> >>> 529: >>> 530: // JvmtiThreadState objects for virtual thread filtered events enabled globally >>> 531: // must be created eagerly if the interp_only_mode is enabled. Otherwise, >> >> This sentence is hard to read. How about: >> >> "If interp_only_mode is enabled then we must eagerly create JvmtiThreadState objects for globally enabled virtual thread filtered events." > > Okay, thanks. The suggestion is taken. Fixed. >> src/hotspot/share/prims/jvmtiThreadState.cpp line 579: >> >>> 577: VTMS_mount_end(vthread); >>> 578: if (JvmtiExport::can_support_virtual_threads() && >>> 579: JvmtiExport::should_post_vthread_mount()) { >> >> It seems odd that "can_support" can be false when "should_post" is true. I would think that "should_post" would always be false when "can_support" is false, and therefore there would be no need to check "can_support". > > Right, thanks. It is why this check was missed in the first place. Will undo this change. Fixed. >> src/hotspot/share/prims/jvmtiThreadState.hpp line 234: >> >>> 232: inline void set_head_env_thread_state(JvmtiEnvThreadState* ets); >>> 233: >>> 234: static bool _seen_interp_only_mode; // needed for optimization >> >> Say what the flag represents, not why we have it. > > Thank you for looking at this PR! > Okay, thanks. Will do. Fixed. >> src/hotspot/share/prims/jvmtiThreadState.hpp line 257: >> >>> 255: // JvmtiThreadState objects for virtual thread filtered events enabled globally >>> 256: // must be created eagerly if the interp_only_mode is enabled. Otherwise, >>> 257: // it is an important optimization to create JvmtiThreadState objects lazily. >> >> No need for this comment here. It is already at the call site, which is where it belongs. Instead the comment here should say what this API does (return true if any thread has entered interp_only_mode at any point during the JVMs execution). > > Thanks, good suggestion. Fixed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396798836 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396801201 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396798590 PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1396799894 From sspitsyn at openjdk.org Fri Nov 17 07:34:44 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 07:34:44 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v2] In-Reply-To: References: Message-ID: <5EZl52o4_wJkz6SVSxWkPGVdc20b2TWgb2KEWT2Np8Q=.3d52eccc-2b72-4a42-9f5d-5c2fd54475bf@github.com> > This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. > However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. > It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. > It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. > > Additionally, the fix includes the following related changes: > - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: > > JvmtiThreadState* state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > MutexLocker mu(JvmtiThreadState_lock); > state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > JvmtiEventController::enter_interp_only_mode(); > } > } > > > - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. > - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. > > Performance mesurements: > - Without this fix the test provided by the bug submitter gives execution numbers: > - no ClassLoad events enabled: 3251 ms > - ClassLoad events are enabled: 40534 ms > > - With the fix: > - no ClassLoad events enabled: 3270 ms > - ClassLoad events are enabled: 3385 ms > > Testing: > - Ran mach5 tiers 1-6, no regressions are noticed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: (1) removed unneeded check for can_support_virtual_threads; (2) corrected some comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16686/files - new: https://git.openjdk.org/jdk/pull/16686/files/c5ba2cb0..2582ae3d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16686&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16686&range=00-01 Stats: 26 lines in 2 files changed: 0 ins; 8 del; 18 mod Patch: https://git.openjdk.org/jdk/pull/16686.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16686/head:pull/16686 PR: https://git.openjdk.org/jdk/pull/16686 From stefank at openjdk.org Fri Nov 17 08:13:37 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 17 Nov 2023 08:13:37 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v2] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Mon, 13 Nov 2023 02:01:09 GMT, David Holmes wrote: >> Long Yang has updated the pull request incrementally with one additional commit since the last revision: >> >> move ResourceMark to before start_vf > > src/hotspot/share/services/threadService.cpp line 698: > >> 696: RegisterMap::ProcessFrames::include, >> 697: RegisterMap::WalkContinuation::skip); >> 698: ResourceMark rm; > > Nit: Use `rm(VMThread::vm_thread());` to avoid the need to call `Thread::current()`. FWIW, I don't see the appeal to micro optimize away one call to Thread::current() in a function that performs a massive amount of work. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16598#discussion_r1396841591 From stefank at openjdk.org Fri Nov 17 08:17:34 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 17 Nov 2023 08:17:34 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v3] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Mon, 13 Nov 2023 10:11:26 GMT, Long Yang wrote: >> I would like to fix this. >> >> Create 4096 threads, and the stack depth of each thread is 256. >> After running jmx.dumpAllThreads(true, true), the RSS reaches 5.3GiB. >> After optimization, the RSS is 250MiB. >> >> I would appreciate it if anyone could review this. >> >> --------- >> update >> >> If the number of `threads` and `stack depth` are relatively large, we need to apply for more space in `ResourceArea` during the execution of `jmx.dumpAllThreads(true, true)`. >> >> The reason is that `VM_ThreadDump::doit` creates `vframe` for each `frame` of each `thread`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/services/threadService.cpp#L704 >> sizeof `vframe` is 4808 (bytes), and sizeof `compiledVFrame` is 4824 (bytes), mainly because the `xmm registers` in `RegisterMap` are relatively large. Assuming there are 4096 `threads` and each `thread` has 256 `frames`, the memory required is 4096 * 256 * 4824 = 4.7GiB? >> >> These memories of all threads are released once by the the initial `ResourceMark` of `VM_ThreadDump::doit`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/vmOperations.cpp#L269 >> My solution is to add a `ResourceMark` for each thread. > > Long Yang has updated the pull request incrementally with one additional commit since the last revision: > > Use VMThread::vm_thread() to avoid the need to call Thread::current() Marked as reviewed by stefank (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16598#pullrequestreview-1736277454 From duke at openjdk.org Fri Nov 17 08:46:46 2023 From: duke at openjdk.org (Long Yang) Date: Fri, 17 Nov 2023 08:46:46 GMT Subject: Integrated: 8319876: Reduce memory consumption of VM_ThreadDump::doit In-Reply-To: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: <6WhQKnnaVOhKTnDWLyUryIdDQWdIUQpN1SAILNs06UM=.5207eb2f-6f03-41f9-b8d8-afcad95e1058@github.com> On Fri, 10 Nov 2023 09:13:06 GMT, Long Yang wrote: > I would like to fix this. > > Create 4096 threads, and the stack depth of each thread is 256. > After running jmx.dumpAllThreads(true, true), the RSS reaches 5.3GiB. > After optimization, the RSS is 250MiB. > > I would appreciate it if anyone could review this. > > --------- > update > > If the number of `threads` and `stack depth` are relatively large, we need to apply for more space in `ResourceArea` during the execution of `jmx.dumpAllThreads(true, true)`. > > The reason is that `VM_ThreadDump::doit` creates `vframe` for each `frame` of each `thread`. > https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/services/threadService.cpp#L704 > sizeof `vframe` is 4808 (bytes), and sizeof `compiledVFrame` is 4824 (bytes), mainly because the `xmm registers` in `RegisterMap` are relatively large. Assuming there are 4096 `threads` and each `thread` has 256 `frames`, the memory required is 4096 * 256 * 4824 = 4.7GiB? > > These memories of all threads are released once by the the initial `ResourceMark` of `VM_ThreadDump::doit`. > https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/vmOperations.cpp#L269 > My solution is to add a `ResourceMark` for each thread. This pull request has now been integrated. Changeset: 8ec6b8de Author: yibo.yl Committer: Denghui Dong URL: https://git.openjdk.org/jdk/commit/8ec6b8de3bb3d7aeebdcb45d761b18cce3bab75e Stats: 3 lines in 2 files changed: 0 ins; 2 del; 1 mod 8319876: Reduce memory consumption of VM_ThreadDump::doit Reviewed-by: dholmes, stefank ------------- PR: https://git.openjdk.org/jdk/pull/16598 From duke at openjdk.org Fri Nov 17 08:46:44 2023 From: duke at openjdk.org (Long Yang) Date: Fri, 17 Nov 2023 08:46:44 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v3] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Mon, 13 Nov 2023 10:11:26 GMT, Long Yang wrote: >> I would like to fix this. >> >> Create 4096 threads, and the stack depth of each thread is 256. >> After running jmx.dumpAllThreads(true, true), the RSS reaches 5.3GiB. >> After optimization, the RSS is 250MiB. >> >> I would appreciate it if anyone could review this. >> >> --------- >> update >> >> If the number of `threads` and `stack depth` are relatively large, we need to apply for more space in `ResourceArea` during the execution of `jmx.dumpAllThreads(true, true)`. >> >> The reason is that `VM_ThreadDump::doit` creates `vframe` for each `frame` of each `thread`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/services/threadService.cpp#L704 >> sizeof `vframe` is 4808 (bytes), and sizeof `compiledVFrame` is 4824 (bytes), mainly because the `xmm registers` in `RegisterMap` are relatively large. Assuming there are 4096 `threads` and each `thread` has 256 `frames`, the memory required is 4096 * 256 * 4824 = 4.7GiB? >> >> These memories of all threads are released once by the the initial `ResourceMark` of `VM_ThreadDump::doit`. >> https://github.com/openjdk/jdk/blob/fe0ccdf5f8a5559a608d2e2cd2b6aecbe245c5ec/src/hotspot/share/runtime/vmOperations.cpp#L269 >> My solution is to add a `ResourceMark` for each thread. > > Long Yang has updated the pull request incrementally with one additional commit since the last revision: > > Use VMThread::vm_thread() to avoid the need to call Thread::current() Thank you all ~ ------------- PR Comment: https://git.openjdk.org/jdk/pull/16598#issuecomment-1815950109 From sspitsyn at openjdk.org Fri Nov 17 10:30:59 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 10:30:59 GMT Subject: RFR: 8320239: add dynamic switch for JvmtiVTMSTransitionDisabler sync protocol [v2] In-Reply-To: References: Message-ID: > This is an update for a performance/scalability enhancement. > > The `JvmtiVTMSTransitionDisabler`sync protocol is on a performance critical path of the virtual threads mount state transitions (VTMS transitions). It has a noticeable performance overhead (about 10%) which contributes to the combined JVMTI performance overhead when Java apps are executed with loaded JVMTI agents. > > Please, also see another/related performance issue which contributes around 70% of total performance overhead: > [8308614](https://bugs.openjdk.org/browse/JDK-8308614): Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 > > Testing: > - Ran mach5 tiers 1-6 with no regressions noticed. Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: make new fields volatile, use Atomic for access/update ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16688/files - new: https://git.openjdk.org/jdk/pull/16688/files/bf093127..a81218fd Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16688&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16688&range=00-01 Stats: 15 lines in 2 files changed: 3 ins; 0 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/16688.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16688/head:pull/16688 PR: https://git.openjdk.org/jdk/pull/16688 From azafari at openjdk.org Fri Nov 17 13:22:36 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Fri, 17 Nov 2023 13:22:36 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 09:38:25 GMT, Afshin Zafari wrote: >>> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). >> >> Not touched since 2008 suggests to me there might not be a rush to make the change as proposed, and instead take >> the (I think small) additional time to do the better thing, e.g. the unary-predicate suggestion made by several folks. > > @kimbarrett , @dholmes-ora , @merykitty > Is there any comment on this PR? > @afshin-zafari I will leave it to other to (re-) review the latest changes. I don't grok this template stuff enough to pass judgement. Thank you very much @dholmes-ora, for your comments and discussions. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1816417460 From azafari at openjdk.org Fri Nov 17 13:22:38 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Fri, 17 Nov 2023 13:22:38 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v8] In-Reply-To: <648SrHxCX6_kRX7cmxyGurxOWecLTWUw0_C79J_okbo=.473eaa8c-7dfc-4aa0-839d-bb580cc9d312@github.com> References: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> <648SrHxCX6_kRX7cmxyGurxOWecLTWUw0_C79J_okbo=.473eaa8c-7dfc-4aa0-839d-bb580cc9d312@github.com> Message-ID: On Thu, 16 Nov 2023 06:46:48 GMT, Quan Anh Mai wrote: >> Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: >> >> function pointer is replaced with template Functor. > > src/hotspot/share/utilities/growableArray.hpp line 213: > >> 211: >> 212: template >> 213: int find(T* token, F f) const { > > Should be > > template > int find(F f) const { > for (int i = 0; i < _len; i++) { > if (f(_data[i]) { > return i; > } > } > return -1; > } We need `token` to find it in the array, don't we? All the invocations pass such a function with two parameters. The change here needs all invocations to be changed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15418#discussion_r1397293350 From stefank at openjdk.org Fri Nov 17 13:34:36 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 17 Nov 2023 13:34:36 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 13:20:22 GMT, Afshin Zafari wrote: >> @kimbarrett , @dholmes-ora , @merykitty >> Is there any comment on this PR? > >> @afshin-zafari I will leave it to other to (re-) review the latest changes. I don't grok this template stuff enough to pass judgement. > > Thank you very much @dholmes-ora, for your comments and discussions. @afshin-zafari I think you misunderstand the feedback given. The suggestions from many of us has been that you should change the functions to accept lambdas / template-typed functions instead. https://github.com/openjdk/jdk/pull/15418#discussion_r1305389552 https://github.com/openjdk/jdk/pull/15418#discussion_r1375386257 https://github.com/openjdk/jdk/pull/15418#discussion_r1376940244 https://github.com/openjdk/jdk/pull/15418#discussion_r1395219935 ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1816438451 From cjplummer at openjdk.org Fri Nov 17 18:55:29 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Fri, 17 Nov 2023 18:55:29 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v2] In-Reply-To: References: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> Message-ID: <_7pFLcmNRjtnifuhYSMmawqAGiqURduxz9ce1splCNc=.4f8d5376-cc8f-4798-a236-abbce62f5027@github.com> On Fri, 17 Nov 2023 07:23:46 GMT, Serguei Spitsyn wrote: >> I see the PR comment, but I don't really understand it. Is this to force some sort of early initialization to avoid a race later on? It just seems odd to create the tlh, but never use it. > > The `tlh` is used to protect any existing at this point JavaThread's from being terminated while it is set. > My understanding is that there is no need to iterate over all threads in the list to get this protection. Ok. I comment indicating that purpose would be useful. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1397741421 From sspitsyn at openjdk.org Fri Nov 17 20:29:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 20:29:11 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: References: Message-ID: > This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. > However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. > It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. > It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. > > Additionally, the fix includes the following related changes: > - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: > > JvmtiThreadState* state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > MutexLocker mu(JvmtiThreadState_lock); > state = thread->jvmti_thread_state(); > if (state != nullptr && state->is_pending_interp_only_mode()) { > JvmtiEventController::enter_interp_only_mode(); > } > } > > > - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. > - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. > > Performance mesurements: > - Without this fix the test provided by the bug submitter gives execution numbers: > - no ClassLoad events enabled: 3251 ms > - ClassLoad events are enabled: 40534 ms > > - With the fix: > - no ClassLoad events enabled: 3270 ms > - ClassLoad events are enabled: 3385 ms > > Testing: > - Ran mach5 tiers 1-6, no regressions are noticed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: add comment for new ThreadsListHandle use ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16686/files - new: https://git.openjdk.org/jdk/pull/16686/files/2582ae3d..de36957a Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16686&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16686&range=01-02 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16686.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16686/head:pull/16686 PR: https://git.openjdk.org/jdk/pull/16686 From sspitsyn at openjdk.org Fri Nov 17 20:29:11 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Fri, 17 Nov 2023 20:29:11 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: <_7pFLcmNRjtnifuhYSMmawqAGiqURduxz9ce1splCNc=.4f8d5376-cc8f-4798-a236-abbce62f5027@github.com> References: <65piquQpnXvDvmbpt-U_EtxYEe7zu8yRCp39ZDA6rZA=.336ea1e4-a339-4a38-9c28-7a2cd1fd2c31@github.com> <_7pFLcmNRjtnifuhYSMmawqAGiqURduxz9ce1splCNc=.4f8d5376-cc8f-4798-a236-abbce62f5027@github.com> Message-ID: On Fri, 17 Nov 2023 18:52:20 GMT, Chris Plummer wrote: >> The `tlh` is used to protect any existing at this point JavaThread's from being terminated while it is set. >> My understanding is that there is no need to iterate over all threads in the list to get this protection. > > Ok. A comment indicating that purpose would be useful. Okay, thanks! I've added a comment: // Protects any existing JavaThread's from being terminated while it is set. // The FJP carrier thread compensating mechanism can create carrier threads concurrently. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1397847729 From dcubed at openjdk.org Fri Nov 17 21:21:33 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Fri, 17 Nov 2023 21:21:33 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 20:29:11 GMT, Serguei Spitsyn wrote: >> This is a fix of a performance/scalability related issue. The `JvmtiThreadState` objects for virtual thread filtered events enabled globally are created eagerly because it is needed when the `interp_only_mode` is enabled. Otherwise, some events which are generated in `interp_only_mode` from the debugging version of interpreter chunks can be missed. >> However, it has to be okay to avoid eager creation of these object if no `interp_only_mode` has ever been requested. >> It seems to be an extremely important optimization to create JvmtiThreadState objects lazily in such cases. >> It is done by introducing the flag `JvmtiThreadState::_seen_interp_only_mode` which indicates when the `JvmtiThreadState` objects have to be created eagerly. >> >> Additionally, the fix includes the following related changes: >> - Use condition double checking idiom for `MutexLocker mu(JvmtiThreadState_lock)` in the function `JvmtiVTMSTransitionDisabler::VTMS_mount_end` which is on a performance-critical path and looks like this: >> >> JvmtiThreadState* state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> MutexLocker mu(JvmtiThreadState_lock); >> state = thread->jvmti_thread_state(); >> if (state != nullptr && state->is_pending_interp_only_mode()) { >> JvmtiEventController::enter_interp_only_mode(); >> } >> } >> >> >> - Add extra check of `JvmtiExport::can_support_virtual_threads()` when virtual thread mount and unmount are posted. >> - Minor: Added a `ThreadsListHandle` to the `JvmtiEventControllerPrivate::enter_interp_only_mode`. It is needed because of the dynamic creation of compensating carrier threads which is racy for JVMTI `SetEventNotificationMode` implementation. >> >> Performance mesurements: >> - Without this fix the test provided by the bug submitter gives execution numbers: >> - no ClassLoad events enabled: 3251 ms >> - ClassLoad events are enabled: 40534 ms >> >> - With the fix: >> - no ClassLoad events enabled: 3270 ms >> - ClassLoad events are enabled: 3385 ms >> >> Testing: >> - Ran mach5 tiers 1-6, no regressions are noticed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: add comment for new ThreadsListHandle use src/hotspot/share/prims/jvmtiEventController.cpp line 374: > 372: // Protects any existing JavaThread's from being terminated while it is set. > 373: // The FJP carrier thread compensating mechanism can create carrier threads concurrently. > 374: ThreadsListHandle tlh(current); A ThreadsListHandle does not prevent a JavaThread from being terminated. It prevents a JavaThread from exiting and being freed. The JavaThread is able to set the terminated state on itself, but will not be able to complete exiting while it is on a ThreadsListHandle. There is a subtle difference. There's a `target` JavaThread that is fetched from a `JvmtiThreadState` object and that `target` JavaThread is only protected by this `tlh` if `target` is included in the ThreadsList that was captured by this `tlh`. In all likelihood, there should be a ThreadsListHandle farther up the stack that's protecting the JavaThread from which the `JvmtiThreadState` object was extracted and passed to this function. As for carrier threads, if they are created _after_ this `tlh` was created, then this `tlh` cannot protect them because they won't be on this `tlh`'s ThreadsList. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1397897633 From duke at openjdk.org Fri Nov 17 22:36:39 2023 From: duke at openjdk.org (duke) Date: Fri, 17 Nov 2023 22:36:39 GMT Subject: Withdrawn: 8314029: Add file name parameter to Compiler.perfmap In-Reply-To: References: Message-ID: On Thu, 21 Sep 2023 20:43:56 GMT, Yi-Fan Tsai wrote: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an option for specifying a file name. > > The help message of Compiler.perfmap: > > Compiler.perfmap > Write map file for Linux perf tool. > > Impact: Low > > Syntax : Compiler.perfmap [options] > > Options: (options must be specified using the or = syntax) > filename : [optional] Name of the map file (STRING, no default value) This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/15871 From coleenp at openjdk.org Fri Nov 17 22:56:31 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Fri, 17 Nov 2023 22:56:31 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 17:56:09 GMT, Jaroslav Bachorik wrote: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ src/hotspot/share/oops/instanceKlass.cpp line 541: > 539: // The previous version will point to them so they're not totally dangling > 540: assert (!method->on_stack(), "shouldn't be called with methods on stack"); > 541: // Do the pointer maintenance before releasing the metadata, but not for incomplete methods I'm confused by what you mean by method holder, which I think of as methodHandle. Or InstanceKlass is the holder of the methods. Maybe this should be more explicit that it's talking about clearing any associated jmethodIDs. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1397970676 From jbachorik at openjdk.org Sat Nov 18 00:26:29 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Sat, 18 Nov 2023 00:26:29 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 22:53:58 GMT, Coleen Phillimore wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > src/hotspot/share/oops/instanceKlass.cpp line 541: > >> 539: // The previous version will point to them so they're not totally dangling >> 540: assert (!method->on_stack(), "shouldn't be called with methods on stack"); >> 541: // Do the pointer maintenance before releasing the metadata, but not for incomplete methods > > I'm confused by what you mean by method holder, which I think of as methodHandle. Or InstanceKlass is the holder of the methods. Maybe this should be more explicit that it's talking about clearing any associated jmethodIDs. The method holder is an `InstanceKlass` object which can be retrieved as `method->method_holder()` (I apologize if I am using not completely correct terms - this is what I grokked from the sources). And incomplete methods created by the `ClassParser` from the class data stream will not have the link to that `InstanceKlass` set up if the `ClassParser` is already having its `_klass` field set to a non-null value. If we are talking about clearing any jmetbodIDs associated with an `InstanceKlass` instance it is not really possible for old method versions because only the current `InstanceKlass` version has the jmethodID cache associated with it and it contains jmethodIDs pointing to bot the old and current methods. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1398009493 From amenkov at openjdk.org Sat Nov 18 01:40:35 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Sat, 18 Nov 2023 01:40:35 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v8] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 05:37:46 GMT, Serguei Spitsyn wrote: >> The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. >> At the low level, the JVMTI code supporting platform and virtual threads still can be different. >> This implementation is based on the `JvmtiVTMSTransitionDisabler` class. >> >> The internal API includes two new classes: >> - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` >> >> The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. >> >> The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: >> - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` >> >> To get the test results clean, the update also fixes the test issue: >> [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" >> >> Testing: >> - the mach5 tiers 1-6 are all passed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: add jdk_internal_vm_Continuation::done(cont) check to JvmtiEnvBase::is_vthread_alive src/hotspot/share/prims/jvmtiEnvBase.cpp line 631: > 629: return !jdk_internal_vm_Continuation::done(cont) && > 630: java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::NEW && > 631: java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::TERMINATED; AFAIU `jdk_internal_vm_Continuation::done(cont)` is correct check that vthread is terminated and works for both mounted and unmounted vthreads. Then `java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::TERMINATED` check is not needed ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1398029697 From amenkov at openjdk.org Sat Nov 18 02:32:30 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Sat, 18 Nov 2023 02:32:30 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v8] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 05:37:46 GMT, Serguei Spitsyn wrote: >> The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. >> At the low level, the JVMTI code supporting platform and virtual threads still can be different. >> This implementation is based on the `JvmtiVTMSTransitionDisabler` class. >> >> The internal API includes two new classes: >> - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` >> >> The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. >> >> The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: >> - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` >> >> To get the test results clean, the update also fixes the test issue: >> [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" >> >> Testing: >> - the mach5 tiers 1-6 are all passed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: add jdk_internal_vm_Continuation::done(cont) check to JvmtiEnvBase::is_vthread_alive src/hotspot/share/prims/jvmtiEnvBase.cpp line 1989: > 1987: } else { > 1988: Handshake::execute(hs_cl, tlh, target_jt); // delegate to Handshake implementation > 1989: } Every implementation of JvmtiUnitedHandshakeClosure has to check if the target thread is virtual and call do_vthread manually. I'd suggest to handle this by proxy class, something like Suggestion: class Adapter : public HandshakeClosure { JvmtiUnitedHandshakeClosure* _hs_cl; Handle _target_h; public: Adapter(JvmtiUnitedHandshakeClosure* hs_cl, Handle target_h) : HandshakeClosure(hs_cl->name()), _hs_cl(hs_cl), _target_h(target_h) {} virtual void do_thread(Thread* thread) { if (java_lang_VirtualThread::is_instance(_target_h())) { // virtual thread _hs_cl->do_vthread(_target_h); } else { _hs_cl->do_thread(target); } } } adapter(hs_cl, target_h); if (self) { // target thread is current adapter.do_thread(target_jt); // execute handshake closure callback on current thread directly } else { Handshake::execute(&adapter, tlh, target_jt); // delegate to Handshake implementation } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1398042638 From elon.azoulay at gmail.com Sat Nov 18 05:20:08 2023 From: elon.azoulay at gmail.com (Elon Azoulay) Date: Fri, 17 Nov 2023 21:20:08 -0800 Subject: [External] : Re: Proposal: Add overwrite heapdump flag to java In-Reply-To: References: <083b1eef-c7b1-4db0-0096-db8d37e690d5@oracle.com> <5e497f58-2e38-38b9-acf2-adf1a99d5e91@oracle.com> Message-ID: Hi, I wanted to know what I can do to move https://github.com/openjdk/jdk/pull/13276 along. There was advice to create a CSR request, in order to do that there needs to be a jira but I don't think I have permissions to create one. Thanks for all the reviews, let me know what I can do, happy to follow your advice! On Thu, May 25, 2023 at 11:39?PM Kevin Walls wrote: > Hi, > > (I put this in the PR, but maybe drafts don?t get updates put on the > mailing list?) > > HeapDumpOverwrite sounds quite general, there are different ways of heap > dumping and this doesn't affect all of them. This could be > HeapDumpOnOutOfMemoryErrorOverwrite, which is a long option even for us 8-) > so could be abbreviated perhaps, BUT that option might not be the way to do > it? > > Is it really that all heap dumps should be permitted to an existing file, > IF that file is a FIFO? From the email I think that's the problem? ..and if > so, we have: > > 8267666: Add option to jcmd GC.heap_dump to use existing file > https://bugs.openjdk.org/browse/JDK-8267666 > > So if jcmd to create a dump can use an existing file, -XX:OnError= can > trigger such a jcmd? > > i.e. Maybe there is a way of doing what you want. A code change could > still be required to make this easier for heap dumps on out of memory. > Writing a JBS bug first is useful to define the problem. > > Thanks > Kevin > > > > > > > > *From:* jdk-dev *On Behalf Of *Elon Azoulay > *Sent:* 25 May 2023 21:21 > *To:* Daniel Daugherty > *Cc:* serviceability-dev at openjdk.org; jdk-dev at openjdk.org > *Subject:* Re: [External] : Re: Proposal: Add overwrite heapdump flag to > java > > > > Sounds good, thanks so much! > > > > On Thu, May 25, 2023 at 12:13?PM wrote: > > Pointing the older thread to this newer incarnation would be a good idea. > > Dan > > On 5/25/23 3:11 PM, Elon Azoulay wrote: > > Hi Dan, > > That's great! Should I follow up in that thread as well? > > > > > > On Thu, May 25, 2023 at 12:04?PM wrote: > > Greetings, > > A similar request came up on the hotspot-dev alias back in 2022.11: > > https://mail.openjdk.org/pipermail/hotspot-dev/2022-November/066956.html > > Dan > > > On 5/25/23 1:48 AM, David Holmes wrote: > > Hi Elon, > > > > I would suggest taking this up on serviceability-dev - cc'd. > > > > On 25/05/2023 3:46 am, Elon Azoulay wrote: > >> Hi, > >> I submitted a pull request > >> > > to expose the overwrite > >> flag to HeapDumpOnOutOfMemoryError. > >> The flag is already exposed to jcmd as -overwrite.This is to > >> facilitate creating a heapdump within a container native environment. > >> We use this internally to dump the heap dump to a fifo so that we > >> could read it in a separate container.Let me know what needs to be > >> done in terms of creating an issue and getting my pr merged. > > > > Have you gone through the guide: > > > > https://openjdk.org/guide/ > > > > ? > > > > You will need a JBS issue and also a CSR request as this proposes to > > add a new manageable product flag. > > > > Cheers, > > David > > > >> I am a new contributor and would be happy to contribute more! > >> > >> Cheers, > >> > >> Elon > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sspitsyn at openjdk.org Sat Nov 18 12:24:57 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 18 Nov 2023 12:24:57 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: References: Message-ID: On Fri, 17 Nov 2023 21:19:05 GMT, Daniel D. Daugherty wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: add comment for new ThreadsListHandle use > > src/hotspot/share/prims/jvmtiEventController.cpp line 374: > >> 372: // Protects any existing JavaThread's from being terminated while it is set. >> 373: // The FJP carrier thread compensating mechanism can create carrier threads concurrently. >> 374: ThreadsListHandle tlh(current); > > A ThreadsListHandle does not prevent a JavaThread from being terminated. It > prevents a JavaThread from exiting and being freed. The JavaThread is able to > set the terminated state on itself, but will not be able to complete exiting while > it is on a ThreadsListHandle. There is a subtle difference. > > There's a `target` JavaThread that is fetched from a `JvmtiThreadState` object > and that `target` JavaThread is only protected by this `tlh` if `target` is included > in the ThreadsList that was captured by this `tlh`. In all likelihood, there should be > a ThreadsListHandle farther up the stack that's protecting the JavaThread from > which the `JvmtiThreadState` object was extracted and passed to this function. > > As for carrier threads, if they are created _after_ this `tlh` was created, then this > `tlh` cannot protect them because they won't be on this `tlh`'s ThreadsList. Thank you for the comment, Dan! Agreed, the comment needs to be corrected in two aspects. I tried to simplify it but failed to do it correctly. It is interesting that there is a `ThreadsListHandle` farther up the stack but it does not help sometimes. It is observed that a `JavaThread` (of a carrier thread) referenced from the `JvmtiThreadState` object can be just created, so we need a `ThreadsListHandle` to avoid possible asserts. With this fix in place I do not see the asserts fired anymore. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1398200015 From sspitsyn at openjdk.org Sat Nov 18 14:35:30 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 18 Nov 2023 14:35:30 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v8] In-Reply-To: References: Message-ID: On Sat, 18 Nov 2023 02:29:26 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: add jdk_internal_vm_Continuation::done(cont) check to JvmtiEnvBase::is_vthread_alive > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 1989: > >> 1987: } else { >> 1988: Handshake::execute(hs_cl, tlh, target_jt); // delegate to Handshake implementation >> 1989: } > > Every implementation of JvmtiUnitedHandshakeClosure has to check if the target thread is virtual and call do_vthread manually. > I'd suggest to handle this by proxy class, something like > Suggestion: > > class Adapter : public HandshakeClosure { > JvmtiUnitedHandshakeClosure* _hs_cl; > Handle _target_h; > public: > Adapter(JvmtiUnitedHandshakeClosure* hs_cl, Handle target_h) > : HandshakeClosure(hs_cl->name()), _hs_cl(hs_cl), _target_h(target_h) {} > virtual void do_thread(Thread* thread) { > if (java_lang_VirtualThread::is_instance(_target_h())) { // virtual thread > _hs_cl->do_vthread(_target_h); > } else { > _hs_cl->do_thread(target); > } > } > } adapter(hs_cl, target_h); > > if (self) { // target thread is current > adapter.do_thread(target_jt); // execute handshake closure callback on current thread directly > } else { > Handshake::execute(&adapter, tlh, target_jt); // delegate to Handshake implementation > } Thank you for the suggestion! Agreed, this should help to get rid of this duplication/ugliness. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1398218934 From sspitsyn at openjdk.org Sat Nov 18 14:38:30 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sat, 18 Nov 2023 14:38:30 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v8] In-Reply-To: References: Message-ID: On Sat, 18 Nov 2023 01:37:21 GMT, Alex Menkov wrote: >> Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: >> >> review: add jdk_internal_vm_Continuation::done(cont) check to JvmtiEnvBase::is_vthread_alive > > src/hotspot/share/prims/jvmtiEnvBase.cpp line 631: > >> 629: return !jdk_internal_vm_Continuation::done(cont) && >> 630: java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::NEW && >> 631: java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::TERMINATED; > > AFAIU `jdk_internal_vm_Continuation::done(cont)` is correct check that vthread is terminated and works for both mounted and unmounted vthreads. > Then `java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::TERMINATED` check is not needed Good suggestion, thanks! ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1398219250 From sspitsyn at openjdk.org Sun Nov 19 00:05:49 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 19 Nov 2023 00:05:49 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v9] In-Reply-To: References: Message-ID: <7oZKrqEQvbpRfz6jDNyRMU_EznRR_3W6zDmO5U-7mdU=.877f1d6b-eee4-4fa0-a088-80c685e7d320@github.com> > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes two new classes: > - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: added AdapterClosure to get rid of duplication ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/e61d0703..fefeb7f1 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=07-08 Stats: 47 lines in 2 files changed: 20 ins; 22 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From dcubed at openjdk.org Sun Nov 19 03:50:34 2023 From: dcubed at openjdk.org (Daniel D. Daugherty) Date: Sun, 19 Nov 2023 03:50:34 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: References: Message-ID: On Sat, 18 Nov 2023 12:22:10 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiEventController.cpp line 374: >> >>> 372: // Protects any existing JavaThread's from being terminated while it is set. >>> 373: // The FJP carrier thread compensating mechanism can create carrier threads concurrently. >>> 374: ThreadsListHandle tlh(current); >> >> A ThreadsListHandle does not prevent a JavaThread from being terminated. It >> prevents a JavaThread from exiting and being freed. The JavaThread is able to >> set the terminated state on itself, but will not be able to complete exiting while >> it is on a ThreadsListHandle. There is a subtle difference. >> >> There's a `target` JavaThread that is fetched from a `JvmtiThreadState` object >> and that `target` JavaThread is only protected by this `tlh` if `target` is included >> in the ThreadsList that was captured by this `tlh`. In all likelihood, there should be >> a ThreadsListHandle farther up the stack that's protecting the JavaThread from >> which the `JvmtiThreadState` object was extracted and passed to this function. >> >> As for carrier threads, if they are created _after_ this `tlh` was created, then this >> `tlh` cannot protect them because they won't be on this `tlh`'s ThreadsList. > > Thank you for the comment, Dan! > Agreed, the comment needs to be corrected in two aspects. > I tried to simplify it but failed to do it correctly. > It is interesting that there is a `ThreadsListHandle` farther up the stack but it does not help sometimes. > It is observed that a `JavaThread` (of a carrier thread) referenced from the `JvmtiThreadState` object can be just created, so we need a `ThreadsListHandle` to avoid possible asserts. With this fix in place I do not see the asserts fired anymore. @sspitsyn - Please point me at the code where these JavaThreads are newly created? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1398317396 From sspitsyn at openjdk.org Sun Nov 19 05:12:50 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 19 Nov 2023 05:12:50 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v10] In-Reply-To: References: Message-ID: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes two new classes: > - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: review: remove java_lang_VirtualThread::NEW check from is_vthread_alive ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16460/files - new: https://git.openjdk.org/jdk/pull/16460/files/fefeb7f1..9db5a300 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16460&range=08-09 Stats: 3 lines in 1 file changed: 0 ins; 1 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16460.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16460/head:pull/16460 PR: https://git.openjdk.org/jdk/pull/16460 From sspitsyn at openjdk.org Sun Nov 19 05:12:51 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 19 Nov 2023 05:12:51 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v8] In-Reply-To: References: Message-ID: <9fmaSnjImG3zuawScS_8mp_0psnV3t0SOKqQuKS0nnA=.bc8bc845-2b4c-4dc9-8743-cad88ee508c4@github.com> On Sat, 18 Nov 2023 14:35:58 GMT, Serguei Spitsyn wrote: >> src/hotspot/share/prims/jvmtiEnvBase.cpp line 631: >> >>> 629: return !jdk_internal_vm_Continuation::done(cont) && >>> 630: java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::NEW && >>> 631: java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::TERMINATED; >> >> AFAIU `jdk_internal_vm_Continuation::done(cont)` is correct check that vthread is terminated and works for both mounted and unmounted vthreads. >> Then `java_lang_VirtualThread::state(vt) != java_lang_VirtualThread::TERMINATED` check is not needed > > Good suggestion, thanks! Added the fix suggested by Alex. >> src/hotspot/share/prims/jvmtiEnvBase.cpp line 1989: >> >>> 1987: } else { >>> 1988: Handshake::execute(hs_cl, tlh, target_jt); // delegate to Handshake implementation >>> 1989: } >> >> Every implementation of JvmtiUnitedHandshakeClosure has to check if the target thread is virtual and call do_vthread manually. >> I'd suggest to handle this by proxy class, something like >> Suggestion: >> >> class Adapter : public HandshakeClosure { >> JvmtiUnitedHandshakeClosure* _hs_cl; >> Handle _target_h; >> public: >> Adapter(JvmtiUnitedHandshakeClosure* hs_cl, Handle target_h) >> : HandshakeClosure(hs_cl->name()), _hs_cl(hs_cl), _target_h(target_h) {} >> virtual void do_thread(Thread* thread) { >> if (java_lang_VirtualThread::is_instance(_target_h())) { // virtual thread >> _hs_cl->do_vthread(_target_h); >> } else { >> _hs_cl->do_thread(target); >> } >> } >> } adapter(hs_cl, target_h); >> >> if (self) { // target thread is current >> adapter.do_thread(target_jt); // execute handshake closure callback on current thread directly >> } else { >> Handshake::execute(&adapter, tlh, target_jt); // delegate to Handshake implementation >> } > > Thank you for the suggestion! Agreed, this should help to get rid of this duplication/ugliness. Added the fix suggested by Alex. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1398324679 PR Review Comment: https://git.openjdk.org/jdk/pull/16460#discussion_r1398324619 From sspitsyn at openjdk.org Sun Nov 19 09:26:28 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 19 Nov 2023 09:26:28 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: References: Message-ID: <1P_TddTTM1eH75Do2Xq-wBrxXSdh7GzJJlgEBH_dSNo=.94392ab2-12a1-4d1b-9131-6164bbb76e7d@github.com> On Sun, 19 Nov 2023 03:47:45 GMT, Daniel D. Daugherty wrote: >> Thank you for the comment, Dan! >> Agreed, the comment needs to be corrected in two aspects. >> I tried to simplify it but failed to do it correctly. >> It is interesting that there is a `ThreadsListHandle` farther up the stack but it does not help sometimes. >> It is observed that a `JavaThread` (of a carrier thread) referenced from the `JvmtiThreadState` object can be just created, so we need a `ThreadsListHandle` to avoid possible asserts. With this fix in place I do not see the asserts fired anymore. > > @sspitsyn - Please point me at the code where these JavaThreads are newly created? @dcubed-ojdk I don't know FJP implementation well enough to point at the code where it happens. However, I observe that new `JavaThread `is being created between two points of the execution path. - First point is in the `JvmtiEventControllerPrivate::recompute_enabled()` at the line where a `ThreadsListHandle` is set. I've added a trap checking if any `JavaThread` pointed by `state->get_thread()` is not protected by the `tlh`. I can see this trap is not fired (I can't say it has never been fired). - Second point is in the `JvmtiEventControllerPrivate::enter_interp_only_mode()`. If a `ThreadsListHandle` is NOT set then I can observe a `JavaThread` referenced by the state->get_thread() which is not protected by any TLH. It a TLH added into `JvmtiEventControllerPrivate::enter_interp_only_mode()` then this `JavaThread` is observed as protected by TLH. I can provide a full stack trace for this `JavaThread` consisting of two parts: carrier thread and virtual thread frames. The name of carrier thread is `ForkJoinPool-1-worker-1`. The virtual thread is a tested virtual thread. The thread dump looks las below: DBG: enter_interp_only_mode: target: 0x7f93f8043d00 virt: 1 carrier: ForkJoinPool-1-worker-1 DBG: ##### NATIVE stacktrace of JavaThread: 0x7f93f8043d00 ##### Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) j fieldacc02.check(Ljava/lang/Object;)I+0 j fieldacc02.lambda$testVirtualThread$0()V+12 j fieldacc02$$Lambda+0x00007f943b001428.run()V+0 j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base at 22-internal j java.lang.VirtualThread.run(Ljava/lang/Runnable;)V+66 java.base at 22-internal j java.lang.VirtualThread$VThreadContinuation$1.run()V+8 java.base at 22-internal j jdk.internal.vm.Continuation.enter0()V+4 java.base at 22-internal j jdk.internal.vm.Continuation.enter(Ljdk/internal/vm/Continuation;Z)V+1 java.base at 22-internal J 124 jdk.internal.vm.Continuation.enterSpecial(Ljdk/internal/vm/Continuation;ZZ)V java.base at 22-internal (0 bytes) @ 0x00007f94c7cdf744 [0x00007f94c7cdf5e0+0x0000000000000164] j jdk.internal.vm.Continuation.run()V+122 java.base at 22-internal j java.lang.VirtualThread.runContinuation()V+70 java.base at 22-internal j java.lang.VirtualThread$$Lambda+0x00007f943b0496c0.run()V+4 java.base at 22-internal j java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute()Ljava/lang/Void;+4 java.base at 22-internal j java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute()Ljava/lang/Object;+1 java.base at 22-internal j java.util.concurrent.ForkJoinTask$InterruptibleTask.exec()Z+51 java.base at 22-internal j java.util.concurrent.ForkJoinTask.doExec()V+10 java.base at 22-internal j java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Ljava/util/concurrent/ForkJoinTask;Ljava/util/concurrent/ForkJoinPool$WorkQueue;I)V+49 java.base at 22-internal j java.util.concurrent.ForkJoinPool.scan(Ljava/util/concurrent/ForkJoinPool$WorkQueue;JI)J+271 java.base at 22-internal j java.util.concurrent.ForkJoinPool.runWorker(Ljava/util/concurrent/ForkJoinPool$WorkQueue;)V+68 java.base at 22-internal j java.util.concurrent.ForkJoinWorkerThread.run()V+31 java.base at 22-internal v ~StubRoutines::call_stub 0x00007f94c7505d21 V [libjvm.so+0xe7b719] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x4a9 (javaCalls.cpp:415) V [libjvm.so+0xe7bdd5] JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*)+0x345 (javaCalls.cpp:329) V [libjvm.so+0xe7bff6] JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*)+0x76 (javaCalls.cpp:191) V [libjvm.so+0xfd9723] thread_entry(JavaThread*, JavaThread*)+0x93 (jvm.cpp:2937) V [libjvm.so+0xeb06ac] JavaThread::thread_main_inner()+0xcc (javaThread.cpp:720) V [libjvm.so+0x1789496] Thread::call_run()+0xb6 (thread.cpp:220) V [libjvm.so+0x1493b27] thread_native_entry(Thread*)+0x127 (os_linux.cpp:787) The observed `JavaThead` does not look as a garbage because no crashes has ever been observed. Apparently, it has been recently created because it is not protected by the TLH which was set in the `JvmtiEventControllerPrivate::recompute_enabled()`. I guess, it has to be possible to find out where exactly in the FJP code it is created but it will take time. I'm not sure why do you need it though. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1398362375 From sspitsyn at openjdk.org Sun Nov 19 09:44:28 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Sun, 19 Nov 2023 09:44:28 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: <1P_TddTTM1eH75Do2Xq-wBrxXSdh7GzJJlgEBH_dSNo=.94392ab2-12a1-4d1b-9131-6164bbb76e7d@github.com> References: <1P_TddTTM1eH75Do2Xq-wBrxXSdh7GzJJlgEBH_dSNo=.94392ab2-12a1-4d1b-9131-6164bbb76e7d@github.com> Message-ID: On Sun, 19 Nov 2023 09:22:43 GMT, Serguei Spitsyn wrote: >> @sspitsyn - Please point me at the code where these JavaThreads are newly created? > > @dcubed-ojdk > I don't know FJP implementation well enough to point at the code where it happens. However, I observe that new `JavaThread `is being created between two points of the execution path. > - First point is in the `JvmtiEventControllerPrivate::recompute_enabled()` at the line where a `ThreadsListHandle` is set. I've added a trap checking if any `JavaThread` pointed by `state->get_thread()` is not protected by the `tlh`. I can see this trap is not fired (I can't say it has never been fired). > - Second point is in the `JvmtiEventControllerPrivate::enter_interp_only_mode()`. If a `ThreadsListHandle` is NOT set then I can observe a `JavaThread` referenced by the state->get_thread() which is not protected by any TLH. It a TLH added into `JvmtiEventControllerPrivate::enter_interp_only_mode()` then this `JavaThread` is observed as protected by TLH. > I can provide a full stack trace for this `JavaThread` consisting of two parts: carrier thread and virtual thread frames. The name of carrier thread is `ForkJoinPool-1-worker-1`. The virtual thread is a tested virtual thread. > The thread dump looks las below: > > DBG: enter_interp_only_mode: target: 0x7f93f8043d00 virt: 1 carrier: ForkJoinPool-1-worker-1 > DBG: ##### NATIVE stacktrace of JavaThread: 0x7f93f8043d00 ##### > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) > j fieldacc02.check(Ljava/lang/Object;)I+0 > j fieldacc02.lambda$testVirtualThread$0()V+12 > j fieldacc02$$Lambda+0x00007f943b001428.run()V+0 > j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base at 22-internal > j java.lang.VirtualThread.run(Ljava/lang/Runnable;)V+66 java.base at 22-internal > j java.lang.VirtualThread$VThreadContinuation$1.run()V+8 java.base at 22-internal > j jdk.internal.vm.Continuation.enter0()V+4 java.base at 22-internal > j jdk.internal.vm.Continuation.enter(Ljdk/internal/vm/Continuation;Z)V+1 java.base at 22-internal > J 124 jdk.internal.vm.Continuation.enterSpecial(Ljdk/internal/vm/Continuation;ZZ)V java.base at 22-internal (0 bytes) @ 0x00007f94c7cdf744 [0x00007f94c7cdf5e0+0x0000000000000164] > j jdk.internal.vm.Continuation.run()V+122 java.base at 22-internal > j java.lang.VirtualThread.runContinuation()V+70 java.base at 22-internal > j java.lang.VirtualThread$$Lambda+0x00007f943b0496c0.run()V+4 java.base at 22-internal > j java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute()Ljava/lang/Void;+4 java.base at 22-internal > j java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute(... The stack trace of current thread (where the assert was fired) can explain what is going on a little bit: Current thread (0x00007f93f8043d00): JavaThread "ForkJoinPool-1-worker-1" daemon [_thread_in_vm, id=16779, stack(0x00007f948a597000,0x00007f948a697000) (1024K)] Stack: [0x00007f948a597000,0x00007f948a697000], sp=0x00007f948a6949e0, free space=1014k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x117937d] JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState*)+0x45d (jvmtiEventController.cpp:402) V [libjvm.so+0x1179520] JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState*) [clone .part.0]+0x190 (jvmtiEventController.cpp:632) V [libjvm.so+0x117a1e1] JvmtiEventControllerPrivate::thread_started(JavaThread*)+0x351 (jvmtiEventController.cpp:1174) V [libjvm.so+0x117e608] JvmtiExport::get_jvmti_thread_state(JavaThread*)+0x98 (jvmtiExport.cpp:424) V [libjvm.so+0x118a86c] JvmtiExport::post_field_access(JavaThread*, Method*, unsigned char*, Klass*, Handle, _jfieldID*)+0x6c (jvmtiExport.cpp:2214) V [libjvm.so+0x118b3a1] JvmtiExport::post_field_access_by_jni(JavaThread*, oop, Klass*, _jfieldID*, bool)+0x321 (jvmtiExport.cpp:2202) V [libjvm.so+0x118b4e9] JvmtiExport::jni_GetField_probe(JavaThread*, _jobject*, oop, Klass*, _jfieldID*, bool)+0x79 (jvmtiExport.cpp:2168) V [libjvm.so+0xf83847] jni_GetStaticBooleanField+0x257 (jni.cpp:2047) C [libfieldacc02.so+0x379b] Java_fieldacc02_check+0x6b (jni.h:1546) j fieldacc02.check(Ljava/lang/Object;)I+0 j fieldacc02.lambda$testVirtualThread$0()V+12 j fieldacc02$$Lambda+0x00007f943b001428.run()V+0 j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base at 22-internal j java.lang.VirtualThread.run(Ljava/lang/Runnable;)V+66 java.base at 22-internal j java.lang.VirtualThread$VThreadContinuation$1.run()V+8 java.base at 22-internal j jdk.internal.vm.Continuation.enter0()V+4 java.base at 22-internal j jdk.internal.vm.Continuation.enter(Ljdk/internal/vm/Continuation;Z)V+1 java.base at 22-internal J 124 jdk.internal.vm.Continuation.enterSpecial(Ljdk/internal/vm/Continuation;ZZ)V java.base at 22-internal (0 bytes) @ 0x00007f94c7cdf744 [0x00007f94c7cdf5e0+0x0000000000000164] j jdk.internal.vm.Continuation.run()V+122 java.base at 22-internal j java.lang.VirtualThread.runContinuation()V+70 java.base at 22-internal j java.lang.VirtualThread$$Lambda+0x00007f943b0496c0.run()V+4 java.base at 22-internal j java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute()Ljava/lang/Void;+4 java.base at 22-internal j java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute()Ljava/lang/Object;+1 java.base at 22-internal j java.util.concurrent.ForkJoinTask$InterruptibleTask.exec()Z+51 java.base at 22-internal j java.util.concurrent.ForkJoinTask.doExec()V+10 java.base at 22-internal j java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Ljava/util/concurrent/ForkJoinTask;Ljava/util/concurrent/ForkJoinPool$WorkQueue;I)V+49 java.base at 22-internal j java.util.concurrent.ForkJoinPool.scan(Ljava/util/concurrent/ForkJoinPool$WorkQueue;JI)J+271 java.base at 22-internal j java.util.concurrent.ForkJoinPool.runWorker(Ljava/util/concurrent/ForkJoinPool$WorkQueue;)V+68 java.base at 22-internal j java.util.concurrent.ForkJoinWorkerThread.run()V+31 java.base at 22-internal v ~StubRoutines::call_stub 0x00007f94c7505d21 V [libjvm.so+0xe7b719] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x4a9 (javaCalls.cpp:415) V [libjvm.so+0xe7bdd5] JavaCalls::call_virtual(JavaValue*, Klass*, Symbol*, Symbol*, JavaCallArguments*, JavaThread*)+0x345 (javaCalls.cpp:329) V [libjvm.so+0xe7bff6] JavaCalls::call_virtual(JavaValue*, Handle, Klass*, Symbol*, Symbol*, JavaThread*)+0x76 (javaCalls.cpp:191) V [libjvm.so+0xfd9723] thread_entry(JavaThread*, JavaThread*)+0x93 (jvm.cpp:2937) V [libjvm.so+0xeb06ac] JavaThread::thread_main_inner()+0xcc (javaThread.cpp:720) V [libjvm.so+0x1789496] Thread::call_run()+0xb6 (thread.cpp:220) V [libjvm.so+0x1493b27] thread_native_entry(Thread*)+0x127 (os_linux.cpp:787) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1398365938 From dholmes at openjdk.org Mon Nov 20 01:02:30 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 20 Nov 2023 01:02:30 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: References: <1P_TddTTM1eH75Do2Xq-wBrxXSdh7GzJJlgEBH_dSNo=.94392ab2-12a1-4d1b-9131-6164bbb76e7d@github.com> Message-ID: <-CGha1yFmQNPbT7s6BtZ0iJFxmPgzoSnozx4pgIZlA4=.77aa51ba-ebcb-419a-9651-dbadb8ef9e91@github.com> On Sun, 19 Nov 2023 09:41:34 GMT, Serguei Spitsyn wrote: >> @dcubed-ojdk >> I don't know FJP implementation well enough to point at the code where it happens. However, I observe that new `JavaThread `is being created between two points of the execution path. >> - First point is in the `JvmtiEventControllerPrivate::recompute_enabled()` at the line where a `ThreadsListHandle` is set. I've added a trap checking if any `JavaThread` pointed by `state->get_thread()` is not protected by the `tlh`. I can see this trap is not fired (I can't say it has never been fired). >> - Second point is in the `JvmtiEventControllerPrivate::enter_interp_only_mode()`. If a `ThreadsListHandle` is NOT set then I can observe a `JavaThread` referenced by the state->get_thread() which is not protected by any TLH. It a TLH added into `JvmtiEventControllerPrivate::enter_interp_only_mode()` then this `JavaThread` is observed as protected by TLH. >> I can provide a full stack trace for this `JavaThread` consisting of two parts: carrier thread and virtual thread frames. The name of carrier thread is `ForkJoinPool-1-worker-1`. The virtual thread is a tested virtual thread. >> The thread dump looks las below: >> >> DBG: enter_interp_only_mode: target: 0x7f93f8043d00 virt: 1 carrier: ForkJoinPool-1-worker-1 >> DBG: ##### NATIVE stacktrace of JavaThread: 0x7f93f8043d00 ##### >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) >> j fieldacc02.check(Ljava/lang/Object;)I+0 >> j fieldacc02.lambda$testVirtualThread$0()V+12 >> j fieldacc02$$Lambda+0x00007f943b001428.run()V+0 >> j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base at 22-internal >> j java.lang.VirtualThread.run(Ljava/lang/Runnable;)V+66 java.base at 22-internal >> j java.lang.VirtualThread$VThreadContinuation$1.run()V+8 java.base at 22-internal >> j jdk.internal.vm.Continuation.enter0()V+4 java.base at 22-internal >> j jdk.internal.vm.Continuation.enter(Ljdk/internal/vm/Continuation;Z)V+1 java.base at 22-internal >> J 124 jdk.internal.vm.Continuation.enterSpecial(Ljdk/internal/vm/Continuation;ZZ)V java.base at 22-internal (0 bytes) @ 0x00007f94c7cdf744 [0x00007f94c7cdf5e0+0x0000000000000164] >> j jdk.internal.vm.Continuation.run()V+122 java.base at 22-internal >> j java.lang.VirtualThread.runContinuation()V+70 java.base at 22-internal >> j java.lang.VirtualThread$$Lambda+0x00007f943b0496c0.run()V+4 java.base at 22-internal >> j java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute()Ljava/lang/Void;+4 java.base at 22-internal >> j java.util.concur... > > The stack trace of current thread (where the assert was fired) can explain what is going on a little bit: > > Current thread (0x00007f93f8043d00): JavaThread "ForkJoinPool-1-worker-1" daemon [_thread_in_vm, id=16779, stack(0x00007f948a597000,0x00007f948a697000) (1024K)] > > Stack: [0x00007f948a597000,0x00007f948a697000], sp=0x00007f948a6949e0, free space=1014k > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) > V [libjvm.so+0x117937d] JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState*)+0x45d (jvmtiEventController.cpp:402) > V [libjvm.so+0x1179520] JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState*) [clone .part.0]+0x190 (jvmtiEventController.cpp:632) > V [libjvm.so+0x117a1e1] JvmtiEventControllerPrivate::thread_started(JavaThread*)+0x351 (jvmtiEventController.cpp:1174) > V [libjvm.so+0x117e608] JvmtiExport::get_jvmti_thread_state(JavaThread*)+0x98 (jvmtiExport.cpp:424) > V [libjvm.so+0x118a86c] JvmtiExport::post_field_access(JavaThread*, Method*, unsigned char*, Klass*, Handle, _jfieldID*)+0x6c (jvmtiExport.cpp:2214) > V [libjvm.so+0x118b3a1] JvmtiExport::post_field_access_by_jni(JavaThread*, oop, Klass*, _jfieldID*, bool)+0x321 (jvmtiExport.cpp:2202) > V [libjvm.so+0x118b4e9] JvmtiExport::jni_GetField_probe(JavaThread*, _jobject*, oop, Klass*, _jfieldID*, bool)+0x79 (jvmtiExport.cpp:2168) > V [libjvm.so+0xf83847] jni_GetStaticBooleanField+0x257 (jni.cpp:2047) > C [libfieldacc02.so+0x379b] Java_fieldacc02_check+0x6b (jni.h:1546) > j fieldacc02.check(Ljava/lang/Object;)I+0 > j fieldacc02.lambda$testVirtualThread$0()V+12 > j fieldacc02$$Lambda+0x00007f943b001428.run()V+0 > j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base at 22-internal > j java.lang.VirtualThread.run(Ljava/lang/Runnable;)V+66 java.base at 22-internal > j java.lang.VirtualThread$VThreadContinuation$1.run()V+8 java.base at 22-internal > j jdk.internal.vm.Continuation.enter0()V+4 java.base at 22-internal > j jdk.internal.vm.Continuation.enter(Ljdk/internal/vm/Continuation;Z)V+1 java.base at 22-internal > J 124 jdk.internal.vm.Continuation.enterSpecial(Ljdk/internal/vm/Continuation;ZZ)V java.base at 22-internal (0 bytes) @ 0x00007f94c7cdf744 [0x00007f94c7cdf5e0+0x0000000000000164] > j jdk.internal.vm.Continuation.run()V+122 java.base at 22-internal > j java.lang.VirtualThread.runContinuation()V+70 java.base at 22-internal > j java.lang.VirtualThread$$Lambda+0x00007f943b0496c0.run()V+4 java.base at 22-internal > j java.util.concur... Just to re-iterate what Dan was saying, the TLH is only of use if you are accessing threads known to be included in the TLH. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1398540533 From qamai at openjdk.org Mon Nov 20 02:38:41 2023 From: qamai at openjdk.org (Quan Anh Mai) Date: Mon, 20 Nov 2023 02:38:41 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v8] In-Reply-To: References: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> <648SrHxCX6_kRX7cmxyGurxOWecLTWUw0_C79J_okbo=.473eaa8c-7dfc-4aa0-839d-bb580cc9d312@github.com> Message-ID: On Fri, 17 Nov 2023 13:19:40 GMT, Afshin Zafari wrote: >> src/hotspot/share/utilities/growableArray.hpp line 213: >> >>> 211: >>> 212: template >>> 213: int find(T* token, F f) const { >> >> Should be >> >> template >> int find(F f) const { >> for (int i = 0; i < _len; i++) { >> if (f(_data[i]) { >> return i; >> } >> } >> return -1; >> } > > We need `token` to find it in the array, don't we? All the invocations pass such a function with two parameters. The change here needs all invocations to be changed. No, it can be embedded into the function object. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15418#discussion_r1397336104 From qamai at openjdk.org Mon Nov 20 02:38:42 2023 From: qamai at openjdk.org (Quan Anh Mai) Date: Mon, 20 Nov 2023 02:38:42 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v8] In-Reply-To: References: <97IBSrr12htoiw751JlhL4f7jiEZeoYVF9hQjas8vrI=.a7143156-e1d5-4774-ba4b-08e29eb05389@github.com> <648SrHxCX6_kRX7cmxyGurxOWecLTWUw0_C79J_okbo=.473eaa8c-7dfc-4aa0-839d-bb580cc9d312@github.com> Message-ID: On Fri, 17 Nov 2023 13:45:14 GMT, Quan Anh Mai wrote: >> We need `token` to find it in the array, don't we? All the invocations pass such a function with two parameters. The change here needs all invocations to be changed. > > No, it can be embedded into the function object. I think you can have 2 versions, `GrowableArray::find(const E& value)` and `GrowableArray::find_if(UnaryPredicate p)`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15418#discussion_r1398608278 From dholmes at openjdk.org Mon Nov 20 08:02:43 2023 From: dholmes at openjdk.org (David Holmes) Date: Mon, 20 Nov 2023 08:02:43 GMT Subject: RFR: 8319876: Reduce memory consumption of VM_ThreadDump::doit [v2] In-Reply-To: References: <2vdyoJ_-mtyaDH7tYcnsHjeJc0O3IxxdNDsS5ofXOE0=.840ef6d9-13d3-400e-a768-246094b3559a@github.com> Message-ID: On Fri, 17 Nov 2023 08:10:21 GMT, Stefan Karlsson wrote: >> src/hotspot/share/services/threadService.cpp line 698: >> >>> 696: RegisterMap::ProcessFrames::include, >>> 697: RegisterMap::WalkContinuation::skip); >>> 698: ResourceMark rm; >> >> Nit: Use `rm(VMThread::vm_thread());` to avoid the need to call `Thread::current()`. > > FWIW, I don't see the appeal to micro optimize away one call to Thread::current() in a function that performs a massive amount of work. In runtime, we prefer to always use an available reference to the current thread than look it up via `Thread::current()`. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16598#discussion_r1398776709 From sspitsyn at openjdk.org Mon Nov 20 10:40:33 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Mon, 20 Nov 2023 10:40:33 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: <-CGha1yFmQNPbT7s6BtZ0iJFxmPgzoSnozx4pgIZlA4=.77aa51ba-ebcb-419a-9651-dbadb8ef9e91@github.com> References: <1P_TddTTM1eH75Do2Xq-wBrxXSdh7GzJJlgEBH_dSNo=.94392ab2-12a1-4d1b-9131-6164bbb76e7d@github.com> <-CGha1yFmQNPbT7s6BtZ0iJFxmPgzoSnozx4pgIZlA4=.77aa51ba-ebcb-419a-9651-dbadb8ef9e91@github.com> Message-ID: On Mon, 20 Nov 2023 00:59:35 GMT, David Holmes wrote: >> The stack trace of current thread (where the assert was fired) can explain what is going on a little bit: >> >> Current thread (0x00007f93f8043d00): JavaThread "ForkJoinPool-1-worker-1" daemon [_thread_in_vm, id=16779, stack(0x00007f948a597000,0x00007f948a697000) (1024K)] >> >> Stack: [0x00007f948a597000,0x00007f948a697000], sp=0x00007f948a6949e0, free space=1014k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) >> V [libjvm.so+0x117937d] JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState*)+0x45d (jvmtiEventController.cpp:402) >> V [libjvm.so+0x1179520] JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState*) [clone .part.0]+0x190 (jvmtiEventController.cpp:632) >> V [libjvm.so+0x117a1e1] JvmtiEventControllerPrivate::thread_started(JavaThread*)+0x351 (jvmtiEventController.cpp:1174) >> V [libjvm.so+0x117e608] JvmtiExport::get_jvmti_thread_state(JavaThread*)+0x98 (jvmtiExport.cpp:424) >> V [libjvm.so+0x118a86c] JvmtiExport::post_field_access(JavaThread*, Method*, unsigned char*, Klass*, Handle, _jfieldID*)+0x6c (jvmtiExport.cpp:2214) >> V [libjvm.so+0x118b3a1] JvmtiExport::post_field_access_by_jni(JavaThread*, oop, Klass*, _jfieldID*, bool)+0x321 (jvmtiExport.cpp:2202) >> V [libjvm.so+0x118b4e9] JvmtiExport::jni_GetField_probe(JavaThread*, _jobject*, oop, Klass*, _jfieldID*, bool)+0x79 (jvmtiExport.cpp:2168) >> V [libjvm.so+0xf83847] jni_GetStaticBooleanField+0x257 (jni.cpp:2047) >> C [libfieldacc02.so+0x379b] Java_fieldacc02_check+0x6b (jni.h:1546) >> j fieldacc02.check(Ljava/lang/Object;)I+0 >> j fieldacc02.lambda$testVirtualThread$0()V+12 >> j fieldacc02$$Lambda+0x00007f943b001428.run()V+0 >> j java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V+5 java.base at 22-internal >> j java.lang.VirtualThread.run(Ljava/lang/Runnable;)V+66 java.base at 22-internal >> j java.lang.VirtualThread$VThreadContinuation$1.run()V+8 java.base at 22-internal >> j jdk.internal.vm.Continuation.enter0()V+4 java.base at 22-internal >> j jdk.internal.vm.Continuation.enter(Ljdk/internal/vm/Continuation;Z)V+1 java.base at 22-internal >> J 124 jdk.internal.vm.Continuation.enterSpecial(Ljdk/internal/vm/Continuation;ZZ)V java.base at 22-internal (0 bytes) @ 0x00007f94c7cdf744 [0x00007f94c7cdf5e0+0x0000000000000164] >> j jdk.internal.vm.Continuation.run()V+122 java.base at 22-internal >> j java.lang.VirtualThread.runContinuation()V+70 java.base at 22-internal >> j java.lang.VirtualThread$$Lambda+0x00007f943b049... > > Just to re-iterate what Dan was saying, the TLH is only of use if you are accessing threads known to be included in the TLH. The issue occurred to be with the current thread which has to be safe to access without a TLH. However, there is this guarantee in the `Hanshake::execute()`: void Handshake::execute(HandshakeClosure* hs_cl, ThreadsListHandle* tlh, JavaThread* target) { . . . guarantee(target != nullptr, "must be"); if (tlh == nullptr) { guarantee(Thread::is_JavaThread_protected_by_TLH(target), "missing ThreadsListHandle in calling context."); This guarantee fired for current thread is a source of confusion and mistakes. Would it be a right thing to correct it? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1398993572 From jiangli at openjdk.org Mon Nov 20 19:28:13 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 20 Nov 2023 19:28:13 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v3] In-Reply-To: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: > Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: Add a check for a thread is_attaching_via_jni, based on David Holmes' comment. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16642/files - new: https://git.openjdk.org/jdk/pull/16642/files/c2f83e8a..7c0214e2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=01-02 Stats: 5 lines in 1 file changed: 3 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16642.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16642/head:pull/16642 PR: https://git.openjdk.org/jdk/pull/16642 From jiangli at openjdk.org Mon Nov 20 19:28:17 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Mon, 20 Nov 2023 19:28:17 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v2] In-Reply-To: <1sURlXfpTlEu9U30aZAojUIhDgtzeyA8MYrJ_q3xDUs=.bda6a027-ebc8-4107-a1f7-be3edf737e5f@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> <1sURlXfpTlEu9U30aZAojUIhDgtzeyA8MYrJ_q3xDUs=.bda6a027-ebc8-4107-a1f7-be3edf737e5f@github.com> Message-ID: On Thu, 16 Nov 2023 09:48:48 GMT, David Holmes wrote: >> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. > > src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 87: > >> 85: // Don't add a JvmtiThreadState to a thread that is exiting. >> 86: return nullptr; >> 87: } > > I'm wondering if there should also be an `is_jni_attaching` check here? That seems to be a good idea. It would cover other cases that we haven't seen yet. Added a check as suggested, thanks. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1399647867 From coleenp at openjdk.org Mon Nov 20 22:12:09 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Mon, 20 Nov 2023 22:12:09 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 17:56:09 GMT, Jaroslav Bachorik wrote: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Good analysis for a very subtle bug. I have a couple of comments, and maybe the test can be simplified but approving the change. src/hotspot/share/classfile/classFileParser.cpp line 5579: > 5577: > 5578: if (_methods != nullptr) { > 5579: // Free methods - those methods are not fully wired and miss the method holder How about saying: for methods whose InstanceKlass as method holder is not yet created? test/hotspot/jtreg/serviceability/jvmti/thread/GetStackTrace/GetStackTraceAndRetransformTest/GetStackTraceAndRetransformTest.java line 53: > 51: import java.util.List; > 52: import java.util.concurrent.CyclicBarrier; > 53: import java.util.concurrent.locks.LockSupport; Do you need all these imports? There's a simple RedefineClassHelper class that does most of the work, but maybe you need the explicit agent code to reproduce the crash? See test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineRunningMethodsWithBacktrace.java as an example. ------------- Marked as reviewed by coleenp (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16662#pullrequestreview-1740746213 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1399811821 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1399810008 From coleenp at openjdk.org Mon Nov 20 22:12:12 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Mon, 20 Nov 2023 22:12:12 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Sat, 18 Nov 2023 00:23:44 GMT, Jaroslav Bachorik wrote: >> src/hotspot/share/oops/instanceKlass.cpp line 541: >> >>> 539: // The previous version will point to them so they're not totally dangling >>> 540: assert (!method->on_stack(), "shouldn't be called with methods on stack"); >>> 541: // Do the pointer maintenance before releasing the metadata, but not for incomplete methods >> >> I'm confused by what you mean by method holder, which I think of as methodHandle. Or InstanceKlass is the holder of the methods. Maybe this should be more explicit that it's talking about clearing any associated jmethodIDs. > > The method holder is an `InstanceKlass` object which can be retrieved as `method->method_holder()` (I apologize if I am using not completely correct terms - this is what I grokked from the sources). And incomplete methods created by the `ClassParser` from the class data stream will not have the link to that `InstanceKlass` set up if the `ClassParser` is already having its `_klass` field set to a non-null value. > > If we are talking about clearing any jmetbodIDs associated with an `InstanceKlass` instance it is not really possible for old method versions because only the current `InstanceKlass` version has the jmethodID cache associated with it and it contains jmethodIDs pointing to bot the old and current methods. I see, holder is the right word and concept. So the parameter means has_method_holder, in that the InstanceKlass has been fully parsed at the point of clearing the jmethodIDs. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1399785853 From coleenp at openjdk.org Mon Nov 20 22:12:14 2023 From: coleenp at openjdk.org (Coleen Phillimore) Date: Mon, 20 Nov 2023 22:12:14 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Thu, 16 Nov 2023 03:11:04 GMT, Jaroslav Bachorik wrote: >> src/hotspot/share/oops/method.cpp line 2277: >> >>> 2275: } >>> 2276: } >>> 2277: >> >> Can this race with redefinition? > > The cleanup of previous versions is executed in VM_Operation at a safepoint - therefore we should be safe against races with class redefinitions. > I am adding an assert to `clear_jmethod_id()` to check for being at a safepoint. Yes, these are cleaned at a safepoint. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1399798259 From pchilanomate at openjdk.org Mon Nov 20 23:08:06 2023 From: pchilanomate at openjdk.org (Patricio Chilano Mateo) Date: Mon, 20 Nov 2023 23:08:06 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v10] In-Reply-To: References: Message-ID: On Sun, 19 Nov 2023 05:12:50 GMT, Serguei Spitsyn wrote: >> The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. >> At the low level, the JVMTI code supporting platform and virtual threads still can be different. >> This implementation is based on the `JvmtiVTMSTransitionDisabler` class. >> >> The internal API includes two new classes: >> - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` >> >> The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. >> >> The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: >> - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` >> >> To get the test results clean, the update also fixes the test issue: >> [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" >> >> Testing: >> - the mach5 tiers 1-6 are all passed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: remove java_lang_VirtualThread::NEW check from is_vthread_alive Thanks Serguei, changes look good to me. ------------- Marked as reviewed by pchilanomate (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16460#pullrequestreview-1740848784 From amenkov at openjdk.org Tue Nov 21 01:28:10 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 21 Nov 2023 01:28:10 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v10] In-Reply-To: References: Message-ID: On Sun, 19 Nov 2023 05:12:50 GMT, Serguei Spitsyn wrote: >> The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. >> At the low level, the JVMTI code supporting platform and virtual threads still can be different. >> This implementation is based on the `JvmtiVTMSTransitionDisabler` class. >> >> The internal API includes two new classes: >> - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` >> >> The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. >> >> The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: >> - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` >> >> To get the test results clean, the update also fixes the test issue: >> [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" >> >> Testing: >> - the mach5 tiers 1-6 are all passed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: remove java_lang_VirtualThread::NEW check from is_vthread_alive Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16460#pullrequestreview-1740951932 From jjoo at openjdk.org Tue Nov 21 02:19:47 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 02:19:47 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v45] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Address comments and refactor TTTC class for simplification ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/ce7dbfcf..17a8eaf3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=44 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=43-44 Stats: 97 lines in 16 files changed: 11 ins; 28 del; 58 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Tue Nov 21 02:19:48 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 02:19:48 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v44] In-Reply-To: References: Message-ID: On Wed, 15 Nov 2023 23:50:02 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace Addressed comments, but am running into an assertion failure here when building: https://github.com/openjdk/jdk/pull/15082/files#diff-d1c5f7a125171a3828bdb3f4488327e03ceb96ff615eacfc9df406aca408cbe6R72 Will investigate tomorrow. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1820104586 From amenkov at openjdk.org Tue Nov 21 02:58:47 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 21 Nov 2023 02:58:47 GMT Subject: RFR: JDK-8318626: GetClassFields does not filter out ConstantPool.constantPoolOop field [v2] In-Reply-To: <38iFcw9EQ-yK-2YU7n5So96EsxCKJo9aFAJxEQV_QTA=.a033561a-90fa-49b5-a112-8d58778cd9df@github.com> References: <38iFcw9EQ-yK-2YU7n5So96EsxCKJo9aFAJxEQV_QTA=.a033561a-90fa-49b5-a112-8d58778cd9df@github.com> Message-ID: > FilteredFieldStream is intended to filter out some fields which does not represent valid java objects. > Currently the only filtered field is "constantPoolOop" from jdk.internal.reflect.ConstantPool class. > The change fixes FilteredFieldStream implementation to handle cases when filtered fields is the last field of the class ("constantPoolOop" is the only field of jdk.internal.reflect.ConstantPool) > > Testing: > - new test added that compares results of GetClassFields JVMTI function (it uses FilteredFieldStream) with Class.getDeclaredFields(); > - test/hotspot/jtreg/vmTestbase/nsk/jvmti/GetClassFields tests Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: feedback ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16328/files - new: https://git.openjdk.org/jdk/pull/16328/files/541f9f32..ce61c093 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16328&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16328&range=00-01 Stats: 168 lines in 3 files changed: 82 ins; 86 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16328.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16328/head:pull/16328 PR: https://git.openjdk.org/jdk/pull/16328 From amenkov at openjdk.org Tue Nov 21 02:58:48 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 21 Nov 2023 02:58:48 GMT Subject: RFR: JDK-8318626: GetClassFields does not filter out ConstantPool.constantPoolOop field [v2] In-Reply-To: <7_F4U-iyeatpOE7XcwLhFTQYj-ebsyPY86w8Tv9FJsA=.b7b76d6c-0ae3-4b4b-bc1f-849a10d59f40@github.com> References: <38iFcw9EQ-yK-2YU7n5So96EsxCKJo9aFAJxEQV_QTA=.a033561a-90fa-49b5-a112-8d58778cd9df@github.com> <7_F4U-iyeatpOE7XcwLhFTQYj-ebsyPY86w8Tv9FJsA=.b7b76d6c-0ae3-4b4b-bc1f-849a10d59f40@github.com> Message-ID: On Sat, 28 Oct 2023 00:45:19 GMT, Serguei Spitsyn wrote: >> Alex Menkov has updated the pull request incrementally with one additional commit since the last revision: >> >> feedback > > test/hotspot/jtreg/serviceability/FilteredFields/FilteredFieldsTest.java line 25: > >> 23: >> 24: /* >> 25: * @test > > @bug and @summary is needed as well. > The test folder has to be at least `test/hotspot/jtreg/serviceability/jvmti`. > It would be nice to check with Leonid on this. Added. Moved test fir to test/hotspot/jtreg/serviceability/jvmti/GetClassFields/ > test/hotspot/jtreg/serviceability/FilteredFields/libFilteredFieldsTest.cpp line 63: > >> 61: jfieldID *fields = nullptr; >> 62: >> 63: jvmtiError err = jvmti->GetClassFields(clazz, &fcount, &fields); > > Nit: The function `check_jvmti_status()` or `check_jvmti_error()` from jvmti_common.h` can be used here. > The function `fatal()` in other cases like at line 54 can be used. Fixed ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16328#discussion_r1399969645 PR Review Comment: https://git.openjdk.org/jdk/pull/16328#discussion_r1399970204 From sspitsyn at openjdk.org Tue Nov 21 08:20:19 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 21 Nov 2023 08:20:19 GMT Subject: RFR: 8319244: implement JVMTI handshakes support for virtual threads [v10] In-Reply-To: References: Message-ID: On Sun, 19 Nov 2023 05:12:50 GMT, Serguei Spitsyn wrote: >> The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. >> At the low level, the JVMTI code supporting platform and virtual threads still can be different. >> This implementation is based on the `JvmtiVTMSTransitionDisabler` class. >> >> The internal API includes two new classes: >> - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` >> >> The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. >> >> The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: >> - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` >> >> To get the test results clean, the update also fixes the test issue: >> [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" >> >> Testing: >> - the mach5 tiers 1-6 are all passed > > Serguei Spitsyn has updated the pull request incrementally with one additional commit since the last revision: > > review: remove java_lang_VirtualThread::NEW check from is_vthread_alive Patricio and Alex, thank you a lot for review! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16460#issuecomment-1820429682 From sspitsyn at openjdk.org Tue Nov 21 08:20:20 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 21 Nov 2023 08:20:20 GMT Subject: Integrated: 8319244: implement JVMTI handshakes support for virtual threads In-Reply-To: References: Message-ID: On Wed, 1 Nov 2023 18:44:04 GMT, Serguei Spitsyn wrote: > The handshakes support for virtual threads is needed to simplify the JVMTI implementation for virtual threads. There is a significant duplication in the JVMTI code to differentiate code intended to support platform, virtual threads or both. The handshakes are unified, so it is enough to define just one handshake for both platform and virtual thread. > At the low level, the JVMTI code supporting platform and virtual threads still can be different. > This implementation is based on the `JvmtiVTMSTransitionDisabler` class. > > The internal API includes two new classes: > - `JvmtiHandshake` and `JvmtiUnifiedHandshakeClosure` > > The `JvmtiUnifiedHandshakeClosure` defines two different callback functions: `do_thread()` and `do_vthread()`. > > The first JVMTI functions are picked first to be converted to use the `JvmtiHandshake`: > - `GetStackTrace`, `GetFrameCount`, `GetFrameLocation`, `NotifyFramePop` > > To get the test results clean, the update also fixes the test issue: > [8318631](https://bugs.openjdk.org/browse/JDK-8318631): GetStackTraceSuspendedStressTest.java failed with "check_jvmti_status: JVMTI function returned error: JVMTI_ERROR_THREAD_NOT_ALIVE (15)" > > Testing: > - the mach5 tiers 1-6 are all passed This pull request has now been integrated. Changeset: 839dd653 Author: Serguei Spitsyn URL: https://git.openjdk.org/jdk/commit/839dd653663867f770fbe4af0a57468675eb12db Stats: 498 lines in 4 files changed: 138 ins; 334 del; 26 mod 8319244: implement JVMTI handshakes support for virtual threads Reviewed-by: pchilanomate, amenkov ------------- PR: https://git.openjdk.org/jdk/pull/16460 From kevin.walls at oracle.com Tue Nov 21 09:12:33 2023 From: kevin.walls at oracle.com (Kevin Walls) Date: Tue, 21 Nov 2023 09:12:33 +0000 Subject: [External] : Re: Proposal: Add overwrite heapdump flag to java In-Reply-To: References: <083b1eef-c7b1-4db0-0096-db8d37e690d5@oracle.com> <5e497f58-2e38-38b9-acf2-adf1a99d5e91@oracle.com> Message-ID: Hi, Just going back to a technical point, did you manage to capture a stacktrace from the hang when running with -XX:OnError="jcmd %p GC.heap_dump -overwrite filename.dmp" It would be great to know if that is a reliable hang, and what it looks like. Maybe gclocker should not be blocking us at the point OnError commands are run. We should understand that before making the change, as it seems like a reasonable alternative.. Do you still suggest HeapDumpOverwrite? Maybe we can think about if that suggests it applies to all heap dumps? Or, should we be handling FIFOs specifically, and not need the overwrite flag? Thanks! Kevin From: Elon Azoulay Sent: 18 November 2023 05:20 To: Kevin Walls Cc: serviceability-dev at openjdk.org; jdk-dev at openjdk.org Subject: Re: [External] : Re: Proposal: Add overwrite heapdump flag to java Hi, I wanted to know what I can do to move https://github.com/openjdk/jdk/pull/13276 along. There was advice to create a CSR request, in order to do that there needs to be a jira but I don't think I have permissions to create one. Thanks for all the reviews, let me know what I can do, happy to follow your advice! On Thu, May 25, 2023 at 11:39?PM Kevin Walls > wrote: Hi, (I put this in the PR, but maybe drafts don?t get updates put on the mailing list?) HeapDumpOverwrite sounds quite general, there are different ways of heap dumping and this doesn't affect all of them. This could be HeapDumpOnOutOfMemoryErrorOverwrite, which is a long option even for us 8-) so could be abbreviated perhaps, BUT that option might not be the way to do it? Is it really that all heap dumps should be permitted to an existing file, IF that file is a FIFO? From the email I think that's the problem? ..and if so, we have: 8267666: Add option to jcmd GC.heap_dump to use existing file https://bugs.openjdk.org/browse/JDK-8267666 So if jcmd to create a dump can use an existing file, -XX:OnError= can trigger such a jcmd? i.e. Maybe there is a way of doing what you want. A code change could still be required to make this easier for heap dumps on out of memory. Writing a JBS bug first is useful to define the problem. Thanks Kevin From: jdk-dev > On Behalf Of Elon Azoulay Sent: 25 May 2023 21:21 To: Daniel Daugherty > Cc: serviceability-dev at openjdk.org; jdk-dev at openjdk.org Subject: Re: [External] : Re: Proposal: Add overwrite heapdump flag to java Sounds good, thanks so much! On Thu, May 25, 2023 at 12:13?PM > wrote: Pointing the older thread to this newer incarnation would be a good idea. Dan On 5/25/23 3:11 PM, Elon Azoulay wrote: Hi Dan, That's great! Should I follow up in that thread as well? On Thu, May 25, 2023 at 12:04?PM > wrote: Greetings, A similar request came up on the hotspot-dev alias back in 2022.11: https://mail.openjdk.org/pipermail/hotspot-dev/2022-November/066956.html Dan On 5/25/23 1:48 AM, David Holmes wrote: > Hi Elon, > > I would suggest taking this up on serviceability-dev - cc'd. > > On 25/05/2023 3:46 am, Elon Azoulay wrote: >> Hi, >> I submitted a pull request >> > to expose the overwrite >> flag to HeapDumpOnOutOfMemoryError. >> The flag is already exposed to jcmd as -overwrite.This is to >> facilitate creating a heapdump within a container native environment. >> We use this internally to dump the heap dump to a fifo so that we >> could read it in a separate container.Let me know what needs to be >> done in terms of creating an issue and getting my pr merged. > > Have you gone through the guide: > > https://openjdk.org/guide/ > > ? > > You will need a JBS issue and also a CSR request as this proposes to > add a new manageable product flag. > > Cheers, > David > >> I am a new contributor and would be happy to contribute more! >> >> Cheers, >> >> Elon -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevinw at openjdk.org Tue Nov 21 13:09:14 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Tue, 21 Nov 2023 13:09:14 GMT Subject: RFR: 8313355: javax/management/remote/mandatory/notif/ListenerScaleTest.java failed with "Exception: Failed: ratio=792.2791601423487" Message-ID: This test fails occasionally, and can be tuned to avoid most of the failures ever seen. It exists to check that notification work does not scale linearly with number of MBeans, so it calculates a ratio of the time taken with one, and with 20,000 MBeans. We should increase point at which that calculated ratio is considered a failure. I chose 2500 here as it solves most of the failures that I think have ever been reported with the test, without being too forgiving that we ignore a serious slowdown in future. ------------- Commit messages: - 8313355: javax/management/remote/mandatory/notif/ListenerScaleTest.java failed with "Exception: Failed: ratio=792.2791601423487" Changes: https://git.openjdk.org/jdk/pull/16761/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16761&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8313355 Stats: 4 lines in 1 file changed: 2 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16761.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16761/head:pull/16761 PR: https://git.openjdk.org/jdk/pull/16761 From vladimir.kozlov at oracle.com Tue Nov 21 17:09:46 2023 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 21 Nov 2023 09:09:46 -0800 Subject: Handling of DTraceMethodProbes In-Reply-To: References: Message-ID: <6f99ca60-4317-46c4-95ea-062190f603fd@oracle.com> CCing to serviceability group. Do we still support dtrace? Thanks, Vladimir K On 11/21/23 4:34 AM, Thomas St?fe wrote: > Hi, > > It seems that the handling of DTraceMethodProbes is inconsistent: in most places, we handle the flag at runtime by > generating code that uses conditional jumps. In one place (C1), we don't. There, generation is conditional on > DTraceMethodProbes at compile time. > > The former looks like it is possible for the value of?DTraceMethodProbes to change at runtime, and that generated code > should be able to react seamlessly. However, DTraceMethodProbes is not a manageable flag. > > Which side is right? Do we need to emit conditional jumps, but if so, why not in C1? > > Thanks, Thomas From cjplummer at openjdk.org Tue Nov 21 20:27:15 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 21 Nov 2023 20:27:15 GMT Subject: RFR: 8320536: problemlist failing serviceability/attach/ConcAttachTest.java test on macosx Message-ID: Problemlisting ConcAttachTest.java on OSX (it's already problem listed on linux and doesn't get run on Windows). This one test seems to be causing all the issues with attach and dcmd tests described in [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866), so if we problem list it we should stop seeing the other test failures. ------------- Commit messages: - Problemlist ConcAttachTest.java on macosx Changes: https://git.openjdk.org/jdk/pull/16772/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16772&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320536 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16772.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16772/head:pull/16772 PR: https://git.openjdk.org/jdk/pull/16772 From cjplummer at openjdk.org Tue Nov 21 20:45:17 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 21 Nov 2023 20:45:17 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap In-Reply-To: References: Message-ID: On Fri, 22 Sep 2023 02:48:57 GMT, David Holmes wrote: >> `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an option for specifying a file name. >> >> The help message of Compiler.perfmap: >> >> Compiler.perfmap >> Write map file for Linux perf tool. >> >> Impact: Low >> >> Syntax : Compiler.perfmap [options] >> >> Options: (options must be specified using the or = syntax) >> filename : [optional] Name of the map file (STRING, no default value) > > src/jdk.jcmd/share/man/jcmd.1 line 1: > >> 1: .\" Copyright (c) 2012, 2023, Oracle and/or its affiliates. All rights reserved. > > The actual markdown source for this file needs to be updated with these changes. Those sources are not open-source unfortunately. Please either coordinate to get the sources updated with an Oracle developer as part of this PR (they will integrate the internal part), or else please defer this to a subtask and let an Oracle developer update the source and output at the same time. Thanks. I filed JDK-8320556 to update the closed source. It's assigned to me. I'll do the update after these changes are pushed. In the meantime I'll make sure the current jcmd.l changes are correct and match the closed changes I'll be making. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15871#discussion_r1401151891 From cjplummer at openjdk.org Tue Nov 21 21:34:14 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Tue, 21 Nov 2023 21:34:14 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap In-Reply-To: References: Message-ID: On Thu, 21 Sep 2023 20:43:56 GMT, Yi-Fan Tsai wrote: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an option for specifying a file name. > > The help message of Compiler.perfmap: > > Compiler.perfmap > Write map file for Linux perf tool. > > Impact: Low > > Syntax : Compiler.perfmap [options] > > Options: (options must be specified using the or = syntax) > filename : [optional] Name of the map file (STRING, no default value) @yftsai Since you requested a CSR review, I assume you want this PR re-opened. Can you please re-open it in that case. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15871#issuecomment-1821721789 From jjoo at openjdk.org Tue Nov 21 21:42:39 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 21:42:39 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: References: Message-ID: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: - Update memory tracking type for CPUTimeCounters - Fix assertion logic ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/17a8eaf3..4ca30f32 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=45 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=44-45 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Tue Nov 21 21:46:18 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Tue, 21 Nov 2023 21:46:18 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> References: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> Message-ID: On Tue, 21 Nov 2023 21:42:39 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Update memory tracking type for CPUTimeCounters > - Fix assertion logic All comments have been addressed and assertion failure fixed - this PR should once again be RFR. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1821740369 From amenkov at openjdk.org Tue Nov 21 22:03:04 2023 From: amenkov at openjdk.org (Alex Menkov) Date: Tue, 21 Nov 2023 22:03:04 GMT Subject: RFR: 8320536: problemlist failing serviceability/attach/ConcAttachTest.java test on macosx In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 20:16:39 GMT, Chris Plummer wrote: > Problemlisting ConcAttachTest.java on OSX (it's already problem listed on linux and doesn't get run on Windows). This one test seems to be causing all the issues with attach and dcmd tests described in [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866), so if we problem list it we should stop seeing the other test failures. Marked as reviewed by amenkov (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/16772#pullrequestreview-1743211441 From duke at openjdk.org Tue Nov 21 22:21:18 2023 From: duke at openjdk.org (Yi-Fan Tsai) Date: Tue, 21 Nov 2023 22:21:18 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v2] In-Reply-To: References: Message-ID: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an option for specifying a file name. > > The help message of `jcmd PID help Compiler.perfmap` will be updated in a separate PR. Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: Remove changes of jcmd man page ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15871/files - new: https://git.openjdk.org/jdk/pull/15871/files/9fee339b..861bda74 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=00-01 Stats: 12 lines in 2 files changed: 1 ins; 10 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/15871.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15871/head:pull/15871 PR: https://git.openjdk.org/jdk/pull/15871 From sspitsyn at openjdk.org Tue Nov 21 22:28:05 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 21 Nov 2023 22:28:05 GMT Subject: RFR: 8308614: Enabling JVMTI ClassLoad event slows down vthread creation by factor 10 [v3] In-Reply-To: <-CGha1yFmQNPbT7s6BtZ0iJFxmPgzoSnozx4pgIZlA4=.77aa51ba-ebcb-419a-9651-dbadb8ef9e91@github.com> References: <1P_TddTTM1eH75Do2Xq-wBrxXSdh7GzJJlgEBH_dSNo=.94392ab2-12a1-4d1b-9131-6164bbb76e7d@github.com> <-CGha1yFmQNPbT7s6BtZ0iJFxmPgzoSnozx4pgIZlA4=.77aa51ba-ebcb-419a-9651-dbadb8ef9e91@github.com> Message-ID: On Mon, 20 Nov 2023 00:59:35 GMT, David Holmes wrote: >> @dcubed-ojdk >> I don't know FJP implementation well enough to point at the code where it happens. However, I observe that new `JavaThread `is being created between two points of the execution path. >> - First point is in the `JvmtiEventControllerPrivate::recompute_enabled()` at the line where a `ThreadsListHandle` is set. I've added a trap checking if any `JavaThread` pointed by `state->get_thread()` is not protected by the `tlh`. I can see this trap is not fired (I can't say it has never been fired). >> - Second point is in the `JvmtiEventControllerPrivate::enter_interp_only_mode()`. If a `ThreadsListHandle` is NOT set then I can observe a `JavaThread` referenced by the state->get_thread() which is not protected by any TLH. It a TLH added into `JvmtiEventControllerPrivate::enter_interp_only_mode()` then this `JavaThread` is observed as protected by TLH. >> >> I've removed a part of this comment with stack traces as my traps were not fully correct, need to double check everything. This issue is not well reproducible but I'm still trying to reproduce it again. >> One approach would be to remove this change in the `src/hotspot/share/prims/jvmtiEventController.cpp` from PR and try address it separately. > > Just to re-iterate what Dan was saying, the TLH is only of use if you are accessing threads known to be included in the TLH. I've spend some time trying to reproduce the original issue to provide some details to Dan but there is no luck yet. I'm sure the issue still exists but it is better to investigate and address it separately from this PR. So, my plan is to remove this line with TLH from the `jvmtiEventController.cpp`. BTW, if it is still needed Alan promised to point to the code where the FJP compensating mechanism adds new carrier threads. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16686#discussion_r1401283561 From sspitsyn at openjdk.org Tue Nov 21 22:49:09 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 21 Nov 2023 22:49:09 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v3] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 20 Nov 2023 19:28:13 GMT, Jiangli Zhou wrote: >> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. > > Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: > > Add a check for a thread is_attaching_via_jni, based on David Holmes' comment. src/hotspot/share/prims/jvmtiExport.cpp line 3144: > 3142: // If the current thread is attaching from native and its thread oop is being > 3143: // allocated, things are not ready for allocation sampling. > 3144: if (thread->is_Java_thread()) { Nit: There is no need for this check at line 3144. There was already check for `!thread->is_Java_thread()` and return with false at line 3138: if (!thread->is_Java_thread() || thread->is_Compiler_thread()) { return false; } ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1401302311 From duke at openjdk.org Tue Nov 21 23:29:21 2023 From: duke at openjdk.org (Yi-Fan Tsai) Date: Tue, 21 Nov 2023 23:29:21 GMT Subject: RFR: 8314029: Add file name parameter to Compiler.perfmap [v3] In-Reply-To: References: Message-ID: > `jcmd Compiler.perfmap` uses the hard-coded file name for a perf map: `/tmp/perf-%d.map`. This change adds an option for specifying a file name. > > The help message of `jcmd PID help Compiler.perfmap` will be updated in a separate PR. Yi-Fan Tsai has updated the pull request incrementally with one additional commit since the last revision: Chagne an option to an argument ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15871/files - new: https://git.openjdk.org/jdk/pull/15871/files/861bda74..61d6f6f4 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15871&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/15871.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15871/head:pull/15871 PR: https://git.openjdk.org/jdk/pull/15871 From sspitsyn at openjdk.org Tue Nov 21 23:35:09 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Tue, 21 Nov 2023 23:35:09 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v3] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 20 Nov 2023 19:28:13 GMT, Jiangli Zhou wrote: >> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. > > Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: > > Add a check for a thread is_attaching_via_jni, based on David Holmes' comment. src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 100: > 98: assert(state->get_thread_oop() != nullptr, "incomplete state"); > 99: } > 100: #endif Nit: I would suggest to write this assert in the form: // Make sure we don't see an incomplete state. An incomplete state can cause // a duplicate JvmtiThreadState being created below and bound to the 'thread' // incorrectly, which leads to stale JavaThread* from the JvmtiThreadState // after the thread exits. assert(state == nullptr || state->get_thread_oop() != nullptr, "incomplete state"); The `#ifdef ASSERT` and `#endif` are not needed then. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1401332452 From david.holmes at oracle.com Wed Nov 22 01:05:17 2023 From: david.holmes at oracle.com (David Holmes) Date: Wed, 22 Nov 2023 11:05:17 +1000 Subject: Handling of DTraceMethodProbes In-Reply-To: <6f99ca60-4317-46c4-95ea-062190f603fd@oracle.com> References: <6f99ca60-4317-46c4-95ea-062190f603fd@oracle.com> Message-ID: On 22/11/2023 3:09 am, Vladimir Kozlov wrote: > CCing to serviceability group. > > Do we still support dtrace? A qualified "yes" - the hooks still exist, mainly for systemTap IIRC but we don't actively update or maintain anything so I expect there is some bitrot. > Thanks, > Vladimir K > > On 11/21/23 4:34 AM, Thomas St?fe wrote: >> Hi, >> >> It seems that the handling of DTraceMethodProbes is inconsistent: in >> most places, we handle the flag at runtime by generating code that >> uses conditional jumps. In one place (C1), we don't. There, generation >> is conditional on DTraceMethodProbes at compile time. Yeah this code in C1 seems wrong: // Change in DTrace flags may invalidate compilation. if (!failing() && ( (!dtrace_method_probes() && DTraceMethodProbes) || (!dtrace_alloc_probes() && DTraceAllocProbes) )) { record_failure("DTrace flags change invalidated dependencies"); } The flag can't change dynamically. Maybe once upon a time it could ? David ----- >> >> The former looks like it is possible for the value >> of?DTraceMethodProbes to change at runtime, and that generated code >> should be able to react seamlessly. However, DTraceMethodProbes is not >> a manageable flag. >> >> Which side is right? Do we need to emit conditional jumps, but if so, >> why not in C1? >> >> Thanks, Thomas From sspitsyn at openjdk.org Wed Nov 22 01:10:07 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 22 Nov 2023 01:10:07 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v2] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Fri, 17 Nov 2023 02:51:03 GMT, Jiangli Zhou wrote: >> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. > >> Thanks. The latest change to `JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample()` looks OK to me. Skipping a few allocations for JVMTI allocation sampler is better than resulting in a problematic `JvmtiThreadState` instance. >> >> My main question is if we can now change `if (state == nullptr || state->get_thread_oop() != thread_oop) ` to `if (state == nullptr)` in `JvmtiThreadState::state_for_while_locked()`. I suspect we would never run into a case of `state != nullptr && state->get_thread_oop() != thread_oop` with the latest change, even with virtual threads. This is backed up by testing with [00ace66](https://github.com/openjdk/jdk/commit/00ace66c36243671a0fb1b673b3f9845460c6d22) not triggering any failure. >> >> If we run into such as a case, it could still be problematic as `JvmtiThreadState::state_for_while_locked()` would allocate a new `JvmtiThreadState` instance pointing to the same JavaThread, and it does not delete the existing instance. >> >> Could anyone with deep knowledge on JvmtiThreadState and virtual threads provide some feedback on this change and https://bugs.openjdk.org/browse/JDK-8319935? @AlanBateman, do you know who would be the best reviewer for this? > > @caoman and I discussed about his suggestion on changing `if (state == nullptr || state->get_thread_oop() != thread_oop)` check in person today. Since it may affect vthread, my main concern is that our current testing may not cover that sufficiently. The suggestion could be worked by a separate enhancement bug. > > @jianglizhou - I fixed a typo in the bug's synopsis line. Change this PR's title: s/is create/is created/ > Thanks, @dcubed-ojdk! Now, the PR title needs to be fixed accordingly. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1821932355 From sspitsyn at openjdk.org Wed Nov 22 01:27:06 2023 From: sspitsyn at openjdk.org (Serguei Spitsyn) Date: Wed, 22 Nov 2023 01:27:06 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v3] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Mon, 20 Nov 2023 19:28:13 GMT, Jiangli Zhou wrote: >> Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. > > Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: > > Add a check for a thread is_attaching_via_jni, based on David Holmes' comment. Thank you for filing and fixing this issue! I'm kind of late here. Sorry for that. Is it hard to create a JTreg test for an attaching native thread? I can help if you have a standalone prototype. You can look for some examples in the folder: `test/hotspot/jtreg/serviceability/jvmti/vthread`. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1821944429 From dholmes at openjdk.org Wed Nov 22 01:30:05 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 22 Nov 2023 01:30:05 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Mon, 20 Nov 2023 21:48:11 GMT, Coleen Phillimore wrote: >> The method holder is an `InstanceKlass` object which can be retrieved as `method->method_holder()` (I apologize if I am using not completely correct terms - this is what I grokked from the sources). And incomplete methods created by the `ClassParser` from the class data stream will not have the link to that `InstanceKlass` set up if the `ClassParser` is already having its `_klass` field set to a non-null value. >> >> If we are talking about clearing any jmetbodIDs associated with an `InstanceKlass` instance it is not really possible for old method versions because only the current `InstanceKlass` version has the jmethodID cache associated with it and it contains jmethodIDs pointing to bot the old and current methods. > > I see, holder is the right word and concept. So the parameter means has_method_holder, in that the InstanceKlass has been fully parsed at the point of clearing the jmethodIDs. Can't we just check `method->method_holder()` for null rather than passing in a parameter like this? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1401396222 From dholmes at openjdk.org Wed Nov 22 01:30:09 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 22 Nov 2023 01:30:09 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes In-Reply-To: References: Message-ID: On Tue, 14 Nov 2023 17:56:09 GMT, Jaroslav Bachorik wrote: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ test/hotspot/jtreg/serviceability/jvmti/thread/GetStackTrace/GetStackTraceAndRetransformTest/GetStackTraceAndRetransformTest.java line 2: > 1: /* > 2: * Copyright (c) 2023 Oracle and/or its affiliates. All rights reserved. An Oracle copyright is not needed here if you wrote this test from scratch. If it is present then we need a comma after the copyright year please. test/hotspot/jtreg/serviceability/jvmti/thread/GetStackTrace/GetStackTraceAndRetransformTest/GetStackTraceAndRetransformTest.java line 29: > 27: * @bug 8313816 > 28: * @summary Test that a sequence of method retransformation and stacktrace capture while the old method > 29: * version is still on stack does not lead to a crash when that's method jmethodID is used as typo: that's method -> that method's test/hotspot/jtreg/serviceability/jvmti/thread/GetStackTrace/GetStackTraceAndRetransformTest/libGetStackTraceAndRetransformTest.cpp line 2: > 1: /* > 2: * Copyright (c) 2023 Oracle and/or its affiliates. All rights reserved. Ditto comment about Oracle copyright. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1401393016 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1401393442 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1401393794 From manc at openjdk.org Wed Nov 22 02:22:24 2023 From: manc at openjdk.org (Man Cao) Date: Wed, 22 Nov 2023 02:22:24 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> References: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> Message-ID: On Tue, 21 Nov 2023 21:42:39 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Update memory tracking type for CPUTimeCounters > - Fix assertion logic Looks pretty clean. Only a few minor cleanups remain. src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 71: > 69: #include "oops/oop.inline.hpp" > 70: #include "runtime/atomic.hpp" > 71: #include "runtime/cpuTimeCounters.hpp" This include could be removed. src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp line 38: > 36: #include "memory/allocation.inline.hpp" > 37: #include "memory/iterator.hpp" > 38: #include "runtime/cpuTimeCounters.hpp" This could be removed too. src/hotspot/share/gc/g1/g1ServiceThread.cpp line 26: > 24: > 25: #include "precompiled.hpp" > 26: #include "gc/g1/g1CollectedHeap.hpp" Could this include be removed? src/hotspot/share/gc/g1/g1ServiceThread.hpp line 30: > 28: #include "gc/shared/concurrentGCThread.hpp" > 29: #include "runtime/mutex.hpp" > 30: #include "runtime/perfData.hpp" This could be removed. src/hotspot/share/gc/shared/collectedHeap.cpp line 52: > 50: #include "oops/instanceMirrorKlass.hpp" > 51: #include "oops/oop.inline.hpp" > 52: #include "runtime/atomic.hpp" This could be removed. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.cpp line 28: > 26: #include "classfile/javaClasses.inline.hpp" > 27: #include "classfile/stringTable.hpp" > 28: #include "gc/shared/collectedHeap.hpp" This include as well as the include for perfData.hpp (line 45) could be removed. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.cpp line 70: > 68: void StringDedup::Processor::initialize() { > 69: _processor = new Processor(); > 70: if (UsePerfData && os::is_thread_cpu_time_supported()) { The if and EXCEPTION_MARK could be removed, because `create_counter()` does that internally. src/hotspot/share/gc/shared/stringdedup/stringDedupProcessor.hpp line 30: > 28: #include "gc/shared/stringdedup/stringDedup.hpp" > 29: #include "memory/allocation.hpp" > 30: #include "runtime/perfData.hpp" This could be removed. src/hotspot/share/runtime/cpuTimeCounters.hpp line 97: > 95: class ThreadTotalCPUTimeClosure: public ThreadClosure { > 96: private: > 97: jlong _gc_total; _total is a more appropriate name for this field. src/hotspot/share/runtime/vmThread.cpp line 140: > 138: PerfDataManager::create_counter(SUN_THREADS, "vmOperationTime", > 139: PerfData::U_Ticks, CHECK); > 140: if (os::is_thread_cpu_time_supported()) { The if could be remove, as `create_counter()` checks it internally. ------------- Marked as reviewed by manc (Committer). PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1743425918 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401413936 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401415033 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401416191 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401416622 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401416969 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401417321 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401418586 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401420582 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401410366 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1401419999 From dholmes at openjdk.org Wed Nov 22 02:56:06 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 22 Nov 2023 02:56:06 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v3] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Tue, 21 Nov 2023 22:45:54 GMT, Serguei Spitsyn wrote: >> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: >> >> Add a check for a thread is_attaching_via_jni, based on David Holmes' comment. > > src/hotspot/share/prims/jvmtiExport.cpp line 3144: > >> 3142: // If the current thread is attaching from native and its thread oop is being >> 3143: // allocated, things are not ready for allocation sampling. >> 3144: if (thread->is_Java_thread()) { > > Nit: There is no need for this check at line 3144. > There was already check for `!thread->is_Java_thread()` and return with false at line 3138: > > if (!thread->is_Java_thread() || thread->is_Compiler_thread()) { > return false; > } +1 ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1401442209 From dholmes at openjdk.org Wed Nov 22 05:10:05 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 22 Nov 2023 05:10:05 GMT Subject: RFR: 8320536: problemlist failing serviceability/attach/ConcAttachTest.java test on macosx In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 20:16:39 GMT, Chris Plummer wrote: > Problemlisting ConcAttachTest.java on OSX (it's already problem listed on linux and doesn't get run on Windows). This one test seems to be causing all the issues with attach and dcmd tests described in [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866), so if we problem list it we should stop seeing the other test failures. Looks good. ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16772#pullrequestreview-1743575066 From azafari at openjdk.org Wed Nov 22 08:41:35 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Wed, 22 Nov 2023 08:41:35 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v9] In-Reply-To: References: Message-ID: > The `find` method now is > ```C++ > template > int find(T* token, bool f(T*, E)) const { > ... > > Any other functions which use this are also changed. > Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: find methods accepts Function and callers provide lambda. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15418/files - new: https://git.openjdk.org/jdk/pull/15418/files/7665b878..9973f5d7 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=07-08 Stats: 56 lines in 9 files changed: 21 ins; 17 del; 18 mod Patch: https://git.openjdk.org/jdk/pull/15418.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15418/head:pull/15418 PR: https://git.openjdk.org/jdk/pull/15418 From stefank at openjdk.org Wed Nov 22 14:36:17 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Wed, 22 Nov 2023 14:36:17 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v9] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 08:41:35 GMT, Afshin Zafari wrote: >> The `find` method now is >> ```C++ >> template >> int find(T* token, bool f(T*, E)) const { >> ... >> >> Any other functions which use this are also changed. >> Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. > > Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: > > find methods accepts Function and callers provide lambda. Thanks for making this change. I'd like to suggest the following cleanups, some documentation, and a few tests: https://github.com/openjdk/jdk/commit/20d4502471ba396ae395512cfa3dab3f87555421 I think it might be easier to review by looking at the final diff: https://github.com/openjdk/jdk/compare/master...stefank:jdk:pr_15418 ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1822887111 From stefank at openjdk.org Wed Nov 22 15:07:34 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Wed, 22 Nov 2023 15:07:34 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object Message-ID: In the rewrites made for: [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. The provided tests provoke this assert form: * the JNI thread detach code * thread dumping with locked monitors, and * the JVMTI GetOwnedMonitorInfo API. While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. Test: the written tests with and without the fix. Tier1-Tier3, so far. ------------- Commit messages: - 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object Changes: https://git.openjdk.org/jdk/pull/16783/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320515 Stats: 258 lines in 8 files changed: 253 ins; 1 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16783/head:pull/16783 PR: https://git.openjdk.org/jdk/pull/16783 From duke at openjdk.org Wed Nov 22 16:19:30 2023 From: duke at openjdk.org (suchismith1993) Date: Wed, 22 Nov 2023 16:19:30 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX Message-ID: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> JDK-8320005 : Native library suffix impact on hotspot code in AIX ------------- Commit messages: - Adapt hotspot coding style - Improve comments and coding style. - Remove macro for file extension. - Move mapping function to aix specific file. - Introduce new macro for AIX archives. - Add support for .a extension in jvm agent. Changes: https://git.openjdk.org/jdk/pull/16604/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320005 Stats: 24 lines in 3 files changed: 24 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16604.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16604/head:pull/16604 PR: https://git.openjdk.org/jdk/pull/16604 From duke at openjdk.org Wed Nov 22 16:24:24 2023 From: duke at openjdk.org (suchismith1993) Date: Wed, 22 Nov 2023 16:24:24 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> Message-ID: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> > JDK-8320005 : Native library suffix impact on hotspot code in AIX suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: change macro position ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16604/files - new: https://git.openjdk.org/jdk/pull/16604/files/6fdfba81..077083d2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16604&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16604.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16604/head:pull/16604 PR: https://git.openjdk.org/jdk/pull/16604 From stuefe at openjdk.org Wed Nov 22 16:38:11 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Wed, 22 Nov 2023 16:38:11 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Wed, 22 Nov 2023 16:24:24 GMT, suchismith1993 wrote: >> JDK-8320005 : Native library suffix impact on hotspot code in AIX > > suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: > > change macro position Hi, is this patch meant for review already? If yes, could you please describe the problem you fix, and how you fix it? If no, I suggest working on it in draft state till its ready for review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1823105203 From cjplummer at openjdk.org Wed Nov 22 18:02:21 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 22 Nov 2023 18:02:21 GMT Subject: Integrated: 8320536: problemlist failing serviceability/attach/ConcAttachTest.java test on macosx In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 20:16:39 GMT, Chris Plummer wrote: > Problemlisting ConcAttachTest.java on OSX (it's already problem listed on linux and doesn't get run on Windows). This one test seems to be causing all the issues with attach and dcmd tests described in [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866), so if we problem list it we should stop seeing the other test failures. This pull request has now been integrated. Changeset: 572b14ac Author: Chris Plummer URL: https://git.openjdk.org/jdk/commit/572b14ac8697497d9c0aefe92864075e712c171e Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8320536: problemlist failing serviceability/attach/ConcAttachTest.java test on macosx Reviewed-by: amenkov, dholmes ------------- PR: https://git.openjdk.org/jdk/pull/16772 From cjplummer at openjdk.org Wed Nov 22 18:02:20 2023 From: cjplummer at openjdk.org (Chris Plummer) Date: Wed, 22 Nov 2023 18:02:20 GMT Subject: RFR: 8320536: problemlist failing serviceability/attach/ConcAttachTest.java test on macosx In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 20:16:39 GMT, Chris Plummer wrote: > Problemlisting ConcAttachTest.java on OSX (it's already problem listed on linux and doesn't get run on Windows). This one test seems to be causing all the issues with attach and dcmd tests described in [JDK-8318866](https://bugs.openjdk.org/browse/JDK-8318866), so if we problem list it we should stop seeing the other test failures. Thank you Alex and David! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16772#issuecomment-1823234948 From jiangli at openjdk.org Wed Nov 22 22:18:23 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Wed, 22 Nov 2023 22:18:23 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v4] In-Reply-To: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: > Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. Jiangli Zhou has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Merge branch 'master' into JDK-8319935 - Add a check for a thread is_attaching_via_jni, based on David Holmes' comment. - Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. - 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16642/files - new: https://git.openjdk.org/jdk/pull/16642/files/7c0214e2..de7fac6d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=02-03 Stats: 30285 lines in 831 files changed: 17825 ins; 7190 del; 5270 mod Patch: https://git.openjdk.org/jdk/pull/16642.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16642/head:pull/16642 PR: https://git.openjdk.org/jdk/pull/16642 From jiangli at openjdk.org Wed Nov 22 22:40:20 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Wed, 22 Nov 2023 22:40:20 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v5] In-Reply-To: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: > Please review JvmtiThreadState::state_for_while_locked change to handle the state->get_thread_oop() null case. Please see https://bugs.openjdk.org/browse/JDK-8319935 for details. Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: Address Serguei Spitsyn's comments/suggestions: - Remove the redundant thread->is_Java_thread() check from JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample(). - Change the assert in JvmtiThreadState::state_for_while_locked to avoid #ifdef ASSERT. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16642/files - new: https://git.openjdk.org/jdk/pull/16642/files/de7fac6d..7c366df0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16642&range=03-04 Stats: 12 lines in 2 files changed: 0 ins; 5 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/16642.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16642/head:pull/16642 PR: https://git.openjdk.org/jdk/pull/16642 From jiangli at openjdk.org Wed Nov 22 22:40:22 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Wed, 22 Nov 2023 22:40:22 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v3] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: <-_KrlwZh4w1qg_mnKcb4OZYNuLf3syK0nrPhZhTXF9I=.1bae1a31-defa-4eac-857f-ae9d28b16b38@github.com> On Wed, 22 Nov 2023 02:53:23 GMT, David Holmes wrote: >> src/hotspot/share/prims/jvmtiExport.cpp line 3144: >> >>> 3142: // If the current thread is attaching from native and its thread oop is being >>> 3143: // allocated, things are not ready for allocation sampling. >>> 3144: if (thread->is_Java_thread()) { >> >> Nit: There is no need for this check at line 3144. >> There was already check for `!thread->is_Java_thread()` and return with false at line 3138: >> >> if (!thread->is_Java_thread() || thread->is_Compiler_thread()) { >> return false; >> } > > +1 Indeed, removed. Thanks. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1402770901 From jiangli at openjdk.org Wed Nov 22 22:40:25 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Wed, 22 Nov 2023 22:40:25 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is create for one JavaThread associated with attached native thread [v3] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Tue, 21 Nov 2023 23:32:13 GMT, Serguei Spitsyn wrote: >> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: >> >> Add a check for a thread is_attaching_via_jni, based on David Holmes' comment. > > src/hotspot/share/prims/jvmtiThreadState.inline.hpp line 100: > >> 98: assert(state->get_thread_oop() != nullptr, "incomplete state"); >> 99: } >> 100: #endif > > Nit: I would suggest to write this assert in the form: > > // Make sure we don't see an incomplete state. An incomplete state can cause > // a duplicate JvmtiThreadState being created below and bound to the 'thread' > // incorrectly, which leads to stale JavaThread* from the JvmtiThreadState > // after the thread exits. > assert(state == nullptr || state->get_thread_oop() != nullptr, "incomplete state"); > > The `#ifdef ASSERT` and `#endif` are not needed then. Changed as suggested. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16642#discussion_r1402771379 From jiangli at openjdk.org Wed Nov 22 22:51:07 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Wed, 22 Nov 2023 22:51:07 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread [v2] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Fri, 17 Nov 2023 02:51:03 GMT, Jiangli Zhou wrote: >> Jiangli Zhou has updated the pull request incrementally with one additional commit since the last revision: >> >> Don't try to setup_jvmti_thread_state for obj allocation sampling if the current thread is attaching from native and is allocating the thread oop. That's to make sure we don't create a 'partial' JvmtiThreadState. > >> Thanks. The latest change to `JvmtiSampledObjectAllocEventCollector::object_alloc_is_safe_to_sample()` looks OK to me. Skipping a few allocations for JVMTI allocation sampler is better than resulting in a problematic `JvmtiThreadState` instance. >> >> My main question is if we can now change `if (state == nullptr || state->get_thread_oop() != thread_oop) ` to `if (state == nullptr)` in `JvmtiThreadState::state_for_while_locked()`. I suspect we would never run into a case of `state != nullptr && state->get_thread_oop() != thread_oop` with the latest change, even with virtual threads. This is backed up by testing with [00ace66](https://github.com/openjdk/jdk/commit/00ace66c36243671a0fb1b673b3f9845460c6d22) not triggering any failure. >> >> If we run into such as a case, it could still be problematic as `JvmtiThreadState::state_for_while_locked()` would allocate a new `JvmtiThreadState` instance pointing to the same JavaThread, and it does not delete the existing instance. >> >> Could anyone with deep knowledge on JvmtiThreadState and virtual threads provide some feedback on this change and https://bugs.openjdk.org/browse/JDK-8319935? @AlanBateman, do you know who would be the best reviewer for this? > > @caoman and I discussed about his suggestion on changing `if (state == nullptr || state->get_thread_oop() != thread_oop)` check in person today. Since it may affect vthread, my main concern is that our current testing may not cover that sufficiently. The suggestion could be worked by a separate enhancement bug. > > > @jianglizhou - I fixed a typo in the bug's synopsis line. Change this PR's title: s/is create/is created/ > > > Thanks, @dcubed-ojdk! > > Now, the PR title needs to be fixed accordingly. Done, thanks for the reminder! ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1823599300 From jjoo at openjdk.org Wed Nov 22 23:08:36 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 22 Nov 2023 23:08:36 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: > 8315149: Add hsperf counters for CPU time of internal GC threads Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: Cleanup and address comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15082/files - new: https://git.openjdk.org/jdk/pull/15082/files/4ca30f32..fcc7e471 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=46 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15082&range=45-46 Stats: 19 lines in 10 files changed: 0 ins; 13 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/15082.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15082/head:pull/15082 PR: https://git.openjdk.org/jdk/pull/15082 From jjoo at openjdk.org Wed Nov 22 23:08:38 2023 From: jjoo at openjdk.org (Jonathan Joo) Date: Wed, 22 Nov 2023 23:08:38 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v46] In-Reply-To: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> References: <4S_iyhdkwxpMar7tdNxHobR6vaRcqKcikQQrrcNBwX0=.29697f18-b7cc-44a9-8900-90f7a3a1e780@github.com> Message-ID: On Tue, 21 Nov 2023 21:42:39 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with two additional commits since the last revision: > > - Update memory tracking type for CPUTimeCounters > - Fix assertion logic Addressed cleanups, PR should be RFR! ------------- PR Comment: https://git.openjdk.org/jdk/pull/15082#issuecomment-1823609497 From jiangli at openjdk.org Thu Nov 23 00:02:08 2023 From: jiangli at openjdk.org (Jiangli Zhou) Date: Thu, 23 Nov 2023 00:02:08 GMT Subject: RFR: 8319935: Ensure only one JvmtiThreadState is created for one JavaThread associated with attached native thread [v3] In-Reply-To: References: <3GC3ckHJhlYl3Vj_oh7DJorPy8NneY9rw2EMBQeyFvY=.6c7281cd-f97d-4805-8695-ddd66e5e6415@github.com> Message-ID: On Wed, 22 Nov 2023 01:24:46 GMT, Serguei Spitsyn wrote: > Thank you for filing and fixing this issue! I'm kind of late here. Sorry for that. Is it hard to create a JTreg test for an attaching native thread? I can help if you have a standalone prototype. You can look for some examples in the folder: `test/hotspot/jtreg/serviceability/jvmti/vthread`. Hi @sspitsyn we don't have an extracted standalone test case (yet) to demonstrate the crashes. The crashes could not reproduce consistently. Outside the debugger (lldb), I ran the test (one of the affected ones) 10 times/per-iteration in order to reproduce. I found the crashes could be affected by both timing and memory layout. During the investigation, I noticed the problem became hidden when I increased allocation size for ThreadsList::_threads (as one of the experiments that I did, I wanted to mprotect the memory to be read-only in order to find who trashed the memory, so was trying to allocate memory up to page boundary). That's the reason why I added noreg-hard tag earlier. I gave some more thoughts today. Perhaps, we could write a whitebox test to check the JvmtiThreadState, without being able to consistently trigger crashes. We could add a WhiteBox api to iterate the JvmtiThreadState list and validate if all the JavaThread pointers were valid after detaching. The test would need to create native threads to attach and detach before the check. That could more reliably test the 1-1 mapping of JvmtiThreadState and JavaThread. What do you think? Thanks for volunteering to help with the test. I created https://bugs.openjdk.org/browse/JDK-8320614 today. Should I assign it to you? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16642#issuecomment-1823672341 From dholmes at openjdk.org Thu Nov 23 00:51:04 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 23 Nov 2023 00:51:04 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 15:00:29 GMT, Stefan Karlsson wrote: > Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code I had not realized that. It explains some confusion in a separate issue I had been looking into! It is important that these monitors are exposed and unlocked at detach time, otherwise it also messes up the `held_monitor_count`. > The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure I think we may need to make that code tolerate the absence of an object. > For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. I think we probably should expose this to be accurate, but I think this needs investigation on the JVMTI side to ensure that the null entry is tolerated okay. So a separate RFE to handle this would be fine. Thanks ------------- PR Comment: https://git.openjdk.org/jdk/pull/16783#issuecomment-1823702358 From dholmes at openjdk.org Thu Nov 23 02:43:15 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 23 Nov 2023 02:43:15 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 15:00:29 GMT, Stefan Karlsson wrote: > In the rewrites made for: > [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` > > I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. > > The provided tests provoke this assert form: > * the JNI thread detach code > * thread dumping with locked monitors, and > * the JVMTI GetOwnedMonitorInfo API. > > While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. > > The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. > > For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. > > Test: the written tests with and without the fix. Tier1-Tier3, so far. Functional fix is simple and fine but quite a lot of commentary on the tests. Thanks src/hotspot/share/runtime/vmOperations.cpp line 354: > 352: // alive. Filter out monitors with dead objects. > 353: return; > 354: } I don't think we need to do this, but even without this filtering I ran a number of tests and was unable to demonstrate any problem. The JNI locked monitor seems to be "invisible" to the frame that locked it and so the thread dump never encounters it. Were you able to provoke a failure here or is this defensive programming? test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 28: > 26: * @test IterateMonitorWithDeadObjectTest > 27: * @summary This locks a monitor, GCs the object, and iterate and perform > 28: * various iteration and operations over this monitor. This doesn't read right with "iterate" and "iteration". Not sure exactly what you were trying to say. test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 29: > 27: * @summary This locks a monitor, GCs the object, and iterate and perform > 28: * various iteration and operations over this monitor. > 29: * @requires os.family == "linux" I know the test this was copied from had this but I'm not sure it is actually a necessary restriction - any Posix platform should work. Though maybe perror is linux only ... test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 31: > 29: * @requires os.family == "linux" > 30: * @library /testlibrary /test/lib > 31: * @build IterateMonitorWithDeadObjectTest You don't need an explicit `@build` step test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 40: > 38: public class IterateMonitorWithDeadObjectTest { > 39: public static native void runTestAndDetachThread(); > 40: public static native void joinTestThread(); I don't think this form of the test needs to separate out the `pthread_join()`, it can just be done in `runTestAndDetachThread` AFAICS. I originally split it out to allow the Java code to do the GC while the native thread was sleeping prior to detaching. test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 57: > 55: // - Drop the last reference to the object > 56: // - GC to clear the weak reference to the object in the monitor > 57: // - Detach the thread - provoke previous bug It also does a thread dump while the lock is held test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 66: > 64: // dead object. The thread dumping code didn't tolerate such a monitor, > 65: // so run a thread dump and make sure that it doesn't crash/assert. > 66: dumpThreadsWithLockedMonitors(); But you've already detached the thread so there is no locked monitor any longer. And `runTestAndDetachThread()` also did a thread dump. test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 43: > 41: static jobject create_object(JNIEnv* env) { > 42: jclass clazz = (*env)->FindClass(env, "java/lang/Object"); > 43: if (clazz == 0) die("No class"); The `die` method is for errors with system calls. It won't show useful information for JNI calls that leave exceptions pending. test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 76: > 74: if (dumpAllThreadsMethod == 0) die("No dumpAllThreads method"); > 75: > 76: // The 'lockedMonitors == true' is what triggers the collection of the monitor with the dead object. "triggers the collection" sounds like a GC interaction but that is not what you mean. Suggestion: // The 'lockedMonitors == true' is what causes the monitor with a dead object to be examined. test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 94: > 92: > 93: // Let the GC clear the weak reference to the object. > 94: system_gc(env); AFAIK there is no guarantee that one call to `System.gc()` will suffice to clear the weakRef. We tend use a loop with a few iterations in other tests, or use a WhiteBox method to achieve it. In my testing I used the finalizer to observe that the objects had been finalized but even then, and with a loop, I did not always see them collected with G1. test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 104: > 102: // source of at least two bugs: > 103: // - When the object reference in the monitor was made weak, the code > 104: // didn't unlock the monitor, leaving it lingering in the system. Suggestion: // - When the object reference in the monitor was cleared, the monitor // iterator code would skip it, preventing it from being unlocked when // the owner thread detached, leaving it lingering in the system. the original made it sound to me like the code that cleared the reference (i.e. the GC) was expected to do the unlocking. test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 107: > 105: // - When the monitor iterator API was rewritten the code was changed to > 106: // assert that we didn't have "owned" monitors with dead objects. This > 107: // test provokes that situation and those asserts. nit: s/those asserts/that assert/ test/hotspot/jtreg/serviceability/jvmti/GetOwnedMonitorInfo/GetOwnedMonitorInfoTest.java line 54: > 52: > 53: private static void jniMonitorEnterAndLetObjectDie() { > 54: // The monitor iterator used GetOwnedMonitorInfo used to s/iterator used/iterator used by/ test/hotspot/jtreg/serviceability/jvmti/GetOwnedMonitorInfo/GetOwnedMonitorInfoTest.java line 59: > 57: // GetOwnedMonitorInfo testing. > 58: Object obj = new Object() { public String toString() {return "";} }; > 59: jniMonitorEnter(obj); I would add a check for `Thread.holdsLock(obj);` after this just to be sure it worked. test/hotspot/jtreg/serviceability/jvmti/GetOwnedMonitorInfo/GetOwnedMonitorInfoTest.java line 61: > 59: jniMonitorEnter(obj); > 60: obj = null; > 61: System.gc(); Again one gc() is generally not sufficient. How can this test tell that the object in the monitor was actually cleared? I think `monitorinflation` logging may be the only way to tell. ------------- Changes requested by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16783#pullrequestreview-1745590548 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402843318 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402843678 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402843923 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402844118 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402846787 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402846942 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402845852 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402857972 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402859246 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402847898 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402848741 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402848946 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402851705 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402857110 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1402852983 From duke at openjdk.org Thu Nov 23 05:27:08 2023 From: duke at openjdk.org (suchismith1993) Date: Thu, 23 Nov 2023 05:27:08 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Wed, 22 Nov 2023 16:35:36 GMT, Thomas Stuefe wrote: > Hi, is this patch meant for review already? If yes, could you please describe the problem you fix, and how you fix it? If no, I suggest working on it in draft state till its ready for review. I have updated the description. Let me know if anything is missing ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1823834982 From amitkumar at openjdk.org Thu Nov 23 05:47:10 2023 From: amitkumar at openjdk.org (Amit Kumar) Date: Thu, 23 Nov 2023 05:47:10 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Wed, 22 Nov 2023 16:24:24 GMT, suchismith1993 wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: > > change macro position some nits you might want to consider. src/hotspot/os/aix/os_aix.cpp line 3064: > 3062: > 3063: //Replaces provided path with alternate path for the given file,if it doesnt exist. > 3064: //For AIX,this replaces .so with .a. Suggestion: // Replaces the specified path with an alternative path for the given file if the original path doesn't exist. // For AIX, this replaces extension from ".so" to ".a". src/hotspot/os/aix/os_aix.cpp line 3065: > 3063: //Replaces provided path with alternate path for the given file,if it doesnt exist. > 3064: //For AIX,this replaces .so with .a. > 3065: void os::Aix::mapAlternateName(char* buffer, const char *extension) { Suggestion: void os::Aix::map_alternate_name(char* buffer, const char *extension) { src/hotspot/os/aix/os_aix.hpp line 181: > 179: static int stat64x_via_LIBPATH(const char* path, struct stat64x* stat); > 180: // Provide alternate path name,if file does not exist. > 181: static void mapAlternateName(char* buffer, const char *extension); Suggestion: // Provides alternate path name, if file does not exist. static void map_alternate_name(char* buffer, const char *extension); ------------- Changes requested by amitkumar (Committer). PR Review: https://git.openjdk.org/jdk/pull/16604#pullrequestreview-1745734586 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402935976 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402936171 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402936497 From amitkumar at openjdk.org Thu Nov 23 05:52:05 2023 From: amitkumar at openjdk.org (Amit Kumar) Date: Thu, 23 Nov 2023 05:52:05 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: <2X-eefx1BKO-hqncC3LF0It1wUxHRqvrX0qm2TsCbd0=.2626d8a4-d549-4a82-a2de-40003af36539@github.com> On Wed, 22 Nov 2023 16:24:24 GMT, suchismith1993 wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: > > change macro position Also are you planning to close this one : https://github.com/openjdk/jdk/pull/16490 ? JBS issue is already closed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1823849323 From dholmes at openjdk.org Thu Nov 23 06:04:06 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 23 Nov 2023 06:04:06 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Wed, 22 Nov 2023 16:24:24 GMT, suchismith1993 wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: > > change macro position A couple of comments. First, if you always want to look for `libfoo.a` if you can't find `libfoo.so` then maybe that should be handled as the `os::dll_load` level? Otherwisae it is not clear why this is something you only do for agents. ?/ Second, the amount of AIX-specific code in the shared `jvmtiAgent.cpp` now seems unreasonable. As I think @tstuefe mentioned in another PR perhaps it is time to find better abstractions here to hide all these AIX specific quirks? src/hotspot/os/aix/os_aix.cpp line 3071: > 3069: } > 3070: buffer[end] = '\0'; > 3071: strcat(buffer, extension); At some point you need to check the length of extension won't overflow buffer. ------------- PR Review: https://git.openjdk.org/jdk/pull/16604#pullrequestreview-1745744207 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402941846 From stuefe at openjdk.org Thu Nov 23 06:20:09 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 23 Nov 2023 06:20:09 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Wed, 22 Nov 2023 16:24:24 GMT, suchismith1993 wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: > > change macro position I'm not a big fan of this approach. We accumulate more and more "#ifdef AIX" in shared code because of many recent AIX additions. No other platform has such a large ifdef footprint in shared code. I argue that all of this should be handled inside os_aix.cpp and not leak out into the external space: If .a is a valid shared object format on AIX, this should be handled in `os::dll_load()`, and be done for all shared objects. If not, why do we try to load a static archive via dlload in this case but not in other cases? *If* this is needed in shared code, the string replacement function should be a generic utility function for all platforms, and it should be tested with a small gtest. A gtest would have likely uncovered the buffer overflow too. src/hotspot/os/aix/os_aix.cpp line 3065: > 3063: //Replaces provided path with alternate path for the given file,if it doesnt exist. > 3064: //For AIX,this replaces .so with .a. > 3065: void os::Aix::mapAlternateName(char* buffer, const char *extension) { The documentation is wrong: // Replaces the specified path with an alternative path for the given file if the original path doesn't exist It does no such thing, it replaces the extension unconditionally. The comment sounds like it does a file system check. That does not happen here. The whole function is not well named - "map alternate name" does not really tell me anything, I need to look at the implementation and the caller to understand what it is doing. There is no mapping here, this is just a string utility function. The function should not modify the original buffer but instead assemble a copy. That is the conventional way to do these things. You can work with immutable strings as input, e.g. literals, and don't risk buffer overflows. All of this should be handled inside os_aix.cpp; see my other comment. This should not live in the external os::aix interface, since it has nothing to do with AIX. *If* this is needed in generic code, which I don't think, then this should be made generic utility API, available on all platforms, and with a small gtest. But I think all of this should be confined to os_aix.cpp. Proposal for a clearer name, comment, and pseudocode // Given a filename with an extension, return a new string containing the filename with the new extension. // New string is allocated in resource area. static char* replace_extension_in_filename(const char* filename, const char* new_extension) { - allocate buffer in RA - assemble new path by contacting old path - old extension + new extension - return new path } src/hotspot/os/aix/os_aix.cpp line 3069: > 3067: while (end > 0 && buffer[end] != '.') { > 3068: end = end - 1; > 3069: } Use strrchr. src/hotspot/os/aix/os_aix.cpp line 3072: > 3070: buffer[end] = '\0'; > 3071: strcat(buffer, extension); > 3072: } This is a buffer overrun waiting to happen if replacement is larger than original extension. ------------- Changes requested by stuefe (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16604#pullrequestreview-1745743102 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402944936 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402941240 PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402941730 From stuefe at openjdk.org Thu Nov 23 06:20:11 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 23 Nov 2023 06:20:11 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Thu, 23 Nov 2023 05:55:28 GMT, Thomas Stuefe wrote: >> suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: >> >> change macro position > > src/hotspot/os/aix/os_aix.cpp line 3069: > >> 3067: while (end > 0 && buffer[end] != '.') { >> 3068: end = end - 1; >> 3069: } > > Use strrchr. Pls handle the case where the string contains no dot. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1402941387 From duke at openjdk.org Thu Nov 23 08:11:05 2023 From: duke at openjdk.org (suchismith1993) Date: Thu, 23 Nov 2023 08:11:05 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <2X-eefx1BKO-hqncC3LF0It1wUxHRqvrX0qm2TsCbd0=.2626d8a4-d549-4a82-a2de-40003af36539@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> <2X-eefx1BKO-hqncC3LF0It1wUxHRqvrX0qm2TsCbd0=.2626d8a4-d549-4a82-a2de-40003af36539@github.com> Message-ID: On Thu, 23 Nov 2023 05:49:41 GMT, Amit Kumar wrote: > #16490 The JBS issue with respect to that has been closed. Need to check if that PR is required. Currently putting it on hold. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1823952398 From jbachorik at openjdk.org Thu Nov 23 08:32:36 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 08:32:36 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v2] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with three additional commits since the last revision: - Clean up imports - Simplify Method::clear_jmethod_id() - Use correct copyrights ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/b2ca5e69..f47e9499 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=00-01 Stats: 25 lines in 6 files changed: 5 ins; 13 del; 7 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From stefank at openjdk.org Thu Nov 23 08:44:10 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 08:44:10 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 01:27:24 GMT, David Holmes wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > src/hotspot/share/runtime/vmOperations.cpp line 354: > >> 352: // alive. Filter out monitors with dead objects. >> 353: return; >> 354: } > > I don't think we need to do this, but even without this filtering I ran a number of tests and was unable to demonstrate any problem. The JNI locked monitor seems to be "invisible" to the frame that locked it and so the thread dump never encounters it. Were you able to provoke a failure here or is this defensive programming? I provoked test failures for all paths I filtered. If I remove this check and run: make -C ../build/fastdebug test TEST=test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java JTREG="JAVA_OPTIONS=-XX:+UseZGC" I hit this assert: # Internal Error (/home/stefank/git/jdk/open/src/hotspot/share/services/management.cpp:1274), pid=1546709, tid=1546754 # assert(object != nullptr) failed: must be a Java object ... V [libjvm.so+0x1330ce8] jmm_DumpThreads+0x1a48 (management.cpp:1274) j sun.management.ThreadImpl.dumpThreads0([JZZI)[Ljava/lang/management/ThreadInfo;+0 java.management at 22-internal j sun.management.ThreadImpl.dumpAllThreads(ZZI)[Ljava/lang/management/ThreadInfo;+28 java.management at 22-internal j sun.management.ThreadImpl.dumpAllThreads(ZZ)[Ljava/lang/management/ThreadInfo;+5 java.management at 22-internal j IterateMonitorWithDeadObjectTest.dumpThreadsWithLockedMonitors()V+7 j IterateMonitorWithDeadObjectTest.main([Ljava/lang/String;)V+11 If I remove that assert I hit an NPE in the Java layer: java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "lock" is null at java.management/java.lang.management.ThreadInfo.(ThreadInfo.java:172) at java.management/sun.management.ThreadImpl.dumpThreads0(Native Method) at java.management/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:518) at java.management/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:506) at IterateMonitorWithDeadObjectTest.dumpThreadsWithLockedMonitors(IterateMonitorWithDeadObjectTest.java:44) at IterateMonitorWithDeadObjectTest.main(IterateMonitorWithDeadObjectTest.java:66) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333) at java.base/java.lang.Thread.run(Thread.java:1570) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403052907 From jbachorik at openjdk.org Thu Nov 23 08:49:30 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 08:49:30 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v3] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: Rewerite the test to use RedefineClassHelper ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/f47e9499..967813b6 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=01-02 Stats: 124 lines in 2 files changed: 15 ins; 93 del; 16 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From jbachorik at openjdk.org Thu Nov 23 08:49:32 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 08:49:32 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v2] In-Reply-To: References: Message-ID: <4m-e190SIpRd0-fkjZ4ja7mhVaQQsTwfqmWd8_82Els=.7b198822-3637-4109-b1e9-2451f882ce4b@github.com> On Thu, 23 Nov 2023 08:32:36 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with three additional commits since the last revision: > > - Clean up imports > - Simplify Method::clear_jmethod_id() > - Use correct copyrights @dholmes-ora > Can't we just check method->method_holder() for null rather than passing in a parameter like this? I have removed the argument and I am now performing the check for `method_holder() != nullptr` as recommended. The code is a bit simpler and the cost of resolving the method holder for each method is probably quite low so we should be ok here. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1823997622 From jbachorik at openjdk.org Thu Nov 23 08:49:37 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 08:49:37 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v3] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 01:27:25 GMT, David Holmes wrote: >> I see, holder is the right word and concept. So the parameter means has_method_holder, in that the InstanceKlass has been fully parsed at the point of clearing the jmethodIDs. > > Can't we just check `method->method_holder()` for null rather than passing in a parameter like this? Yes. I changed the code to do that. We will attempt to resolve the method holder for each method instead of just using one single boolean argument but I believe the resolution should be fast enough not to matter in this context. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403068144 From jbachorik at openjdk.org Thu Nov 23 08:49:35 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 08:49:35 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v3] In-Reply-To: References: Message-ID: <-NQyZq95fCsl2kAwAjrQXuxJoKqAdnoI3Y0ccQ9CNsk=.5fe65c5d-c326-45a3-8777-13c5b3f8cf06@github.com> On Mon, 20 Nov 2023 22:08:49 GMT, Coleen Phillimore wrote: >> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: >> >> Rewerite the test to use RedefineClassHelper > > src/hotspot/share/classfile/classFileParser.cpp line 5579: > >> 5577: >> 5578: if (_methods != nullptr) { >> 5579: // Free methods - those methods are not fully wired and miss the method holder > > How about saying: for methods whose InstanceKlass as method holder is not yet created? This comment went away with the change @dholmes-ora recommended > test/hotspot/jtreg/serviceability/jvmti/thread/GetStackTrace/GetStackTraceAndRetransformTest/GetStackTraceAndRetransformTest.java line 53: > >> 51: import java.util.List; >> 52: import java.util.concurrent.CyclicBarrier; >> 53: import java.util.concurrent.locks.LockSupport; > > Do you need all these imports? > > There's a simple RedefineClassHelper class that does most of the work, but maybe you need the explicit agent code to reproduce the crash? See test/hotspot/jtreg/serviceability/jvmti/RedefineClasses/RedefineRunningMethodsWithBacktrace.java as an example. I did clean up the imports and switched to `RedefineClassHelper`. Thanks for pointing that out! ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403064332 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403054332 From stefank at openjdk.org Thu Nov 23 08:50:09 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 08:50:09 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 01:28:32 GMT, David Holmes wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 28: > >> 26: * @test IterateMonitorWithDeadObjectTest >> 27: * @summary This locks a monitor, GCs the object, and iterate and perform >> 28: * various iteration and operations over this monitor. > > This doesn't read right with "iterate" and "iteration". Not sure exactly what you were trying to say. I agree. I'll reword that. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403073457 From jbachorik at openjdk.org Thu Nov 23 08:49:39 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 08:49:39 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v3] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 01:21:15 GMT, David Holmes wrote: >> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: >> >> Rewerite the test to use RedefineClassHelper > > test/hotspot/jtreg/serviceability/jvmti/thread/GetStackTrace/GetStackTraceAndRetransformTest/GetStackTraceAndRetransformTest.java line 2: > >> 1: /* >> 2: * Copyright (c) 2023 Oracle and/or its affiliates. All rights reserved. > > An Oracle copyright is not needed here if you wrote this test from scratch. If it is present then we need a comma after the copyright year please. Fixed > test/hotspot/jtreg/serviceability/jvmti/thread/GetStackTrace/GetStackTraceAndRetransformTest/libGetStackTraceAndRetransformTest.cpp line 2: > >> 1: /* >> 2: * Copyright (c) 2023 Oracle and/or its affiliates. All rights reserved. > > Ditto comment about Oracle copyright. Fixed ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403054811 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403055037 From sjohanss at openjdk.org Thu Nov 23 08:51:20 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 23 Nov 2023 08:51:20 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 23:08:36 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Cleanup and address comments I think this looks good now. I've started some additional testing to make sure we don't run into anything unexpected with the newly added test on any platforms. Will approve once the testing is green. src/hotspot/share/runtime/cpuTimeCounters.hpp line 2: > 1: /* > 2: * Copyright (c) 2002, 2023, Oracle and/or its affiliates. All rights reserved. This is probably on me, but this 2002 is wrong: Suggestion: * Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. ------------- PR Review: https://git.openjdk.org/jdk/pull/15082#pullrequestreview-1745951725 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403070166 From amitkumar at openjdk.org Thu Nov 23 08:53:02 2023 From: amitkumar at openjdk.org (Amit Kumar) Date: Thu, 23 Nov 2023 08:53:02 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> <2X-eefx1BKO-hqncC3LF0It1wUxHRqvrX0qm2TsCbd0=.2626d8a4-d549-4a82-a2de-40003af36539@github.com> Message-ID: On Thu, 23 Nov 2023 08:08:51 GMT, suchismith1993 wrote: > The JBS issue with respect to that has been closed. Need to check if that PR is required. Currently putting it on hold. This response on the issue suggest otherwise: : The JDK does not support dynamically loaded archive files (.a files) and there are no plans to add this support. Closing as will not fix ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1824009424 From stefank at openjdk.org Thu Nov 23 08:55:11 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 08:55:11 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 01:29:24 GMT, David Holmes wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 29: > >> 27: * @summary This locks a monitor, GCs the object, and iterate and perform >> 28: * various iteration and operations over this monitor. >> 29: * @requires os.family == "linux" > > I know the test this was copied from had this but I'm not sure it is actually a necessary restriction - any Posix platform should work. Though maybe perror is linux only ... I couldn't find any test filtering for posix, but some tests have: @requires os.family != "windows" That seems to work on my Mac. > test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 31: > >> 29: * @requires os.family == "linux" >> 30: * @library /testlibrary /test/lib >> 31: * @build IterateMonitorWithDeadObjectTest > > You don't need an explicit `@build` step Sure. This was just copied brought over from your original reproducer. > test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 66: > >> 64: // dead object. The thread dumping code didn't tolerate such a monitor, >> 65: // so run a thread dump and make sure that it doesn't crash/assert. >> 66: dumpThreadsWithLockedMonitors(); > > But you've already detached the thread so there is no locked monitor any longer. And `runTestAndDetachThread()` also did a thread dump. Right. This is left-overs from my earlier attempt. I'll remove this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403077397 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403077988 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403078851 From stefank at openjdk.org Thu Nov 23 08:59:07 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 08:59:07 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 01:38:57 GMT, David Holmes wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 40: > >> 38: public class IterateMonitorWithDeadObjectTest { >> 39: public static native void runTestAndDetachThread(); >> 40: public static native void joinTestThread(); > > I don't think this form of the test needs to separate out the `pthread_join()`, it can just be done in `runTestAndDetachThread` AFAICS. I originally split it out to allow the Java code to do the GC while the native thread was sleeping prior to detaching. All this is left-overs that I thought I had removed. I'm removing this. > test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java line 57: > >> 55: // - Drop the last reference to the object >> 56: // - GC to clear the weak reference to the object in the monitor >> 57: // - Detach the thread - provoke previous bug > > It also does a thread dump while the lock is held Updated ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403082283 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403082528 From kevinw at openjdk.org Thu Nov 23 09:38:14 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Thu, 23 Nov 2023 09:38:14 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) Message-ID: RMI Connections (in general) should use a timeout on the Socket connect call by default. JMX connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java has other timeouts but nothing to control the initial Socket connect. Defaulting to a 1 minute timeout on connect has no effect on existing tests for RMI and JMX, and should go unnoticed in applications unless there really is a significant connection delay. Specifying zero for the new System Property sun.rmi.transport.tcp.initialConnectTimeout uses the old code path of a connect with no timeout. I have a test, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will often pass as the connection happens immediately. ------------- Commit messages: - Ignore negative timeout values. - Make connect timeout static - 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) Changes: https://git.openjdk.org/jdk/pull/16771/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16771&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8316649 Stats: 21 lines in 1 file changed: 19 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16771.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16771/head:pull/16771 PR: https://git.openjdk.org/jdk/pull/16771 From duke at openjdk.org Thu Nov 23 09:51:06 2023 From: duke at openjdk.org (suchismith1993) Date: Thu, 23 Nov 2023 09:51:06 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Thu, 23 Nov 2023 06:16:54 GMT, Thomas Stuefe wrote: > If .a is a valid shared object format on AIX, this should be handled in `os::dll_load()`, and be done for all shared objects. If not, why do we try to load a static archive via dlload in this case but not in other cases? In AIX, we have shared objects with .a extension and also static archives with .a extension.I think in other platforms the format for shared objects is fixed? . In that case does this become specific to AIX? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1824086655 From stefank at openjdk.org Thu Nov 23 10:14:09 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 10:14:09 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 00:48:19 GMT, David Holmes wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > >> Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code > > I had not realized that. It explains some confusion in a separate issue I had been looking into! It is important that these monitors are exposed and unlocked at detach time, otherwise it also messes up the `held_monitor_count`. > >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure > > I think we may need to make that code tolerate the absence of an object. > >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. > > I think we probably should expose this to be accurate, but I think this needs investigation on the JVMTI side to ensure that the null entry is tolerated okay. So a separate RFE to handle this would be fine. > > Thanks While addressing @dholmes-ora's comments about the tests I found that the JNI implementation was incorrect and caused a failure, that seems to prevent the thread detach, and then the Java layer thread dumping caught the thread dumping bug. I'm going to see if I can split the various test cases so that they can be tested individually. Don't look too closely at the current test until it has been updated. > test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 94: > >> 92: >> 93: // Let the GC clear the weak reference to the object. >> 94: system_gc(env); > > AFAIK there is no guarantee that one call to `System.gc()` will suffice to clear the weakRef. We tend use a loop with a few iterations in other tests, or use a WhiteBox method to achieve it. In my testing I used the finalizer to observe that the objects had been finalized but even then, and with a loop, I did not always see them collected with G1. ZGC will clear the weak reference. G1 clears the weak reference, except for in a few specific situations. I've verified that G1 clears this weak reference, so I don't think it would be worth making this test longer or more complicated. > test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 104: > >> 102: // source of at least two bugs: >> 103: // - When the object reference in the monitor was made weak, the code >> 104: // didn't unlock the monitor, leaving it lingering in the system. > > Suggestion: > > // - When the object reference in the monitor was cleared, the monitor > // iterator code would skip it, preventing it from being unlocked when > // the owner thread detached, leaving it lingering in the system. > > the original made it sound to me like the code that cleared the reference (i.e. the GC) was expected to do the unlocking. Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16783#issuecomment-1824116115 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403168662 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403169323 From azafari at openjdk.org Thu Nov 23 10:52:35 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Thu, 23 Nov 2023 10:52:35 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v10] In-Reply-To: References: Message-ID: > The `find` method now is > ```C++ > template > int find(T* token, bool f(T*, E)) const { > ... > > Any other functions which use this are also changed. > Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: Some cleanups, documentaions and tests. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15418/files - new: https://git.openjdk.org/jdk/pull/15418/files/9973f5d7..74c4076b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=09 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=08-09 Stats: 165 lines in 13 files changed: 99 ins; 33 del; 33 mod Patch: https://git.openjdk.org/jdk/pull/15418.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15418/head:pull/15418 PR: https://git.openjdk.org/jdk/pull/15418 From jbachorik at openjdk.org Thu Nov 23 10:58:46 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 10:58:46 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v4] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: Move assert ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/967813b6..9f61d8e0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=02-03 Stats: 4 lines in 1 file changed: 2 ins; 2 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From azafari at openjdk.org Thu Nov 23 11:03:13 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Thu, 23 Nov 2023 11:03:13 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v10] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 10:52:35 GMT, Afshin Zafari wrote: >> The `find` method now is >> ```C++ >> template >> int find(T* token, bool f(T*, E)) const { >> ... >> >> Any other functions which use this are also changed. >> Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. > > Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: > > Some cleanups, documentaions and tests. > Thanks for making this change. > > I'd like to suggest the following cleanups, some documentation, and a few tests: [20d4502](https://github.com/openjdk/jdk/commit/20d4502471ba396ae395512cfa3dab3f87555421) > > I think it might be easier to review by looking at the final diff: [master...stefank:jdk:pr_15418](https://github.com/openjdk/jdk/compare/master...stefank:jdk:pr_15418) One question: the `private static bool by_name(const char* name, PerfData* pd);` is added to `PerfDataList` class but is never used. Is something missing? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1824226437 From stefank at openjdk.org Thu Nov 23 11:25:07 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 11:25:07 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 02:07:59 GMT, David Holmes wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > test/hotspot/jtreg/serviceability/jvmti/GetOwnedMonitorInfo/GetOwnedMonitorInfoTest.java line 59: > >> 57: // GetOwnedMonitorInfo testing. >> 58: Object obj = new Object() { public String toString() {return "";} }; >> 59: jniMonitorEnter(obj); > > I would add a check for `Thread.holdsLock(obj);` after this just to be sure it worked. Done > test/hotspot/jtreg/serviceability/jvmti/GetOwnedMonitorInfo/GetOwnedMonitorInfoTest.java line 61: > >> 59: jniMonitorEnter(obj); >> 60: obj = null; >> 61: System.gc(); > > Again one gc() is generally not sufficient. > > How can this test tell that the object in the monitor was actually cleared? I think `monitorinflation` logging may be the only way to tell. Yes, probably. I've been looking at the `monitorinflation` logging to very that it gets cleared. I think it would be messy to try to get this test to also start to parse logs. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403245976 PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403244666 From msheppar at openjdk.org Thu Nov 23 11:40:05 2023 From: msheppar at openjdk.org (Mark Sheppard) Date: Thu, 23 Nov 2023 11:40:05 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: <19duWGlPAF6oED9N344lKIW9sssuEWQWCr9c-vI4MxE=.882aeec7-6ec0-47c0-b737-5f4e6300bf31@github.com> On Tue, 21 Nov 2023 17:57:32 GMT, Kevin Walls wrote: > RMI Connections (in general) should use a timeout on the Socket connect call by default. > > JMX connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. > > src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java > has other timeouts but nothing to control the initial Socket connect. > > Defaulting to a 1 minute timeout on connect has no effect on existing tests for RMI and JMX, and should go unnoticed in applications unless there really is a significant connection delay. Specifying zero for the new System Property sun.rmi.transport.tcp.initialConnectTimeout uses the old code path of a connect with no timeout. > > I have a test, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will often pass as the connection happens immediately. The introduction of the system property is reasonable. I have questions over the veracity of the context described in the bug. I think it's a firewall configuration issue. If there is not listening end point for a TCP connect request, this will, result in a ConnectException being thrown in the RMI client. As such, for no response to happen, It means that a TCP connection has been established with the firewall (at the TCP level), from the RMI (JMX) client and that connection has not been accepted or passed through by the firewall application. Thus, it would appear that the shutdown of the target server has not been registered in the firewall. In any case the change looks fine. Do you have a test c.f. bug comment ------------- PR Comment: https://git.openjdk.org/jdk/pull/16771#issuecomment-1824272681 From dfuchs at openjdk.org Thu Nov 23 11:40:07 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 23 Nov 2023 11:40:07 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: On Tue, 21 Nov 2023 17:57:32 GMT, Kevin Walls wrote: > RMI Connections (in general) should use a timeout on the Socket connect call by default. > > JMX connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. > > src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java > has other timeouts but nothing to control the initial Socket connect. > > Defaulting to a 1 minute timeout on connect has no effect on existing tests for RMI and JMX, and should go unnoticed in applications unless there really is a significant connection delay. Specifying zero for the new System Property sun.rmi.transport.tcp.initialConnectTimeout uses the old code path of a connect with no timeout. > > I have a test, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will often pass as the connection happens immediately. src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPDirectSocketFactory.java line 46: > 44: private static final int connectTimeout = // default 1 minute > 45: AccessController.doPrivileged((PrivilegedAction) () -> > 46: Integer.getInteger("sun.rmi.transport.tcp.initialConnectTimeout", 60 * 1000).intValue()); An exception here will prevent the class from being initialized, any subsequent attempts to use the class will then produce hard to diagnose error. If the class is not loaded by the main thread, such exception/error could be swallowed unless an uncaught exception handler was registered. I would suggest implementing the logic in a static block, catch any exception and revert to default behaviour (i.e. 0) if the value supplied for the property is not an integer, or not a valid integer, etc... ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1403258620 From dfuchs at openjdk.org Thu Nov 23 11:40:07 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 23 Nov 2023 11:40:07 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 11:35:11 GMT, Daniel Fuchs wrote: >> RMI Connections (in general) should use a timeout on the Socket connect call by default. >> >> JMX connections use RMI and some connection failures never terminate. The hang described in 8316649 is hard to reproduce manually: the description says it can be caused by a firewall that never returns a response. >> >> src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java >> has other timeouts but nothing to control the initial Socket connect. >> >> Defaulting to a 1 minute timeout on connect has no effect on existing tests for RMI and JMX, and should go unnoticed in applications unless there really is a significant connection delay. Specifying zero for the new System Property sun.rmi.transport.tcp.initialConnectTimeout uses the old code path of a connect with no timeout. >> >> I have a test, but it is not realistically usable: although specifying a 1 millisecond timeout will often fail (as expected/desired for the test), it will often pass as the connection happens immediately. > > src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPDirectSocketFactory.java line 46: > >> 44: private static final int connectTimeout = // default 1 minute >> 45: AccessController.doPrivileged((PrivilegedAction) () -> >> 46: Integer.getInteger("sun.rmi.transport.tcp.initialConnectTimeout", 60 * 1000).intValue()); > > An exception here will prevent the class from being initialized, any subsequent attempts to use the class will then produce hard to diagnose error. If the class is not loaded by the main thread, such exception/error could be swallowed unless an uncaught exception handler was registered. I would suggest implementing the logic in a static block, catch any exception and revert to default behaviour (i.e. 0) if the value supplied for the property is not an integer, or not a valid integer, etc... IIRC, the default agent uses / may use its own socket factories - are we still coming here even with those factories? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1403260460 From stefank at openjdk.org Thu Nov 23 11:44:57 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 11:44:57 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v2] In-Reply-To: References: Message-ID: > In the rewrites made for: > [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` > > I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. > > The provided tests provoke this assert form: > * the JNI thread detach code > * thread dumping with locked monitors, and > * the JVMTI GetOwnedMonitorInfo API. > > While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. > > The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. > > For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. > > Test: the written tests with and without the fix. Tier1-Tier3, so far. Stefan Karlsson has updated the pull request incrementally with two additional commits since the last revision: - Tweaked comment in test - Rewrite tests ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16783/files - new: https://git.openjdk.org/jdk/pull/16783/files/b1dd4cf8..4b0976a8 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=00-01 Stats: 605 lines in 9 files changed: 388 ins; 214 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16783/head:pull/16783 PR: https://git.openjdk.org/jdk/pull/16783 From stefank at openjdk.org Thu Nov 23 11:45:00 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 11:45:00 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v2] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 02:10:41 GMT, David Holmes wrote: >> Stefan Karlsson has updated the pull request incrementally with two additional commits since the last revision: >> >> - Tweaked comment in test >> - Rewrite tests > > test/hotspot/jtreg/runtime/Monitor/libIterateMonitorWithDeadObjectTest.c line 43: > >> 41: static jobject create_object(JNIEnv* env) { >> 42: jclass clazz = (*env)->FindClass(env, "java/lang/Object"); >> 43: if (clazz == 0) die("No class"); > > The `die` method is for errors with system calls. It won't show useful information for JNI calls that leave exceptions pending. I've added better checks. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403263549 From stefank at openjdk.org Thu Nov 23 11:52:38 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 11:52:38 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object In-Reply-To: References: Message-ID: <56AIMDRiXJZpaiHjEYPlnT1t7jlf4gKJk7UxmtWmpo8=.a9a08009-2048-4354-80c2-9cb0c8746369@github.com> On Thu, 23 Nov 2023 00:48:19 GMT, David Holmes wrote: > > Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code > > I had not realized that. It explains some confusion in a separate issue I had been looking into! It is important that these monitors are exposed and unlocked at detach time, otherwise it also messes up the `held_monitor_count`. > I think I managed to reproduce that while reworking the tests and temporarily reintroducing the bug. > > The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure > > I think we may need to make that code tolerate the absence of an object. IDK. That's another thing that would be good to discuss in a separate RFE. > > > For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. > > I think we probably should expose this to be accurate, but I think this needs investigation on the JVMTI side to ensure that the null entry is tolerated okay. So a separate RFE to handle this would be fine. This will lead to NPE in the management code, but it even if we fixed that, it could potentially causing NPEs in user code. So, yes, I wouldn't mind if someone wanted to investigate this as a separate RFE. > > Thanks ------------- PR Comment: https://git.openjdk.org/jdk/pull/16783#issuecomment-1824286615 From stefank at openjdk.org Thu Nov 23 11:52:38 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 11:52:38 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v3] In-Reply-To: References: Message-ID: > In the rewrites made for: > [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` > > I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. > > The provided tests provoke this assert form: > * the JNI thread detach code > * thread dumping with locked monitors, and > * the JVMTI GetOwnedMonitorInfo API. > > While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. > > The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. > > For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. > > Test: the written tests with and without the fix. Tier1-Tier3, so far. Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: Tweak test comments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16783/files - new: https://git.openjdk.org/jdk/pull/16783/files/4b0976a8..3239b822 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=01-02 Stats: 6 lines in 1 file changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/16783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16783/head:pull/16783 PR: https://git.openjdk.org/jdk/pull/16783 From stefank at openjdk.org Thu Nov 23 11:56:19 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 11:56:19 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v10] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 11:00:32 GMT, Afshin Zafari wrote: > > Thanks for making this change. > > I'd like to suggest the following cleanups, some documentation, and a few tests: [20d4502](https://github.com/openjdk/jdk/commit/20d4502471ba396ae395512cfa3dab3f87555421) > > I think it might be easier to review by looking at the final diff: [master...stefank:jdk:pr_15418](https://github.com/openjdk/jdk/compare/master...stefank:jdk:pr_15418) > > One question: the `private static bool by_name(const char* name, PerfData* pd);` is added to `PerfDataList` class but is never used. Is something missing? That should have been removed when I added name_equals. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1824302304 From kevinw at openjdk.org Thu Nov 23 12:43:06 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Thu, 23 Nov 2023 12:43:06 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 11:37:12 GMT, Daniel Fuchs wrote: >> src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPDirectSocketFactory.java line 46: >> >>> 44: private static final int connectTimeout = // default 1 minute >>> 45: AccessController.doPrivileged((PrivilegedAction) () -> >>> 46: Integer.getInteger("sun.rmi.transport.tcp.initialConnectTimeout", 60 * 1000).intValue()); >> >> An exception here will prevent the class from being initialized, any subsequent attempts to use the class will then produce hard to diagnose error. If the class is not loaded by the main thread, such exception/error could be swallowed unless an uncaught exception handler was registered. I would suggest implementing the logic in a static block, catch any exception and revert to default behaviour (i.e. 0) if the value supplied for the property is not an integer, or not a valid integer, etc... > > IIRC, the default agent uses / may use its own socket factories - are we still coming here even with those factories? > An exception here will prevent the class from being initialized... Maybe we can break it, but this new property is following the same pattern I think as the neighbouring file src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java when it reads the system props. I see Integer.getInteger handles the obvious Exceptions, so specifying a strange value does not immediately break us. If there's more protection needed then we have various other places to apply it that might need separate follow-up if you think there's a case? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1403331157 From kevinw at openjdk.org Thu Nov 23 12:46:08 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Thu, 23 Nov 2023 12:46:08 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 12:40:18 GMT, Kevin Walls wrote: >> IIRC, the default agent uses / may use its own socket factories - are we still coming here even with those factories? > >> An exception here will prevent the class from being initialized... > > Maybe we can break it, but this new property is following the same pattern I think as the neighbouring file src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java when it reads the system props. I see Integer.getInteger handles the obvious Exceptions, so specifying a strange value does not immediately break us. > > If there's more protection needed then we have various other places to apply it that might need separate follow-up if you think there's a case? > IIRC, the default agent uses / may use its own socket factories - are we still coming here even with those factories? We do get here, yes (not saying you can't customise your way out of getting here). This is a stack from a test I was experimenting with, when it did see the timeout: ---------System.out:(4/132)---------- JMX URL = service:jmx:rmi://x expectTimeout = true sun.rmi.transport.tcp.initialConnectTimeout = 1 Test passed ----------System.err:(24/1791)---------- java.rmi.ConnectIOException: Exception creating connection to: x.x.x.x; nested exception is: java.net.SocketTimeoutException at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:637) at java.rmi/sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:217) at java.rmi/sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:204) at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:134) at java.management.rmi/javax.management.remote.rmi.RMIServerImpl_Stub.newClient(RMIServerImpl_Stub.java:85) at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2106) at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:321) at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) at RMIConnectionTimeoutTest.main(RMIConnectionTimeoutTest.java:65) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) at java.base/java.lang.Thread.run(Thread.java:1570) Caused by: java.net.SocketTimeoutException at java.base/java.net.SocksSocketImpl.remainingMillis(SocksSocketImpl.java:112) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:752) at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createSocket(TCPDirectSocketFactory.java:57) at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) ... 13 more STATUS:Passed. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1403334446 From kevinw at openjdk.org Thu Nov 23 12:49:06 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Thu, 23 Nov 2023 12:49:06 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: <19duWGlPAF6oED9N344lKIW9sssuEWQWCr9c-vI4MxE=.882aeec7-6ec0-47c0-b737-5f4e6300bf31@github.com> References: <19duWGlPAF6oED9N344lKIW9sssuEWQWCr9c-vI4MxE=.882aeec7-6ec0-47c0-b737-5f4e6300bf31@github.com> Message-ID: On Thu, 23 Nov 2023 11:35:47 GMT, Mark Sheppard wrote: > In any case the change looks fine. Do you have a test c.f. bug comment Thanks @msheppar yes I agree with the comments. For testing I got as far as a test that often cannot see the timeout as we are too quick for a 1ms timeout. I was thinking that as the change is basic enough, getting a property and passing on a value, the usable test seems rather elaborate... We do have test/jdk/java/rmi/transport/handshakeTimeout/HandshakeTimeout.java which tests -Dsun.rmi.transport.tcp.handshakeTimeout=1 but that must reliably fail to handshake in 1ms. The basic connect is much faster. 8-) ------------- PR Comment: https://git.openjdk.org/jdk/pull/16771#issuecomment-1824372808 From jbachorik at openjdk.org Thu Nov 23 13:02:39 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 13:02:39 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v5] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: Add exhaustive check for method holder availability ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/9f61d8e0..00f644a0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=03-04 Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From azafari at openjdk.org Thu Nov 23 13:35:28 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Thu, 23 Nov 2023 13:35:28 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v11] In-Reply-To: References: Message-ID: > The `find` method now is > ```C++ > template > int find(T* token, bool f(T*, E)) const { > ... > > Any other functions which use this are also changed. > Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. Afshin Zafari has updated the pull request incrementally with one additional commit since the last revision: by_name method is removed from PerfDataList. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15418/files - new: https://git.openjdk.org/jdk/pull/15418/files/74c4076b..8d5f54ab Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=09-10 Stats: 4 lines in 1 file changed: 0 ins; 4 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/15418.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15418/head:pull/15418 PR: https://git.openjdk.org/jdk/pull/15418 From azafari at openjdk.org Thu Nov 23 13:35:29 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Thu, 23 Nov 2023 13:35:29 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v10] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 11:53:28 GMT, Stefan Karlsson wrote: >>> Thanks for making this change. >>> >>> I'd like to suggest the following cleanups, some documentation, and a few tests: [20d4502](https://github.com/openjdk/jdk/commit/20d4502471ba396ae395512cfa3dab3f87555421) >>> >>> I think it might be easier to review by looking at the final diff: [master...stefank:jdk:pr_15418](https://github.com/openjdk/jdk/compare/master...stefank:jdk:pr_15418) >> >> One question: the `private static bool by_name(const char* name, PerfData* pd);` is added to `PerfDataList` class but is never used. Is something missing? > >> > Thanks for making this change. >> > I'd like to suggest the following cleanups, some documentation, and a few tests: [20d4502](https://github.com/openjdk/jdk/commit/20d4502471ba396ae395512cfa3dab3f87555421) >> > I think it might be easier to review by looking at the final diff: [master...stefank:jdk:pr_15418](https://github.com/openjdk/jdk/compare/master...stefank:jdk:pr_15418) >> >> One question: the `private static bool by_name(const char* name, PerfData* pd);` is added to `PerfDataList` class but is never used. Is something missing? > > That should have been removed when I added name_equals. @stefank, the changes are applied as suggested and ready for review. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1824446141 From jbachorik at openjdk.org Thu Nov 23 13:37:41 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 13:37:41 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v6] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: Add exhaustive check for method holder availability (1) ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/00f644a0..eaf2720e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=04-05 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From ayang at openjdk.org Thu Nov 23 13:52:28 2023 From: ayang at openjdk.org (Albert Mingkun Yang) Date: Thu, 23 Nov 2023 13:52:28 GMT Subject: RFR: 8315149: Add hsperf counters for CPU time of internal GC threads [v47] In-Reply-To: References: Message-ID: On Wed, 22 Nov 2023 23:08:36 GMT, Jonathan Joo wrote: >> 8315149: Add hsperf counters for CPU time of internal GC threads > > Jonathan Joo has updated the pull request incrementally with one additional commit since the last revision: > > Cleanup and address comments src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 2433: > 2431: } > 2432: WorkerThreads* worker_threads = workers(); > 2433: if (worker_threads != nullptr) { When will this be null? src/hotspot/share/runtime/cpuTimeCounters.cpp line 61: > 59: return true; > 60: case CPUTimeType::gc_service: > 61: return true; I think it would look cleaner if these are grouped into a single return. src/hotspot/share/runtime/cpuTimeCounters.cpp line 96: > 94: if (UsePerfData) { > 95: EXCEPTION_MARK; > 96: if (os::is_thread_cpu_time_supported()) { Why is this check inside the scope of `EXCEPTION_MARK`? (I'd expect sth like ` if (use-perf-data && is-cpu-time-supported)` at the top.) src/hotspot/share/runtime/cpuTimeCounters.cpp line 119: > 117: if (CPUTimeGroups::is_gc_counter(_name)) { > 118: instance->inc_gc_total_cpu_time(net_cpu_time); > 119: } I feel much of this is on the wrong abstraction level; `CPUTimeCounters::update_counter(_name, _total);` should be sufficient here. (The caller handles diff calculation and inc gc-counter if needed.) src/hotspot/share/runtime/cpuTimeCounters.cpp line 126: > 124: // pthread_getcpuclockid() and clock_gettime() must return 0. Thus caller > 125: // must ensure the thread exists and has not terminated. > 126: assert(os::is_thread_cpu_time_supported(), "os must support cpu time"); Could this assert be moved to the constructor? src/hotspot/share/runtime/cpuTimeCounters.hpp line 74: > 72: assert(_instance != nullptr, "no instance found"); > 73: return _instance; > 74: } Seems that this is needed solely for accessing the following instance methods. I wonder if it's possible to expose only static methods in the public API. src/hotspot/share/runtime/cpuTimeCounters.hpp line 83: > 81: // Prevent copy of singleton object. > 82: CPUTimeCounters(const CPUTimeCounters& copy) = delete; > 83: void operator=(const CPUTimeCounters& copy) = delete; I think `NONCOPYABLE` can be used here. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403333939 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403220339 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403219626 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403411402 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403333335 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403413301 PR Review Comment: https://git.openjdk.org/jdk/pull/15082#discussion_r1403225116 From stefank at openjdk.org Thu Nov 23 13:55:13 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 13:55:13 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v10] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 13:32:23 GMT, Afshin Zafari wrote: >>> > Thanks for making this change. >>> > I'd like to suggest the following cleanups, some documentation, and a few tests: [20d4502](https://github.com/openjdk/jdk/commit/20d4502471ba396ae395512cfa3dab3f87555421) >>> > I think it might be easier to review by looking at the final diff: [master...stefank:jdk:pr_15418](https://github.com/openjdk/jdk/compare/master...stefank:jdk:pr_15418) >>> >>> One question: the `private static bool by_name(const char* name, PerfData* pd);` is added to `PerfDataList` class but is never used. Is something missing? >> >> That should have been removed when I added name_equals. > > @stefank, the changes are applied as suggested and ready for review. @afshin-zafari you toke my changes and then you made a few small whitespace changes that messed up the patch. I merged our two branches to correct those issues. Could you make sure to pull the changes from my branch (don't copy the changes): https://github.com/stefank/jdk/tree/pr_15418 ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1824474455 From stefank at openjdk.org Thu Nov 23 14:09:07 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 14:09:07 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v3] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 11:52:38 GMT, Stefan Karlsson wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: > > Tweak test comments GHA complains that the lib gets loaded into multiple class loaders. I need to figure out how to share code between the tests without so that I don't have to duplicate the code. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16783#issuecomment-1824494844 From stuefe at openjdk.org Thu Nov 23 14:19:08 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 23 Nov 2023 14:19:08 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v2] In-Reply-To: <4m-e190SIpRd0-fkjZ4ja7mhVaQQsTwfqmWd8_82Els=.7b198822-3637-4109-b1e9-2451f882ce4b@github.com> References: <4m-e190SIpRd0-fkjZ4ja7mhVaQQsTwfqmWd8_82Els=.7b198822-3637-4109-b1e9-2451f882ce4b@github.com> Message-ID: <8-swJyF28KZeFLyqa2awML3umu2vog2Du9zVlJZfRDI=.5c435f8e-78af-47fa-a020-6bfce16b6c65@github.com> On Thu, 23 Nov 2023 08:44:08 GMT, Jaroslav Bachorik wrote: >> Jaroslav Bachorik has updated the pull request incrementally with three additional commits since the last revision: >> >> - Clean up imports >> - Simplify Method::clear_jmethod_id() >> - Use correct copyrights > > @dholmes-ora >> Can't we just check method->method_holder() for null rather than passing in a parameter like this? > > I have removed the argument and I am now performing the check for `method_holder() != nullptr` as recommended. The code is a bit simpler and the cost of resolving the method holder for each method is probably quite low so we should be ok here. @jbachorik You are aware that this fix only works for some uncommon corner cases, right? It only works if the Method is explicitly deallocated. The vast bulk of Method aren't. Method, as a Metaspace object, is released in bulk when the class is unloaded. The `::deallocate` path you fixed up - that eventually ends up in `MetaspaceArena::deallocate()` - is a rare case that only gets invoked if - a class cannot be loaded but parts of it have already been loaded into Metaspace. - a class gets transformed In case the class gets unloaded via conventional means, your fix won't get invoked (nor should it; releasing in bulk without having to care for individual allocations is the point of Metaspace). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1824509686 From ihse at openjdk.org Thu Nov 23 14:20:11 2023 From: ihse at openjdk.org (Magnus Ihse Bursie) Date: Thu, 23 Nov 2023 14:20:11 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v3] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 11:52:38 GMT, Stefan Karlsson wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: > > Tweak test comments Build changes are trivially fine. ------------- Marked as reviewed by ihse (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/16783#pullrequestreview-1746557040 From stuefe at openjdk.org Thu Nov 23 14:25:13 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 23 Nov 2023 14:25:13 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v6] In-Reply-To: References: Message-ID: <4Sx6iriS9oGkl2xQRhOOzWMraDoP6DyvHtaCqkDn3IQ=.2a7e0fa3-81ad-4859-a57e-531179dddf3d@github.com> On Thu, 23 Nov 2023 13:37:41 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: > > Add exhaustive check for method holder availability (1) src/hotspot/share/oops/instanceKlass.cpp line 541: > 539: assert (!method->on_stack(), "shouldn't be called with methods on stack"); > 540: // Do the pointer maintenance before releasing the metadata > 541: method->clear_jmethod_id(); IIUC this is O(n^2) over number of methods. Seeing that this is a workaround for a special case (an app that does a lot of retransforms *and* uses async profiler), I'd opt for making it conditional. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403449172 From jbachorik at openjdk.org Thu Nov 23 15:22:09 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 15:22:09 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v6] In-Reply-To: References: Message-ID: <0ul1T9qxWhSKv9NTx2Ejvm0nUoAHTYTHZzkEQnkR5NI=.e08e133f-7ea6-4817-b0ea-8319658b0668@github.com> On Mon, 20 Nov 2023 22:08:49 GMT, Coleen Phillimore wrote: >> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: >> >> Add exhaustive check for method holder availability (1) > > src/hotspot/share/classfile/classFileParser.cpp line 5579: > >> 5577: >> 5578: if (_methods != nullptr) { >> 5579: // Free methods - those methods are not fully wired and miss the method holder > > How about saying: for methods whose InstanceKlass as method holder is not yet created? And is now back, but I applied your suggestion, @coleenp ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403513054 From jbachorik at openjdk.org Thu Nov 23 15:22:12 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 15:22:12 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v6] In-Reply-To: <4Sx6iriS9oGkl2xQRhOOzWMraDoP6DyvHtaCqkDn3IQ=.2a7e0fa3-81ad-4859-a57e-531179dddf3d@github.com> References: <4Sx6iriS9oGkl2xQRhOOzWMraDoP6DyvHtaCqkDn3IQ=.2a7e0fa3-81ad-4859-a57e-531179dddf3d@github.com> Message-ID: On Thu, 23 Nov 2023 14:20:48 GMT, Thomas Stuefe wrote: >> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: >> >> Add exhaustive check for method holder availability (1) > > src/hotspot/share/oops/instanceKlass.cpp line 541: > >> 539: assert (!method->on_stack(), "shouldn't be called with methods on stack"); >> 540: // Do the pointer maintenance before releasing the metadata >> 541: method->clear_jmethod_id(); > > IIUC this is O(n^2) over number of methods. Seeing that this is a workaround for a special case (an app that does a lot of retransforms *and* uses async profiler), I'd opt for making it conditional. Sadly, this is not async-profiler specific. The same issue can be observed by JVMTI only code grabbing a stacktrace. What do you mean exactly by 'conditional'? Introducing a new JVM flag or something else? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403515842 From jbachorik at openjdk.org Thu Nov 23 15:49:09 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 15:49:09 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v2] In-Reply-To: <8-swJyF28KZeFLyqa2awML3umu2vog2Du9zVlJZfRDI=.5c435f8e-78af-47fa-a020-6bfce16b6c65@github.com> References: <4m-e190SIpRd0-fkjZ4ja7mhVaQQsTwfqmWd8_82Els=.7b198822-3637-4109-b1e9-2451f882ce4b@github.com> <8-swJyF28KZeFLyqa2awML3umu2vog2Du9zVlJZfRDI=.5c435f8e-78af-47fa-a020-6bfce16b6c65@github.com> Message-ID: On Thu, 23 Nov 2023 14:15:54 GMT, Thomas Stuefe wrote: >> @dholmes-ora >>> Can't we just check method->method_holder() for null rather than passing in a parameter like this? >> >> I have removed the argument and I am now performing the check for `method_holder() != nullptr` as recommended. The code is a bit simpler and the cost of resolving the method holder for each method is probably quite low so we should be ok here. > > @jbachorik You are aware that this fix only works for some uncommon corner cases, right? > > It only works if the Method is explicitly deallocated. The vast bulk of Method aren't. Method, as a Metaspace object, is released in bulk when the class is unloaded. The `::deallocate` path you fixed up - that eventually ends up in `MetaspaceArena::deallocate()` - is a rare case that only gets invoked if > - a class cannot be loaded but parts of it have already been loaded into Metaspace. > - a class gets transformed > > In case the class gets unloaded via conventional means, your fix won't get invoked (nor should it; releasing in bulk without having to care for individual allocations is the point of Metaspace). @tstuefe > In case the class gets unloaded via conventional means, your fix won't get invoked (nor should it; releasing in bulk without having to care for individual allocations is the point of Metaspace). Yes. This is intended. The deallocation path via Metaspace is fine. It is just the code that is purging previous versions of a redefined class that has this bug. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1824641628 From jbachorik at openjdk.org Thu Nov 23 15:49:12 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 15:49:12 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v6] In-Reply-To: References: <4Sx6iriS9oGkl2xQRhOOzWMraDoP6DyvHtaCqkDn3IQ=.2a7e0fa3-81ad-4859-a57e-531179dddf3d@github.com> Message-ID: On Thu, 23 Nov 2023 15:19:43 GMT, Jaroslav Bachorik wrote: >> src/hotspot/share/oops/instanceKlass.cpp line 541: >> >>> 539: assert (!method->on_stack(), "shouldn't be called with methods on stack"); >>> 540: // Do the pointer maintenance before releasing the metadata >>> 541: method->clear_jmethod_id(); >> >> IIUC this is O(n^2) over number of methods. Seeing that this is a workaround for a special case (an app that does a lot of retransforms *and* uses async profiler), I'd opt for making it conditional. > > Sadly, this is not async-profiler specific. The same issue can be observed by JVMTI only code grabbing a stacktrace. > What do you mean exactly by 'conditional'? Introducing a new JVM flag or something else? Ok, I see now - I could do the jmethodID maintenance only from `purge_previous_version_list()` call, leaving the proper metaspace deallocation untouched, therefore not adding unnecessary overhead there. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403544462 From stuefe at openjdk.org Thu Nov 23 16:05:10 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 23 Nov 2023 16:05:10 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v2] In-Reply-To: <8-swJyF28KZeFLyqa2awML3umu2vog2Du9zVlJZfRDI=.5c435f8e-78af-47fa-a020-6bfce16b6c65@github.com> References: <4m-e190SIpRd0-fkjZ4ja7mhVaQQsTwfqmWd8_82Els=.7b198822-3637-4109-b1e9-2451f882ce4b@github.com> <8-swJyF28KZeFLyqa2awML3umu2vog2Du9zVlJZfRDI=.5c435f8e-78af-47fa-a020-6bfce16b6c65@github.com> Message-ID: On Thu, 23 Nov 2023 14:15:54 GMT, Thomas Stuefe wrote: >> @dholmes-ora >>> Can't we just check method->method_holder() for null rather than passing in a parameter like this? >> >> I have removed the argument and I am now performing the check for `method_holder() != nullptr` as recommended. The code is a bit simpler and the cost of resolving the method holder for each method is probably quite low so we should be ok here. > > @jbachorik You are aware that this fix only works for some uncommon corner cases, right? > > It only works if the Method is explicitly deallocated. The vast bulk of Method aren't. Method, as a Metaspace object, is released in bulk when the class is unloaded. The `::deallocate` path you fixed up - that eventually ends up in `MetaspaceArena::deallocate()` - is a rare case that only gets invoked if > - a class cannot be loaded but parts of it have already been loaded into Metaspace. > - a class gets transformed > > In case the class gets unloaded via conventional means, your fix won't get invoked (nor should it; releasing in bulk without having to care for individual allocations is the point of Metaspace). > @tstuefe > > > In case the class gets unloaded via conventional means, your fix won't get invoked (nor should it; releasing in bulk without having to care for individual allocations is the point of Metaspace). > > Yes. This is intended. The deallocation path via Metaspace is fine. It is just the code that is purging previous versions of a redefined class that has this bug. I see it now. On class unloading, we reset the jmethodID "master table" linked list nodes, set all slots to null, but don't delete them to keep outside users from crashing. And we release the IK itself and before we do that free the methodID cache in IK. So we are covered. I thought this was about class unloading, sorry for not reading the description carefully. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1824662450 From azafari at openjdk.org Thu Nov 23 16:11:32 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Thu, 23 Nov 2023 16:11:32 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v12] In-Reply-To: References: Message-ID: > The `find` method now is > ```C++ > template > int find(T* token, bool f(T*, E)) const { > ... > > Any other functions which use this are also changed. > Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. Afshin Zafari has updated the pull request incrementally with two additional commits since the last revision: - Merge remote-tracking branch 'upstream/pr/15418' into pr_15418 - Suggested cleanups and tests ------------- Changes: - all: https://git.openjdk.org/jdk/pull/15418/files - new: https://git.openjdk.org/jdk/pull/15418/files/8d5f54ab..cb88988e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15418&range=10-11 Stats: 14 lines in 7 files changed: 5 ins; 6 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/15418.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/15418/head:pull/15418 PR: https://git.openjdk.org/jdk/pull/15418 From stefank at openjdk.org Thu Nov 23 16:18:43 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 16:18:43 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v4] In-Reply-To: References: Message-ID: > In the rewrites made for: > [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` > > I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. > > The provided tests provoke this assert form: > * the JNI thread detach code > * thread dumping with locked monitors, and > * the JVMTI GetOwnedMonitorInfo API. > > While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. > > The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. > > For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. > > Test: the written tests with and without the fix. Tier1-Tier3, so far. Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: Rewrite tests to prevent problem with native libs ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16783/files - new: https://git.openjdk.org/jdk/pull/16783/files/3239b822..9521b26d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=02-03 Stats: 299 lines in 7 files changed: 99 ins; 194 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/16783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16783/head:pull/16783 PR: https://git.openjdk.org/jdk/pull/16783 From stefank at openjdk.org Thu Nov 23 16:18:44 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 16:18:44 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v3] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 14:17:18 GMT, Magnus Ihse Bursie wrote: >> Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: >> >> Tweak test comments > > Build changes are trivially fine. Thanks @magicus. I'm removing the build label now. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16783#issuecomment-1824675404 From stefank at openjdk.org Thu Nov 23 16:18:45 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 16:18:45 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v3] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 11:52:38 GMT, Stefan Karlsson wrote: >> In the rewrites made for: >> [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` >> >> I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. >> >> The provided tests provoke this assert form: >> * the JNI thread detach code >> * thread dumping with locked monitors, and >> * the JVMTI GetOwnedMonitorInfo API. >> >> While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. >> >> The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. >> >> For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. >> >> Test: the written tests with and without the fix. Tier1-Tier3, so far. > > Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: > > Tweak test comments I reworked the MonitorWithDeadObject tests so that they all run in one class loader. Jtreg loads the tests in different class loaders, which causes problems because it's only allowed to load a library from *one* class loader. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16783#issuecomment-1824679121 From stefank at openjdk.org Thu Nov 23 16:20:14 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 23 Nov 2023 16:20:14 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v12] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 16:11:32 GMT, Afshin Zafari wrote: >> The `find` method now is >> ```C++ >> template >> int find(T* token, bool f(T*, E)) const { >> ... >> >> Any other functions which use this are also changed. >> Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. > > Afshin Zafari has updated the pull request incrementally with two additional commits since the last revision: > > - Merge remote-tracking branch 'upstream/pr/15418' into pr_15418 > - Suggested cleanups and tests Marked as reviewed by stefank (Reviewer). ------------- PR Review: https://git.openjdk.org/jdk/pull/15418#pullrequestreview-1746766388 From jbachorik at openjdk.org Thu Nov 23 16:25:29 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 16:25:29 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v7] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: Do expensive cleanup only in `purge_previous_version_list()` ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/eaf2720e..4c6a57e8 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=06 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=05-06 Stats: 25 lines in 3 files changed: 17 ins; 6 del; 2 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From jbachorik at openjdk.org Thu Nov 23 16:25:30 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 16:25:30 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v6] In-Reply-To: References: <4Sx6iriS9oGkl2xQRhOOzWMraDoP6DyvHtaCqkDn3IQ=.2a7e0fa3-81ad-4859-a57e-531179dddf3d@github.com> Message-ID: On Thu, 23 Nov 2023 15:46:10 GMT, Jaroslav Bachorik wrote: >> Sadly, this is not async-profiler specific. The same issue can be observed by JVMTI only code grabbing a stacktrace. >> What do you mean exactly by 'conditional'? Introducing a new JVM flag or something else? > > Ok, I see now - I could do the jmethodID maintenance only from `purge_previous_version_list()` call, leaving the proper metaspace deallocation untouched, therefore not adding unnecessary overhead there. I have modified the code to do jmethodID cleanup only when in `purge_previous_version_list()` - this should help with the added overhead. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403578138 From jbachorik at openjdk.org Thu Nov 23 16:28:06 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 16:28:06 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v2] In-Reply-To: References: <4m-e190SIpRd0-fkjZ4ja7mhVaQQsTwfqmWd8_82Els=.7b198822-3637-4109-b1e9-2451f882ce4b@github.com> <8-swJyF28KZeFLyqa2awML3umu2vog2Du9zVlJZfRDI=.5c435f8e-78af-47fa-a020-6bfce16b6c65@github.com> Message-ID: On Thu, 23 Nov 2023 16:02:18 GMT, Thomas Stuefe wrote: > sorry for not reading the description carefully. No worries. This is a rather convoluted issue. I don't mind being challenged - I am quite new to this part of the code and I don't want to accidentally break something ... ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1824691458 From jbachorik at openjdk.org Thu Nov 23 16:42:35 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Thu, 23 Nov 2023 16:42:35 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v8] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: Reinstate mistakenly deleted comment ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/4c6a57e8..46eff8d3 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=06-07 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From duke at openjdk.org Thu Nov 23 17:09:09 2023 From: duke at openjdk.org (suchismith1993) Date: Thu, 23 Nov 2023 17:09:09 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Wed, 22 Nov 2023 16:24:24 GMT, suchismith1993 wrote: >> J2SE agent does not start and throws error when it tries to find the shared library ibm_16_am. >> After searching for ibm_16_am.so ,the jvm agent throws and error as dll_load fails.It fails to identify the shared library ibm_16_am.a shared archive file on AIX. >> Hence we are providing a function which will additionally search for .a file on AIX ,when the search for .so file fails. > > suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: > > change macro position > I'm not a big fan of this approach. We accumulate more and more "#ifdef AIX" in shared code because of many recent AIX additions. No other platform has such a large ifdef footprint in shared code. > > I argue that all of this should be handled inside os_aix.cpp and not leak out into the external space: > > If .a is a valid shared object format on AIX, this should be handled in `os::dll_load()`, and be done for all shared objects. If not, why do we try to load a static archive via dlload in this case but not in other cases? > > _If_ this is needed in shared code, the string replacement function should be a generic utility function for all platforms, and it should be tested with a small gtest. A gtest would have likely uncovered the buffer overflow too. So i tried to check how to move the code to os_aix file. A few problems is see : 1. When i have to implemented the logic at dll_load function, i would have to repeat a lot of code after dlopen, i.e i have to call dlopen again for .so files and hence i have to copy the logic again for it. 2. Currently using function before dll_load,in the shared code makes this a bit easier. I have an alternate suggestion . Shall we declare the utlity function as part of os ? and implement it platform wise. In that way we just keep the actual implentation and aix and in windows and linux we keep it empty. So that way we can just call the utility function in shared code and it wouldnt affect other platform and will run the usecase for AIX. If that is not acceptable, then is there a better way to avoid repeating the dlopen again in os_aix file ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1824737158 From dfuchs at openjdk.org Thu Nov 23 17:17:03 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 23 Nov 2023 17:17:03 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: Message-ID: <6R7B_a7VdF5Fg8B38rMhx0Fe1WT53KQw0yt596rzmS4=.9902a48f-f143-4d81-8940-d9c273e94812@github.com> On Thu, 23 Nov 2023 12:43:33 GMT, Kevin Walls wrote: >>> An exception here will prevent the class from being initialized... >> >> Maybe we can break it, but this new property is following the same pattern I think as the neighbouring file src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java when it reads the system props. I see Integer.getInteger handles the obvious Exceptions, so specifying a strange value does not immediately break us. >> >> If there's more protection needed then we have various other places to apply it that might need separate follow-up if you think there's a case? > >> IIRC, the default agent uses / may use its own socket factories - are we still coming here even with those factories? > > We do get here, yes (not saying you can't customise your way out of getting here). This is a stack from a test I was experimenting with, when it did see the timeout: > > > ---------System.out:(4/132)---------- > JMX URL = service:jmx:rmi://x > expectTimeout = true > sun.rmi.transport.tcp.initialConnectTimeout = 1 > Test passed > ----------System.err:(24/1791)---------- > java.rmi.ConnectIOException: Exception creating connection to: x.x.x.x; nested exception is: > java.net.SocketTimeoutException > at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:637) > at java.rmi/sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:217) > at java.rmi/sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:204) > at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:134) > at java.management.rmi/javax.management.remote.rmi.RMIServerImpl_Stub.newClient(RMIServerImpl_Stub.java:85) > at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2106) > at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:321) > at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) > at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) > at RMIConnectionTimeoutTest.main(RMIConnectionTimeoutTest.java:65) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) > at java.base/java.lang.Thread.run(Thread.java:1570) > Caused by: java.net.SocketTimeoutException > at java.base/java.net.SocksSocketImpl.remainingMillis(SocksSocketImpl.java:112) > at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) > at java.base/java.net.Socket.connect(Socket.java:752) > at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createSocket(TCPDirectSocketFactory.java:57) > at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) > ... 13 more > STATUS:Passed. > I see Integer.getInteger handles the obvious Exceptions, so specifying a strange value does not immediately break us. Oh - right. It's `Integer.getInteger`, not `Integer.parseInt` Ok, then - I withdraw my first comment :-) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1403618437 From dfuchs at openjdk.org Thu Nov 23 17:20:05 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 23 Nov 2023 17:20:05 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: <6R7B_a7VdF5Fg8B38rMhx0Fe1WT53KQw0yt596rzmS4=.9902a48f-f143-4d81-8940-d9c273e94812@github.com> References: <6R7B_a7VdF5Fg8B38rMhx0Fe1WT53KQw0yt596rzmS4=.9902a48f-f143-4d81-8940-d9c273e94812@github.com> Message-ID: On Thu, 23 Nov 2023 17:14:52 GMT, Daniel Fuchs wrote: >>> IIRC, the default agent uses / may use its own socket factories - are we still coming here even with those factories? >> >> We do get here, yes (not saying you can't customise your way out of getting here). This is a stack from a test I was experimenting with, when it did see the timeout: >> >> >> ---------System.out:(4/132)---------- >> JMX URL = service:jmx:rmi://x >> expectTimeout = true >> sun.rmi.transport.tcp.initialConnectTimeout = 1 >> Test passed >> ----------System.err:(24/1791)---------- >> java.rmi.ConnectIOException: Exception creating connection to: x.x.x.x; nested exception is: >> java.net.SocketTimeoutException >> at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:637) >> at java.rmi/sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:217) >> at java.rmi/sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:204) >> at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:134) >> at java.management.rmi/javax.management.remote.rmi.RMIServerImpl_Stub.newClient(RMIServerImpl_Stub.java:85) >> at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2106) >> at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:321) >> at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) >> at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:229) >> at RMIConnectionTimeoutTest.main(RMIConnectionTimeoutTest.java:65) >> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) >> at java.base/java.lang.reflect.Method.invoke(Method.java:580) >> at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138) >> at java.base/java.lang.Thread.run(Thread.java:1570) >> Caused by: java.net.SocketTimeoutException >> at java.base/java.net.SocksSocketImpl.remainingMillis(SocksSocketImpl.java:112) >> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) >> at java.base/java.net.Socket.connect(Socket.java:752) >> at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createSocket(TCPDirectSocketFactory.java:57) >> at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) >> ... 13 more >> STATUS:Passed. > >> I see Integer.getInteger handles the obvious Exceptions, so specifying a strange value does not immediately break us. > > Oh - right. It's `Integer.getInteger`, not `Integer.parseInt` Ok, then - I withdraw my first comment :-) > This is a stack from a test I was experimenting with, when it did see the timeout: Ah - that's for testing with a particular test. So the question now is: Should the RMISocketFactories provided / implemented in the JMX implementation - such as those used by the JMX default agent, also honour this system property? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1403620057 From dfuchs at openjdk.org Thu Nov 23 17:24:03 2023 From: dfuchs at openjdk.org (Daniel Fuchs) Date: Thu, 23 Nov 2023 17:24:03 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: <6R7B_a7VdF5Fg8B38rMhx0Fe1WT53KQw0yt596rzmS4=.9902a48f-f143-4d81-8940-d9c273e94812@github.com> Message-ID: On Thu, 23 Nov 2023 17:17:34 GMT, Daniel Fuchs wrote: >>> I see Integer.getInteger handles the obvious Exceptions, so specifying a strange value does not immediately break us. >> >> Oh - right. It's `Integer.getInteger`, not `Integer.parseInt` Ok, then - I withdraw my first comment :-) > >> This is a stack from a test I was experimenting with, when it did see the timeout: > > Ah - that's for testing with a particular test. So the question now is: > > Should the RMISocketFactories provided / implemented in the JMX implementation - such as those used by the JMX default agent, also honour this system property? (Look for socket factories in the module `jdk.management.agent`) ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1403622189 From stuefe at openjdk.org Thu Nov 23 18:06:05 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Thu, 23 Nov 2023 18:06:05 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Thu, 23 Nov 2023 17:05:29 GMT, suchismith1993 wrote: > > I'm not a big fan of this approach. We accumulate more and more "#ifdef AIX" in shared code because of many recent AIX additions. No other platform has such a large ifdef footprint in shared code. > > I argue that all of this should be handled inside os_aix.cpp and not leak out into the external space: > > If .a is a valid shared object format on AIX, this should be handled in `os::dll_load()`, and be done for all shared objects. If not, why do we try to load a static archive via dlload in this case but not in other cases? > > _If_ this is needed in shared code, the string replacement function should be a generic utility function for all platforms, and it should be tested with a small gtest. A gtest would have likely uncovered the buffer overflow too. > > So i tried to check how to move the code to os_aix file. A few problems is see : > > 1. When i have to implemented the logic at dll_load function, i would have to repeat a lot of code after dlopen, i.e i have to call dlopen again for .so files and hence i have to copy the logic again for it. > 2. Currently using function before dll_load,in the shared code makes this a bit easier. > I have an alternate suggestion . > Shall we declare the utlity function as part of os ? and implement it platform wise. Not without any need. If this is an AIX specific issue, it should be handed in os::dll_load on AIX. > In that way we just keep the actual implentation and aix and in windows and linux we keep it empty. > So that way we can just call the utility function in shared code and it wouldnt affect other platform and will run the usecase for AIX. > If that is not acceptable, then is there a better way to avoid repeating the dlopen again in os_aix file ? I don't understand the problem. What is preventing you from using a file local scope utility function inside os::dll_load? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1824787258 From duke at openjdk.org Thu Nov 23 18:30:05 2023 From: duke at openjdk.org (suchismith1993) Date: Thu, 23 Nov 2023 18:30:05 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Thu, 23 Nov 2023 18:03:33 GMT, Thomas Stuefe wrote: > > > I'm not a big fan of this approach. We accumulate more and more "#ifdef AIX" in shared code because of many recent AIX additions. No other platform has such a large ifdef footprint in shared code. > > > I argue that all of this should be handled inside os_aix.cpp and not leak out into the external space: > > > If .a is a valid shared object format on AIX, this should be handled in `os::dll_load()`, and be done for all shared objects. If not, why do we try to load a static archive via dlload in this case but not in other cases? > > > _If_ this is needed in shared code, the string replacement function should be a generic utility function for all platforms, and it should be tested with a small gtest. A gtest would have likely uncovered the buffer overflow too. > > > > > > So i tried to check how to move the code to os_aix file. A few problems is see : > > > > 1. When i have to implemented the logic at dll_load function, i would have to repeat a lot of code after dlopen, i.e i have to call dlopen again for .so files and hence i have to copy the logic again for it. > > 2. Currently using function before dll_load,in the shared code makes this a bit easier. > > I have an alternate suggestion . > > Shall we declare the utlity function as part of os ? and implement it platform wise. > > Not without any need. If this is an AIX specific issue, it should be handed in os::dll_load on AIX. > > > In that way we just keep the actual implentation and aix and in windows and linux we keep it empty. > > So that way we can just call the utility function in shared code and it wouldnt affect other platform and will run the usecase for AIX. > > If that is not acceptable, then is there a better way to avoid repeating the dlopen again in os_aix file ? > > I don't understand the problem. What is preventing you from using a file local scope utility function inside os::dll_load? i would have to repeat the line 1132 and 1139 in os_aix.cpp again , if the condition fails for .so files, because i have to reload it again and check if the .a exists. In the shared code i had repeat less number of lines i believe. Do you suggest moving lines 1132 to 1139 to another function then ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1824804477 From azafari at openjdk.org Thu Nov 23 22:20:21 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Thu, 23 Nov 2023 22:20:21 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: References: Message-ID: On Sun, 29 Oct 2023 08:07:55 GMT, Kim Barrett wrote: >> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). > >> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). > > Not touched since 2008 suggests to me there might not be a rush to make the change as proposed, and instead take > the (I think small) additional time to do the better thing, e.g. the unary-predicate suggestion made by several folks. Dear @kimbarrett , @dholmes-ora , @stefank, @rose00 , @jdksjolen, @sspitsyn, @merykitty, Thank you all. The final code is now cleaner and nicer. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1824962932 From azafari at openjdk.org Thu Nov 23 22:20:21 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Thu, 23 Nov 2023 22:20:21 GMT Subject: Integrated: 8314502: Change the comparator taking version of GrowableArray::find to be a template method In-Reply-To: References: Message-ID: On Thu, 24 Aug 2023 14:09:46 GMT, Afshin Zafari wrote: > The `find` method now is > ```C++ > template > int find(T* token, bool f(T*, E)) const { > ... > > Any other functions which use this are also changed. > Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. This pull request has now been integrated. Changeset: 14557e72 Author: Afshin Zafari URL: https://git.openjdk.org/jdk/commit/14557e72ef55c6161a3fa0c1960f7be618a34bf1 Stats: 155 lines in 13 files changed: 98 ins; 32 del; 25 mod 8314502: Change the comparator taking version of GrowableArray::find to be a template method Reviewed-by: jsjolen, sspitsyn, stefank ------------- PR: https://git.openjdk.org/jdk/pull/15418 From duke at openjdk.org Fri Nov 24 04:48:16 2023 From: duke at openjdk.org (duke) Date: Fri, 24 Nov 2023 04:48:16 GMT Subject: Withdrawn: 8313997: The jdi/ListeningConnector/startListening/startlis001 may fail if the hosts file is modified In-Reply-To: References: Message-ID: <2v9r9UgGEloHyhromQoCVxkaI4t9_9HKCtAnrXNxnnI=.8b11cf7e-8ea6-44e8-ac1e-78cb3ad6455b@github.com> On Wed, 9 Aug 2023 07:46:58 GMT, Sergey Bylokhov wrote: > The test uses this code to create a list of valid addresses for the localhost: > > String hostname = "localhost"; > List validAddresses = new LinkedList<>(); > validAddresses.add(hostname); > Arrays.stream(InetAddress.getAllByName(hostname)) > .forEach(address -> validAddresses.add(address.getHostAddress())); > > It does not work properly if the custom name is set for the 127.0.01 via /etc/hosts file. In that case, the address returned by the "startListen()" will be "SomeCustomName:port", for example we can return it here: > https://github.com/openjdk/jdk/blob/360f65d7b15b327e2f160c42f318945cc6548bda/src/jdk.jdi/share/classes/com/sun/tools/jdi/SocketTransportService.java#L82 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.org/jdk/pull/15203 From dholmes at openjdk.org Fri Nov 24 05:52:16 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 24 Nov 2023 05:52:16 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v8] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 16:42:35 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: > > Reinstate mistakenly deleted comment This has gotten a lot more complicated. All I was suggesting was if this: if (with_method_holders) { method->clear_jmethod_id(); } could be changed to if (method->method_holder() == nullptr) { method->clear_jmethod_id(); } Now I'm not at all sure what you are doing. src/hotspot/share/oops/instanceKlass.hpp line 1084: > 1082: inline void release_set_methods_jmethod_ids(jmethodID* jmeths); > 1083: // Used to explicitly clear jmethodIDs for the contained methods > 1084: // This is required for JDK-8313816 but should not be used otherwise! This comment is not helpful - what does it actually mean? Referring to a bugid doesn't help. ------------- PR Review: https://git.openjdk.org/jdk/pull/16662#pullrequestreview-1747340307 PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403962272 From jpai at openjdk.org Fri Nov 24 06:11:27 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 06:11:27 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently Message-ID: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. ------------- Commit messages: - 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently Changes: https://git.openjdk.org/jdk/pull/16805/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16805&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320687 Stats: 133 lines in 2 files changed: 116 ins; 5 del; 12 mod Patch: https://git.openjdk.org/jdk/pull/16805.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16805/head:pull/16805 PR: https://git.openjdk.org/jdk/pull/16805 From jpai at openjdk.org Fri Nov 24 06:17:07 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 06:17:07 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently In-Reply-To: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 06:06:16 GMT, Jaikiran Pai wrote: > The fix proposes to guard the usage of the shared ServiceLoader instance through the monitoredHosts object monitor. An alternative was to just create a new instance of `ServiceLoader` within the method implementation whenever a `MonitoredHost` was not found in the `monitoredHosts`. That works fine too. However, that will have a (relative) disadvantage that we keep loading the MonitoredHostService instances afresh and thus loses the benefits that using a single `ServiceLoader` instance provides (`ServiceLoader` instance internally maintains a cache of already loaded `Service` instances as noted in its javadoc). So I decided to stick to using a single instance of `ServiceLoader` and have its access guarded by synchronization. Loading and instantiating `MonitoredHostService` instances within this synchronized block, I think, shouldn't cause issues because the `ServiceLoader`'s load method is passed the classloader of `sun.jvmstat.monitor.MonitoredHostService` class, which happens to be the `jdk.internal.jvmstat` JDK module. Thus it wouldn't involve unexpected user/application class instances being loaded while holding this lock. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16805#issuecomment-1825194163 From alanb at openjdk.org Fri Nov 24 06:27:08 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 24 Nov 2023 06:27:08 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently In-Reply-To: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 06:06:16 GMT, Jaikiran Pai wrote: > Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? > > As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. > > The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). > > A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. > > tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. Good sleuthing. src/jdk.internal.jvmstat/share/classes/sun/jvmstat/monitor/MonitoredHost.java line 142: > 140: // not thread safe, access MUST be guarded through synchronization on "monitoredHosts" > 141: // field > 142: private static final ServiceLoader monitoredHostServiceLoader = Can you put `assert Threads.holdLock(monitoredHosts)` at the top of the method, that will check that the monitor is held? Probably should fix the comment too, a bit strange to have a /* .. */ and // comment on the same method, looks like the style used in this code is /* .. */. test/jdk/sun/jvmstat/monitor/MonitoredVm/GetMonitoredHost.java line 43: > 41: * @run main/othervm GetMonitoredHost > 42: */ > 43: public class GetMonitoredHost { This is more of a regression test for the concurrent case rather than a unit test, so I think rename to Concurrent GetMonitoredHost to make it clearer what the test is far. ------------- PR Review: https://git.openjdk.org/jdk/pull/16805#pullrequestreview-1747367518 PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1403978916 PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1403979304 From dholmes at openjdk.org Fri Nov 24 06:27:28 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 24 Nov 2023 06:27:28 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 22:15:35 GMT, Afshin Zafari wrote: >>> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). >> >> Not touched since 2008 suggests to me there might not be a rush to make the change as proposed, and instead take >> the (I think small) additional time to do the better thing, e.g. the unary-predicate suggestion made by several folks. > > Dear @kimbarrett , @dholmes-ora , @stefank, @rose00 , @jdksjolen, @sspitsyn, @merykitty, > Thank you all. > The final code is now cleaner and nicer. @afshin-zafari I'm happy to hear the code is now cleaner and nicer, but AFAICS this new version of the code has only had a single review. It should really have been re-reviewed by at least one of the earlier reviewers. Thanks. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1825200669 From dholmes at openjdk.org Fri Nov 24 06:46:08 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 24 Nov 2023 06:46:08 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v4] In-Reply-To: References: Message-ID: <-YLzEt2tPsupH0CLU6278f4yX3si2I60OvHfDitr-tM=.e5d3dad2-b846-4da5-812f-ea2600cd2780@github.com> On Thu, 23 Nov 2023 08:40:55 GMT, Stefan Karlsson wrote: >> src/hotspot/share/runtime/vmOperations.cpp line 354: >> >>> 352: // alive. Filter out monitors with dead objects. >>> 353: return; >>> 354: } >> >> I don't think we need to do this, but even without this filtering I ran a number of tests and was unable to demonstrate any problem. The JNI locked monitor seems to be "invisible" to the frame that locked it and so the thread dump never encounters it. Were you able to provoke a failure here or is this defensive programming? > > I provoked test failures for all paths I filtered. If I remove this check and run: > > make -C ../build/fastdebug test TEST=test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java JTREG="JAVA_OPTIONS=-XX:+UseZGC" > > > I hit this assert: > > # Internal Error (/home/stefank/git/jdk/open/src/hotspot/share/services/management.cpp:1274), pid=1546709, tid=1546754 > # assert(object != nullptr) failed: must be a Java object > ... > V [libjvm.so+0x1330ce8] jmm_DumpThreads+0x1a48 (management.cpp:1274) > j sun.management.ThreadImpl.dumpThreads0([JZZI)[Ljava/lang/management/ThreadInfo;+0 java.management at 22-internal > j sun.management.ThreadImpl.dumpAllThreads(ZZI)[Ljava/lang/management/ThreadInfo;+28 java.management at 22-internal > j sun.management.ThreadImpl.dumpAllThreads(ZZ)[Ljava/lang/management/ThreadInfo;+5 java.management at 22-internal > j IterateMonitorWithDeadObjectTest.dumpThreadsWithLockedMonitors()V+7 > j IterateMonitorWithDeadObjectTest.main([Ljava/lang/String;)V+11 > > > If I remove that assert I hit an NPE in the Java layer: > > java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "lock" is null > at java.management/java.lang.management.ThreadInfo.(ThreadInfo.java:172) > at java.management/sun.management.ThreadImpl.dumpThreads0(Native Method) > at java.management/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:518) > at java.management/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:506) > at IterateMonitorWithDeadObjectTest.dumpThreadsWithLockedMonitors(IterateMonitorWithDeadObjectTest.java:44) > at IterateMonitorWithDeadObjectTest.main(IterateMonitorWithDeadObjectTest.java:66) > at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base/java.lang.reflect.Method.invoke(Method.java:580) > at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333) > at java.base/java.lang.Thread.run(Thread.java:1570) Thanks for that. Looks like JMM thread dump is different to VM Thread dump. Okay we definitely need RFEs to look into how to handle this. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1403992144 From stuefe at openjdk.org Fri Nov 24 06:53:13 2023 From: stuefe at openjdk.org (Thomas Stuefe) Date: Fri, 24 Nov 2023 06:53:13 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v8] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 16:42:35 GMT, Jaroslav Bachorik wrote: >> Please, review this fix for a corner case handling of `jmethodID` values. >> >> The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. >> Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. >> >> If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. >> However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. >> This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. >> >> This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. >> >> Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. >> >> _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ > > Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: > > Reinstate mistakenly deleted comment src/hotspot/share/oops/instanceKlass.hpp line 1087: > 1085: // - We can not use the jmethodID cache associated with klass directly because the 'previous' versions > 1086: // do not have the jmethodID cache filled in. Instead, we need to lookup jmethodID for each method and this > 1087: // is expensive - O(n) for one jmethodID lookup. For all contained methods it is O(n^2). The comment is helpful, but its an implementation comment, and I would move this part to the implementation. Here, what matters to know is that this function nulls out jmethodIDs for all methods in this IK. The comment also refers to class transformation specifically, I'd mention that so that readers get a context. Do I understand correctly that this comment tries to explain why we - instead of just iterating IK->_methods_jmethod_ids directly - we iterate all methods of this IK and then lookup the associated jmethodID in IK->_methods_jmethod_ids, right? I wondered about that but IIUC the pointers inside IK->_methods_jmethod_ids may refer to jmethodID slots that have been reused for different methods? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1403995913 From jpai at openjdk.org Fri Nov 24 06:57:33 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 06:57:33 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v2] In-Reply-To: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: > Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? > > As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. > > The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). > > A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. > > tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: - Alan's review suggestion - rename GetMonitoredHost to ConcurrentGetMonitoredHost - fix code comment ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16805/files - new: https://git.openjdk.org/jdk/pull/16805/files/acc9e3dd..78e5cd7b Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16805&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16805&range=00-01 Stats: 7 lines in 2 files changed: 2 ins; 2 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/16805.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16805/head:pull/16805 PR: https://git.openjdk.org/jdk/pull/16805 From jpai at openjdk.org Fri Nov 24 06:57:37 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 06:57:37 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v2] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 06:23:03 GMT, Alan Bateman wrote: >> Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: >> >> - Alan's review suggestion - rename GetMonitoredHost to ConcurrentGetMonitoredHost >> - fix code comment > > src/jdk.internal.jvmstat/share/classes/sun/jvmstat/monitor/MonitoredHost.java line 142: > >> 140: // not thread safe, access MUST be guarded through synchronization on "monitoredHosts" >> 141: // field >> 142: private static final ServiceLoader monitoredHostServiceLoader = > > Can you put `assert Threads.holdLock(monitoredHosts)` at the top of the method, that will check that the monitor is held? Probably should fix the comment too, a bit strange to have a /* .. */ and // comment on the same method, looks like the style used in this code is /* .. */. Hello Alan, I've updated the PR to fix the code comment section to use the existing `/* */` style. > Can you put assert Threads.holdLock(monitoredHosts) at the top of the method, that will check that the monitor is held? Right now, the sole usage of the `monitoredHostServiceLoader` instance is within a `synchronized (monitoredHosts) {...}` block within a method. So it wouldn't require this `assert`. > test/jdk/sun/jvmstat/monitor/MonitoredVm/GetMonitoredHost.java line 43: > >> 41: * @run main/othervm GetMonitoredHost >> 42: */ >> 43: public class GetMonitoredHost { > > This is more of a regression test for the concurrent case rather than a unit test, so I think rename to Concurrent GetMonitoredHost to make it clearer what the test is far. Done. I've now updated the PR to rename this new test to ConcurrentGetMonitoredHost. Test continues to pass. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1403999159 PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1403997773 From azafari at openjdk.org Fri Nov 24 08:02:21 2023 From: azafari at openjdk.org (Afshin Zafari) Date: Fri, 24 Nov 2023 08:02:21 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: References: Message-ID: <-zu_qHuhBvXKpJp4e0Hn1tE1a2Gsk4GpRBi-kapasFo=.362a1fd6-03b9-4b9f-a22f-c5df4ab243f0@github.com> On Thu, 23 Nov 2023 22:15:35 GMT, Afshin Zafari wrote: >>> I still approve of this patch as it's better than what we had before. There are a lot of suggested improvements that can be done either in this PR or in a future RFE. `git blame` shows that this hasn't been touched since 2008, so I don't think applying all suggestions now is in any sense critical :-). >> >> Not touched since 2008 suggests to me there might not be a rush to make the change as proposed, and instead take >> the (I think small) additional time to do the better thing, e.g. the unary-predicate suggestion made by several folks. > > Dear @kimbarrett , @dholmes-ora , @stefank, @rose00 , @jdksjolen, @sspitsyn, @merykitty, > Thank you all. > The final code is now cleaner and nicer. > @afshin-zafari I'm happy to hear the code is now cleaner and nicer, but AFAICS this new version of the code has only had a single review. It should really have been re-reviewed by at least one of the earlier reviewers. Thanks. Oh, really sorry. That's obviously my mistake. Your are right, I should have waited for others. What to do now? ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1825277911 From stefank at openjdk.org Fri Nov 24 08:04:06 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 24 Nov 2023 08:04:06 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v4] In-Reply-To: <-YLzEt2tPsupH0CLU6278f4yX3si2I60OvHfDitr-tM=.e5d3dad2-b846-4da5-812f-ea2600cd2780@github.com> References: <-YLzEt2tPsupH0CLU6278f4yX3si2I60OvHfDitr-tM=.e5d3dad2-b846-4da5-812f-ea2600cd2780@github.com> Message-ID: On Fri, 24 Nov 2023 06:43:35 GMT, David Holmes wrote: >> I provoked test failures for all paths I filtered. If I remove this check and run: >> >> make -C ../build/fastdebug test TEST=test/hotspot/jtreg/runtime/Monitor/IterateMonitorWithDeadObjectTest.java JTREG="JAVA_OPTIONS=-XX:+UseZGC" >> >> >> I hit this assert: >> >> # Internal Error (/home/stefank/git/jdk/open/src/hotspot/share/services/management.cpp:1274), pid=1546709, tid=1546754 >> # assert(object != nullptr) failed: must be a Java object >> ... >> V [libjvm.so+0x1330ce8] jmm_DumpThreads+0x1a48 (management.cpp:1274) >> j sun.management.ThreadImpl.dumpThreads0([JZZI)[Ljava/lang/management/ThreadInfo;+0 java.management at 22-internal >> j sun.management.ThreadImpl.dumpAllThreads(ZZI)[Ljava/lang/management/ThreadInfo;+28 java.management at 22-internal >> j sun.management.ThreadImpl.dumpAllThreads(ZZ)[Ljava/lang/management/ThreadInfo;+5 java.management at 22-internal >> j IterateMonitorWithDeadObjectTest.dumpThreadsWithLockedMonitors()V+7 >> j IterateMonitorWithDeadObjectTest.main([Ljava/lang/String;)V+11 >> >> >> If I remove that assert I hit an NPE in the Java layer: >> >> java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "lock" is null >> at java.management/java.lang.management.ThreadInfo.(ThreadInfo.java:172) >> at java.management/sun.management.ThreadImpl.dumpThreads0(Native Method) >> at java.management/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:518) >> at java.management/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:506) >> at IterateMonitorWithDeadObjectTest.dumpThreadsWithLockedMonitors(IterateMonitorWithDeadObjectTest.java:44) >> at IterateMonitorWithDeadObjectTest.main(IterateMonitorWithDeadObjectTest.java:66) >> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) >> at java.base/java.lang.reflect.Method.invoke(Method.java:580) >> at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333) >> at java.base/java.lang.Thread.run(Thread.java:1570) > > Thanks for that. Looks like JMM thread dump is different to VM Thread dump. Okay we definitely need RFEs to look into how to handle this. Will you create the RFE? I'm not as convinced that this is something that needs to be fixed, so it would be better if you create the RFE with the proper motivation. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1404046328 From stefank at openjdk.org Fri Nov 24 08:38:26 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 24 Nov 2023 08:38:26 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v3] In-Reply-To: <-zu_qHuhBvXKpJp4e0Hn1tE1a2Gsk4GpRBi-kapasFo=.362a1fd6-03b9-4b9f-a22f-c5df4ab243f0@github.com> References: <-zu_qHuhBvXKpJp4e0Hn1tE1a2Gsk4GpRBi-kapasFo=.362a1fd6-03b9-4b9f-a22f-c5df4ab243f0@github.com> Message-ID: <-4W-9hUd7pls0UYyItZ9kF0jnWHIjv2Lk9u4noy0GMY=.334e45a7-75c7-4bf1-93ff-ffec226545ee@github.com> On Fri, 24 Nov 2023 07:59:51 GMT, Afshin Zafari wrote: > > @afshin-zafari I'm happy to hear the code is now cleaner and nicer, but AFAICS this new version of the code has only had a single review. It should really have been re-reviewed by at least one of the earlier reviewers. Thanks. > > Oh, really sorry. That's obviously my mistake. Your are right, I should have waited for others. What to do now? I think the other reviewers can give their review feedback then you create a new RFE to fix whatever they'd like to get fixed. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1825313436 From stefank at openjdk.org Fri Nov 24 09:18:46 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Fri, 24 Nov 2023 09:18:46 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v5] In-Reply-To: References: Message-ID: > In the rewrites made for: > [JDK-8318757](https://bugs.openjdk.org/browse/JDK-8318757) `VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls` > > I removed the filtering of *owned ObjectMonitors with dead objects*. The reasoning was that you should never have an owned ObjectMonitor with a dead object. I added an assert to check this assumption. It turns out that the assumption was wrong *if* you use JNI to call MonitorEnter and then remove all references to the locked object. > > The provided tests provoke this assert form: > * the JNI thread detach code > * thread dumping with locked monitors, and > * the JVMTI GetOwnedMonitorInfo API. > > While investigating this we've found that the thread detach code becomes more correct when this filter was removed. Previously, the locked monitors never got unlocked because the ObjectMonitor iterator never exposed these monitors to the JNI detach code that unlocks the thread's monitors. That bug caused an ObjectMonitor leak. So, for this case I'm leaving these ObjectMonitors unfiltered so that we don't reintroduce the leak. > > The thread dumping case doesn't tolerate ObjectMonitor with dead objects, so I'm filtering those in the closure that collects ObjectMonitor. Side note: We have discussions about ways to completely rewrite this by letting each thread have thread-local information about JNI held locks. If we have this we could probably throw away the entire ObjectMonitorDump hashtable, and its walk of the `_in_use_list.`. > > For GetOwnedMonitorInfo it is unclear if we should expose these weird ObjectMonitor. If we do, then the users can detect that a thread holds a lock with a dead object, and the code will return NULL as one of the "owned monitors" returned. I don't think that's a good idea, so I'm filtering out these ObjectMonitor for those calls. > > Test: the written tests with and without the fix. Tier1-Tier3, so far. Stefan Karlsson has updated the pull request incrementally with one additional commit since the last revision: Split test and use othervm ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16783/files - new: https://git.openjdk.org/jdk/pull/16783/files/9521b26d..bad51926 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16783&range=03-04 Stats: 29 lines in 1 file changed: 23 ins; 1 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/16783.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16783/head:pull/16783 PR: https://git.openjdk.org/jdk/pull/16783 From jsjolen at openjdk.org Fri Nov 24 10:03:26 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Fri, 24 Nov 2023 10:03:26 GMT Subject: RFR: 8314502: Change the comparator taking version of GrowableArray::find to be a template method [v12] In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 16:11:32 GMT, Afshin Zafari wrote: >> The `find` method now is >> ```C++ >> template >> int find(T* token, bool f(T*, E)) const { >> ... >> >> Any other functions which use this are also changed. >> Local linux-x64-debug hotspot:tier1 passed. Mach5 tier1 build on linux and Windows passed. > > Afshin Zafari has updated the pull request incrementally with two additional commits since the last revision: > > - Merge remote-tracking branch 'upstream/pr/15418' into pr_15418 > - Suggested cleanups and tests Still LGTM. ------------- PR Comment: https://git.openjdk.org/jdk/pull/15418#issuecomment-1825422568 From alanb at openjdk.org Fri Nov 24 10:35:28 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 24 Nov 2023 10:35:28 GMT Subject: RFR: 8320532: Remove Thread/ThreadGroup suspend/resume Message-ID: The deadlock prone Thread/ThreadGroup suspend/resume were deprecated since JDK 1.2, deprecated for removal in Java 14, and re-specified/degraded to throw UnsupportedOperationException unconditionally in Java 19/20. Early in Java 23 seems a fine time to finally remove these methods. Corpus analysis of 176 million classes in 485k artifacts found no remaining usages of ThreadGroup.suspend/resume beyond the artifacts that include a copy of java.lang.ThreadGroup (!). It found 87 remaining uses of Thread.suspend and 86 remaining usages of Thread.resume, some of these are tests. Thread.suspend/resume have always linked to the "Java Thread Primitive Deprecation" page. This originally explained the reasons why suspend/resume were deprecated. When these methods were degraded to throw UOE we changed the text to explain why the ability to suspend or resume a thread was removed. Now we must change it again. One choice is to re-word to explain why the Java APIs were removed or why the Java APIs don't define a way to suspend/resume threads, the other choice (which I prefer) is to remove the text. The method description of java.lang.management.ThreadInfo.isSuspended is tweaked to link to JVMTI SuspendThread instead of Thread.suspend ------------- Commit messages: - Clarify ThreadInfo.isSuspended - Initial commit Changes: https://git.openjdk.org/jdk/pull/16789/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16789&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8320532 Stats: 552 lines in 9 files changed: 116 ins; 430 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/16789.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16789/head:pull/16789 PR: https://git.openjdk.org/jdk/pull/16789 From kevinw at openjdk.org Fri Nov 24 10:48:04 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Fri, 24 Nov 2023 10:48:04 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v2] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 06:57:33 GMT, Jaikiran Pai wrote: >> Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? >> >> As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. >> >> The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). >> >> A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. >> >> tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. > > Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: > > - Alan's review suggestion - rename GetMonitoredHost to ConcurrentGetMonitoredHost > - fix code comment Yes this looks useful 8-) sync on monitoredHosts protects monitoredHosts and monitoredHostServiceLoader. Do we really need both synchronized(monitoredHosts) blocks -- is the first one needed, now at lines 159 -164 ? If we don't, then you might not feel the need to create the getCachedMonitoredHost() method which separates the locking from the access you want to protect. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16805#issuecomment-1825484587 From jbachorik at openjdk.org Fri Nov 24 11:27:08 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Fri, 24 Nov 2023 11:27:08 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v8] In-Reply-To: References: Message-ID: <039WrJCh_1K3zhoGN8MlABuPMgPJ4Kom7XYCl5glqxY=.f8369b03-0f87-40d8-9763-1df4f0043815@github.com> On Fri, 24 Nov 2023 05:47:14 GMT, David Holmes wrote: >> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: >> >> Reinstate mistakenly deleted comment > > This has gotten a lot more complicated. All I was suggesting was if this: > > if (with_method_holders) { > method->clear_jmethod_id(); > } > > could be changed to > > if (method->method_holder() == nullptr) { > method->clear_jmethod_id(); > } > > Now I'm not at all sure what you are doing. @dholmes-ora Unfortunately, I can not just do `method->method_holder() == nullptr` as `method_holder()` is expanding to `Method::constants()->pool_holder()` and `Method::constants()` is expanding to `Method::constMethod()->constants()`. This can cause SIGSEGV if either `Method::_constMethod` or `ConstMethod::_constants` is NULL. ------------- PR Comment: https://git.openjdk.org/jdk/pull/16662#issuecomment-1825532431 From jpai at openjdk.org Fri Nov 24 11:28:25 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 11:28:25 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v3] In-Reply-To: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: <1IvAf18lR_PiowmOG2gQ7qykjtGvPq16S6mwhjZwhVU=.a32338f2-d923-4049-b187-0b9059a8e7b4@github.com> > Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? > > As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. > > The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). > > A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. > > tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision: Kevin's inputs - simplify the synchronized block ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16805/files - new: https://git.openjdk.org/jdk/pull/16805/files/78e5cd7b..669e5e5e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16805&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16805&range=01-02 Stats: 39 lines in 1 file changed: 7 ins; 28 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/16805.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16805/head:pull/16805 PR: https://git.openjdk.org/jdk/pull/16805 From jpai at openjdk.org Fri Nov 24 11:31:07 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 11:31:07 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v2] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 06:57:33 GMT, Jaikiran Pai wrote: >> Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? >> >> As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. >> >> The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). >> >> A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. >> >> tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. > > Jaikiran Pai has updated the pull request incrementally with two additional commits since the last revision: > > - Alan's review suggestion - rename GetMonitoredHost to ConcurrentGetMonitoredHost > - fix code comment Hello Kevin, > Do we really need both synchronized(monitoredHosts) blocks -- is the first one needed, now at lines 159 -164 ? If we don't, then you might not feel the need to create the getCachedMonitoredHost() method which separates the locking from the access you want to protect. That's a good point. I have taken your input and updated the PR to simplify the synchronized block. The test continues to pass locally (I'll re-run tier testing and svc_tools later today). ------------- PR Comment: https://git.openjdk.org/jdk/pull/16805#issuecomment-1825536913 From jbachorik at openjdk.org Fri Nov 24 11:42:09 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Fri, 24 Nov 2023 11:42:09 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v8] In-Reply-To: References: Message-ID: <3Sf09SHxmBuy_R7aIUi3zZ7hdvr_HWtXmNlTY05XSqc=.bd601b5c-be27-48ca-ac89-0b4ab41c74a7@github.com> On Fri, 24 Nov 2023 06:49:14 GMT, Thomas Stuefe wrote: >I wondered about that but IIUC the pointers inside IK->_methods_jmethod_ids may refer to jmethodID slots that have been reused for different methods? Yes. The reason is that if a class has previous versions, these versions do not contain their own jmethodID cache but rather delegate to the 'main' version. So we have nothing to iterate over when a previous class version gets unloaded - also, to make things more interesting, the previous method versions are pointing to the main class version as their `method_holder`. Making it impossible to iterate over the full jmethodID cache in the main class version and nulling out a jmethodID only if the method it points to has `method_holder` pointing to the class being unloaded (it will always point to the main class version ...). If there is an easy way out of this without incurring O(n^2) complexity, please speak up. I was not able to figure out an alternative without eg. keeping an explicit method->jmethodID link but that would increase the Method instance footprint and it felt like a more invasive change for what I want to achieve. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16662#discussion_r1404260866 From alanb at openjdk.org Fri Nov 24 11:43:04 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 24 Nov 2023 11:43:04 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v3] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 06:54:27 GMT, Jaikiran Pai wrote: > Right now, the sole usage of the `monitoredHostServiceLoader` instance is within a `synchronized (monitoredHosts) {...}` block within a method. So it wouldn't require this `assert`. Okay, I guess part of me wonders why this field is needed in the first place. Why can't getMonitoredHost just create a ServiceLoader instead instead of trying to share between threads? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1404261936 From duke at openjdk.org Fri Nov 24 11:48:05 2023 From: duke at openjdk.org (suchismith1993) Date: Fri, 24 Nov 2023 11:48:05 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Thu, 23 Nov 2023 18:26:56 GMT, suchismith1993 wrote: > > > > I'm not a big fan of this approach. We accumulate more and more "#ifdef AIX" in shared code because of many recent AIX additions. No other platform has such a large ifdef footprint in shared code. > > > > I argue that all of this should be handled inside os_aix.cpp and not leak out into the external space: > > > > If .a is a valid shared object format on AIX, this should be handled in `os::dll_load()`, and be done for all shared objects. If not, why do we try to load a static archive via dlload in this case but not in other cases? > > > > _If_ this is needed in shared code, the string replacement function should be a generic utility function for all platforms, and it should be tested with a small gtest. A gtest would have likely uncovered the buffer overflow too. > > > > > > > > > So i tried to check how to move the code to os_aix file. A few problems is see : > > > > > > 1. When i have to implemented the logic at dll_load function, i would have to repeat a lot of code after dlopen, i.e i have to call dlopen again for .so files and hence i have to copy the logic again for it. > > > 2. Currently using function before dll_load,in the shared code makes this a bit easier. > > > I have an alternate suggestion . > > > Shall we declare the utlity function as part of os ? and implement it platform wise. > > > > > > Not without any need. If this is an AIX specific issue, it should be handed in os::dll_load on AIX. > > > In that way we just keep the actual implentation and aix and in windows and linux we keep it empty. > > > So that way we can just call the utility function in shared code and it wouldnt affect other platform and will run the usecase for AIX. > > > If that is not acceptable, then is there a better way to avoid repeating the dlopen again in os_aix file ? > > > > > > I don't understand the problem. What is preventing you from using a file local scope utility function inside os::dll_load? > > i would have to repeat the line 1132 and 1139 in os_aix.cpp again , if the condition fails for .so files, because i have to reload it again and check if the .a exists. In the shared code i had repeat less number of lines i believe. Do you suggest moving lines 1132 to 1139 to another function then ? @tstuefe Any suggestion on this ? ------------- PR Comment: https://git.openjdk.org/jdk/pull/16604#issuecomment-1825557400 From duke at openjdk.org Fri Nov 24 11:48:09 2023 From: duke at openjdk.org (suchismith1993) Date: Fri, 24 Nov 2023 11:48:09 GMT Subject: RFR: JDK-8320005 : Native library suffix impact on hotspot code in AIX [v2] In-Reply-To: References: <8-buFPL9W3149qcnluk_XqTQr-cJYqu_XvwU5ovyAIA=.396e5005-f896-48b9-919c-94164229d7bf@github.com> <5RZicS1WS5xiFzcJMhxg_Gjrtdc2I1c4vNMMb37OK-4=.e4ba7692-b18a-4b91-9b35-e444710e38b1@github.com> Message-ID: On Thu, 23 Nov 2023 06:03:15 GMT, Thomas Stuefe wrote: >> suchismith1993 has updated the pull request incrementally with one additional commit since the last revision: >> >> change macro position > > src/hotspot/os/aix/os_aix.cpp line 3065: > >> 3063: //Replaces provided path with alternate path for the given file,if it doesnt exist. >> 3064: //For AIX,this replaces .so with .a. >> 3065: void os::Aix::mapAlternateName(char* buffer, const char *extension) { > > The documentation is wrong: > > // Replaces the specified path with an alternative path for the > given file if the original path doesn't exist > > It does no such thing; it replaces the extension unconditionally. The comment sounds like it does a file system check. > > The whole function is not that well named - "map alternate name" does not really tell me anything, I need to look at the implementation and the caller to understand what it is doing. There is no mapping here, this is just a string utility function. > > The function should not modify the original buffer but instead assemble a copy. That is the conventional way to do these things. You can work with immutable strings as input, e.g. literals, and don't risk buffer overflows. > > All of this should be handled inside os_aix.cpp; see my other comment. This should not live in the external os::aix interface, since it has nothing to do with AIX. But I think all of this should be confined to os_aix.cpp. > > Proposal for a clearer name, comment, and pseudocode > > // Given a filename with an extension, return a new string containing the filename with the new extension. > // New string is allocated in resource area. > static char* replace_extension_in_filename(const char* filename, const char* new_extension) { > - allocate buffer in RA > - assemble new path by contacting old path - old extension + new extension > - return new path > } thank you for the explanation. I am working on it. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16604#discussion_r1404265530 From jpai at openjdk.org Fri Nov 24 11:50:07 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 11:50:07 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v3] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 11:40:40 GMT, Alan Bateman wrote: > > Right now, the sole usage of the `monitoredHostServiceLoader` instance is within a `synchronized (monitoredHosts) {...}` block within a method. So it wouldn't require this `assert`. > > Okay, I guess part of me wonders why this field is needed in the first place. Why can't getMonitoredHost just create a ServiceLoader instead instead of trying to share between threads? It can and that works too. That was one of the alternatives I had initially tried. I explain in this previous comment https://github.com/openjdk/jdk/pull/16805#issuecomment-1825194163 the reason why I thought sharing the ServiceLoader might be better. Do you think I should just use the local ServiceLoader approach instead? ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1404266670 From kevinw at openjdk.org Fri Nov 24 11:50:06 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Fri, 24 Nov 2023 11:50:06 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v3] In-Reply-To: <1IvAf18lR_PiowmOG2gQ7qykjtGvPq16S6mwhjZwhVU=.a32338f2-d923-4049-b187-0b9059a8e7b4@github.com> References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> <1IvAf18lR_PiowmOG2gQ7qykjtGvPq16S6mwhjZwhVU=.a32338f2-d923-4049-b187-0b9059a8e7b4@github.com> Message-ID: On Fri, 24 Nov 2023 11:28:25 GMT, Jaikiran Pai wrote: >> Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? >> >> As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. >> >> The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). >> >> A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. >> >> tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. > > Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision: > > Kevin's inputs - simplify the synchronized block Great, well I like that, obviously. Good luck with testing! ------------- Marked as reviewed by kevinw (Committer). PR Review: https://git.openjdk.org/jdk/pull/16805#pullrequestreview-1747810501 From alanb at openjdk.org Fri Nov 24 11:58:05 2023 From: alanb at openjdk.org (Alan Bateman) Date: Fri, 24 Nov 2023 11:58:05 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v3] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: On Fri, 24 Nov 2023 11:46:20 GMT, Jaikiran Pai wrote: >>> Right now, the sole usage of the `monitoredHostServiceLoader` instance is within a `synchronized (monitoredHosts) {...}` block within a method. So it wouldn't require this `assert`. >> >> Okay, I guess part of me wonders why this field is needed in the first place. Why can't getMonitoredHost just create a ServiceLoader instead instead of trying to share between threads? > >> > Right now, the sole usage of the `monitoredHostServiceLoader` instance is within a `synchronized (monitoredHosts) {...}` block within a method. So it wouldn't require this `assert`. >> >> Okay, I guess part of me wonders why this field is needed in the first place. Why can't getMonitoredHost just create a ServiceLoader instead instead of trying to share between threads? > > > It can and that works too. That was one of the alternatives I had initially tried. I explain in this previous comment https://github.com/openjdk/jdk/pull/16805#issuecomment-1825194163 the reason why I thought sharing the ServiceLoader might be better. Do you think I should just use the local ServiceLoader approach instead? > It can and that works too. That was one of the alternatives I had initially tried. I explain in this previous comment [#16805 (comment)](https://github.com/openjdk/jdk/pull/16805#issuecomment-1825194163) the reason why I thought sharing the ServiceLoader might be better. Do you think I should just use the local ServiceLoader approach instead? I assume almost all usages just fetch the iterator just once, in which case the provider cache doesn't help. So yes, I think I would be tempted to just remove the field and make this a lot simpler. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1404274170 From jbachorik at openjdk.org Fri Nov 24 12:01:27 2023 From: jbachorik at openjdk.org (Jaroslav Bachorik) Date: Fri, 24 Nov 2023 12:01:27 GMT Subject: RFR: 8313816: Accessing jmethodID might lead to spurious crashes [v9] In-Reply-To: References: Message-ID: > Please, review this fix for a corner case handling of `jmethodID` values. > > The issue is related to the interplay between `jmethodID` values and method redefinitions. Each `jmethodID` value is effectively a pointer to a `Method` instance. Once that method gets redefined, the `jmethodID` is updated to point to the last `Method` version. > Unless the method is still on stack/running, in which case the original `jmethodID` will be redirected to the latest `Method` version and at the same time the 'previous' `Method` version will receive a new `jmethodID` pointing to that previous version. > > If we happen to capture stacktrace via `GetStackTrace` or `GetAllStackTraces` JVMTI calls while this previous `Method` version is still on stack we will have the corresponding frame identified by a `jmethodID` pointing to that version. > However, sooner or later the 'previous' class version becomes eligible for cleanup at what time all contained `Method` instances. The cleanup process will not perform the `jmethodID` pointer maintenance and we will end up with pointers to deallocated memory. > This is caused by the fact that the `jmethodID` lifecycle is bound to `ClassLoaderData` instance and all relevant `jmethodID`s will get batch-updated when the class loader is being released and all its classes are getting unloaded. > > This means that we need to make sure that if a `Method` instance is being deallocate the associated `jmethodID` (if any) must not point to the deallocated instance once we are finished. Unfortunately, we can not just update the `jmethodID` values in bulk when purging an old class version - the per `InstanceKlass` jmethodID cache is present only for the main class version and contains `jmethodID` values for both the old and current method versions. > > Therefore we need to perform `jmethodID` lookup when we are about to deallocate a `Method` instance and clean up the pointer only if that `jmethodID` is pointing to the `Method` instance which is being deallocated. > > _(For anyone interested, a much lengthier writeup is available in [my blog](https://jbachorik.github.io/posts/mysterious-jmethodid))_ Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision: Comment adjustments ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16662/files - new: https://git.openjdk.org/jdk/pull/16662/files/46eff8d3..554b3ae0 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16662&range=07-08 Stats: 12 lines in 2 files changed: 7 ins; 4 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/16662.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16662/head:pull/16662 PR: https://git.openjdk.org/jdk/pull/16662 From kevinw at openjdk.org Fri Nov 24 12:16:08 2023 From: kevinw at openjdk.org (Kevin Walls) Date: Fri, 24 Nov 2023 12:16:08 GMT Subject: RFR: 8316649: JMX connection timeout cannot be changed and uses the default of 0 (infinite) In-Reply-To: References: <6R7B_a7VdF5Fg8B38rMhx0Fe1WT53KQw0yt596rzmS4=.9902a48f-f143-4d81-8940-d9c273e94812@github.com> Message-ID: On Thu, 23 Nov 2023 17:21:02 GMT, Daniel Fuchs wrote: >>> This is a stack from a test I was experimenting with, when it did see the timeout: >> >> Ah - that's for testing with a particular test. So the question now is: >> >> Should the RMISocketFactories provided / implemented in the JMX implementation - such as those used by the JMX default agent, also honour this system property? > > (Look for socket factories in the module `jdk.management.agent`) OK yes, we also have: java.rmi/share/classes/javax/rmi/ssl/SslRMIClientSocketFactory.java with its own createSocket(String host, int port) method. This is used if we use JMX over SSL. So SslRMIClientSocketFactory could specifically implement the connect timeout. Next q, should it? 8-) The reported hang and those I have seen in testing have only been in: sun.rmi.transport.tcp.TCPDirectSocketFactory.createSocket calling Socket.init. javax/rmi/ssl/SslRMIClientSocketFactory.java reads some properties named "javax.rmi.ssl.client...." so it would be odd for it to read "sun.rmi.transport.tcp.initialConnectTimeout" I was proposing here. It could implement "javax.rmi.ssl.client.initialConnectTimeout", or we could leave SSL alone for now, possibly handling it in a separate issue if it's wanted. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16771#discussion_r1404290257 From dholmes at openjdk.org Fri Nov 24 12:53:06 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 24 Nov 2023 12:53:06 GMT Subject: RFR: 8320515: assert(monitor->object_peek() != nullptr) failed: Owned monitors should not have a dead object [v5] In-Reply-To: References: <-YLzEt2tPsupH0CLU6278f4yX3si2I60OvHfDitr-tM=.e5d3dad2-b846-4da5-812f-ea2600cd2780@github.com> Message-ID: <29uxp2DVbW7p6RLM3yZUXnhS8Bc47Il0CVLJAKY21jA=.8f7b43cc-28ab-454d-9d69-ddaf75296943@github.com> On Fri, 24 Nov 2023 08:01:15 GMT, Stefan Karlsson wrote: >> Thanks for that. Looks like JMM thread dump is different to VM Thread dump. Okay we definitely need RFEs to look into how to handle this. > > Will you create the RFE? I'm not as convinced that this is something that needs to be fixed, so it would be better if you create the RFE with the proper motivation. Yes I will create it. Thanks ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16783#discussion_r1404321515 From jpai at openjdk.org Fri Nov 24 13:00:33 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 13:00:33 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v4] In-Reply-To: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: > Can I please get a review of this change which proposes to fix the issue noted in https://bugs.openjdk.org/browse/JDK-8320687? > > As noted in the issue, the `sun.jvmstat.monitor.MonitoredHost.getMonitoredHost()` uses a shared instance of `java.util.ServiceLoader` to load `MonitoredHostService` services. The `ServiceLoader` class javadoc explicitly notes that it isn't thread safe. The issue at hand is caused to due using an instance of `ServiceLoader` concurrently by multiple threads. > > The fix proposes to guard the usage of the shared `ServiceLoader` instance through the `monitoredHosts` object monitor. We already use that monitor when dealing with the internal cache which is populated after loading the relevant `MonitoredHostService`(s). > > A new jtreg test has been introduced which always reproduces the issue without the source changes and passes with this fix. > > tier1, tier2, tier3 and svc_tools tests have been run with this change and all passed. Jaikiran Pai has updated the pull request incrementally with one additional commit since the last revision: Alan's suggestion - don't share ServiceLoader ------------- Changes: - all: https://git.openjdk.org/jdk/pull/16805/files - new: https://git.openjdk.org/jdk/pull/16805/files/669e5e5e..6e7bf7fb Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=16805&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16805&range=02-03 Stats: 45 lines in 1 file changed: 16 ins; 20 del; 9 mod Patch: https://git.openjdk.org/jdk/pull/16805.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/16805/head:pull/16805 PR: https://git.openjdk.org/jdk/pull/16805 From jpai at openjdk.org Fri Nov 24 13:00:35 2023 From: jpai at openjdk.org (Jaikiran Pai) Date: Fri, 24 Nov 2023 13:00:35 GMT Subject: RFR: 8320687: sun.jvmstat.monitor.MonitoredHost.getMonitoredHost() throws unexpected exceptions when invoked concurrently [v4] In-Reply-To: References: <3BzevX5rfQoTyzsAm_apM0oYy0Vd4q7D5-HOaGHEwso=.68db1ec1-798a-4c81-9ebe-7cd096e0f0aa@github.com> Message-ID: <5sVd7u4r9lu6DQ7IhbSKoYZQvdNw7nhbQgcL5rCRLF8=.ec1126e2-70f8-4c21-bed7-17ba5875b16e@github.com> On Fri, 24 Nov 2023 11:55:15 GMT, Alan Bateman wrote: >>> > Right now, the sole usage of the `monitoredHostServiceLoader` instance is within a `synchronized (monitoredHosts) {...}` block within a method. So it wouldn't require this `assert`. >>> >>> Okay, I guess part of me wonders why this field is needed in the first place. Why can't getMonitoredHost just create a ServiceLoader instead instead of trying to share between threads? >> >> >> It can and that works too. That was one of the alternatives I had initially tried. I explain in this previous comment https://github.com/openjdk/jdk/pull/16805#issuecomment-1825194163 the reason why I thought sharing the ServiceLoader might be better. Do you think I should just use the local ServiceLoader approach instead? > >> It can and that works too. That was one of the alternatives I had initially tried. I explain in this previous comment [#16805 (comment)](https://github.com/openjdk/jdk/pull/16805#issuecomment-1825194163) the reason why I thought sharing the ServiceLoader might be better. Do you think I should just use the local ServiceLoader approach instead? > > I assume almost all usages just fetch the iterator just once, in which case the provider cache doesn't help. So yes, I think I would be tempted to just remove the field and make this a lot simpler. Done. I've updated the PR to just use a local ServiceLoader instead of a shared one. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/16805#discussion_r1404327944 From dholmes at openjdk.org Fri Nov 24 13:01:12 2023 From: dholmes at openjdk.org (David Holmes) Date: Fri, 24 Nov 2023 13:01:12 GMT Subject: RFR: 8320532: Remove Thread/ThreadGroup suspend/resume In-Reply-To: References: Message-ID: On Thu, 23 Nov 2023 09:18:44 GMT, Alan Bateman wrote: > The deadlock prone Thread/ThreadGroup suspend/resume were deprecated since JDK 1.2, deprecated for removal in Java 14, and re-specified/degraded to throw UnsupportedOperationException unconditionally in Java 19/20. Early in Java 23 seems a fine time to finally remove these methods. > > Corpus analysis of 176 million classes in 485k artifacts found no remaining usages of ThreadGroup.suspend/resume beyond the artifacts that include a copy of java.lang.ThreadGroup (!). It found 87