From mdoerr at openjdk.org Wed Mar 1 05:31:34 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 05:31:34 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v4] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: HFA: Add support for nested structures. See JDK-8300294. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/a4d844f7..b461d80c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=02-03 Stats: 37 lines in 2 files changed: 24 ins; 0 del; 13 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 1 06:12:54 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 06:12:54 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v5] In-Reply-To: References: Message-ID: <3kCGyBFd4OIVzBcT_6KQIvJIFCGVRTRq2dy3qKP0Piw=.ef00d405-43e7-402a-a1d3-1ebaed3b7830@github.com> > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Minor cleanup. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/b461d80c..75b5c78f Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=04 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=03-04 Stats: 6 lines in 3 files changed: 0 ins; 2 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 1 06:12:59 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 06:12:59 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Tue, 28 Feb 2023 16:54:50 GMT, Jorn Vernee wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove size restriction for structs. Add TODO for Big Endian. > > src/hotspot/cpu/ppc/downcallLinker_ppc.cpp line 343: > >> 341: >> 342: __ flush(); >> 343: // Disassembler::decode((u_char*)start, (u_char*)__ pc(), tty); > > Leftover commented code? > > (note that the stub can also be disassembled with `-Xlog:foreign+downcall=trace` now) Removed. Thanks for the hint! > src/hotspot/cpu/ppc/upcallLinker_ppc.cpp line 356: > >> 354: } >> 355: #endif >> 356: //blob->print_on(tty); > > Leftover commented code? Removed. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 1 06:13:03 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 06:13:03 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v5] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Wed, 22 Feb 2023 20:25:07 GMT, Jorn Vernee wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Minor cleanup. > > src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/CallArranger.java line 81: > >> 79: new VMStorage[] { f1, f2, f3, f4, f5, f6, f7, f8 }, // FP output >> 80: new VMStorage[] { r0, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12 }, // volatile GP >> 81: new VMStorage[] { f0, f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12, f13 }, // volatile FP > > Note that argument registers are assumed volatile, so they don't have to be duplicated here. Removed. > src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/linux/LinuxPPC64CallArranger.java line 33: > >> 31: * PPC64 CallArranger specialized for Linux ABI. >> 32: */ >> 33: public class LinuxPPC64CallArranger extends CallArranger { > > I don't really see the point in having a separate subclass with `CallArranger` being abstract, unless you are planning to add other implementations later? > > (edit: see also later comment in CallArranger https://github.com/openjdk/jdk/pull/12708#discussion_r1120753657) AIX support will need to get implemented, yet. I guess @backwaterred will work on it. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 1 06:20:11 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 06:20:11 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Tue, 28 Feb 2023 19:45:28 GMT, Jorn Vernee wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove size restriction for structs. Add TODO for Big Endian. > > src/hotspot/cpu/ppc/downcallLinker_ppc.cpp line 133: > >> 131: Register callerSP = R2, // C/C++ uses R2 as TOC, but we can reuse it here >> 132: tmp = R11_scratch1, // same as shuffle_reg >> 133: call_target_address = R12_scratch2; // same as _abi._scratch2 (ABIv2 requires this reg!) > > Do I understand correctly that the ABI requires the register to be used for the call to be `R12`? How does that make a difference? I guess in some cases the callee might want to know the address through which it is called? (so it looks at `R12`) ABI v2 requires R12 to point to the function entry point. It is used to access constants relative to it. > src/hotspot/cpu/ppc/downcallLinker_ppc.cpp line 154: > >> 152: // (abi_reg_args is abi_minframe plus space for 8 argument register spill slots) >> 153: assert(_abi._shadow_space_bytes == frame::abi_minframe_size, "expected space according to ABI"); >> 154: int allocated_frame_size = frame::abi_minframe_size + MAX2(_input_registers.length(), 8) * BytesPerWord; > > This is hard-coding an assumption about the ABI that's being called. Ok for now. > > If it needs to be addressed in the future, it could be done by adding another field to `ABIDescriptor` like `min_stack_arg_bytes`, or something like that (which is set to zero for other ABIs). It seems to be different from `shadow_space` since it's also used by the caller to put stack arguments. Yeah, I think this should be done on demand. > src/hotspot/cpu/ppc/foreignGlobals_ppc.cpp line 229: > >> 227: >> 228: void ArgumentShuffle::pd_generate(MacroAssembler* masm, VMStorage tmp, int in_stk_bias, int out_stk_bias, const StubLocations& locs) const { >> 229: Register callerSP = as_Register(tmp); // preset > > It looks like `tmp` is being used to hold the caller's SP. I'm guessing this can not be computed the same way as we do on x86 and aarch64? (based on `RBP`, `RFP_BIAS`) > > If you want, you could add another register argument to `pd_generate` that is just invalid/unused on other platforms. That way you could use `tmp` for the shuffling instead of having to go through the stack. (looks like `R0` is already used in some cases as a temp register) There's no BP register. It could get loaded from the back chain, but would still need a register. R0 is always available as scratch reg, so there's no need to pass it. (Note that I'm not going through stack because of the lack of registers.) Yeah, maybe passing callerSP separately would make it better readable on PPC64. Not sure if it's worth changing. > src/hotspot/cpu/ppc/upcallLinker_ppc.cpp line 137: > >> 135: ArgumentShuffle arg_shuffle(in_sig_bt, total_in_args, out_sig_bt, total_out_args, &in_conv, &out_conv, shuffle_reg); >> 136: // The Java call uses the JIT ABI, but we also call C. >> 137: int out_arg_area = MAX2(frame::jit_out_preserve_size + arg_shuffle.out_arg_bytes(), (int)frame::abi_reg_args_size); > > We need `frame::abi_reg_args_size` since we call `on_entry`/`on_exit` which require the stack space I guess? Correct. C functions are allowed to write into the space which is part of the caller's frame. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 1 06:28:12 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 06:28:12 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Tue, 28 Feb 2023 20:00:15 GMT, Jorn Vernee wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove size restriction for structs. Add TODO for Big Endian. > > src/hotspot/cpu/ppc/upcallLinker_ppc.cpp line 240: > >> 238: __ ld(call_target_address, in_bytes(Method::from_compiled_offset()), R19_method); >> 239: __ mtctr(call_target_address); >> 240: __ bctrl(); > > Ok, I see. I guess there is some special purpose register called `CTR` which we are moving to for `bctrl` here. Does ABIv2 require that move to always come from `R12`? (from the comment in downcallLinker). > > (I'm trying to understand the requirements for possibly tweaking shared code). There's no instruction which can use a GP reg as branch target directly. That's why we use CTR. In this case, using R12 is not required since we call Java and our PPC64 VM code does not rely on it. If we were calling C, using R12 would be required by ABI v2. > src/hotspot/cpu/ppc/upcallLinker_ppc.cpp line 347: > >> 345: FunctionDescriptor* fd = (FunctionDescriptor*)fd_addr; >> 346: fd->set_entry(fd_addr + sizeof(FunctionDescriptor)); >> 347: #endif > > Had to do a double take. Looks like we're not the only one who are using the name `FunctionDescriptor` :) Yeah, ABI v1 (Big Endian) treats function pointers as pointers to a structure called `FunctionDescriptor` which contains the entry point plus additional information like a pointer to a constant table. > src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/CallArranger.java line 286: > >> 284: // "no partial DW rule": Mark first stack slot to get filled. >> 285: // Note: Can only happen with forArguments = true. >> 286: VMStorage overlappingReg = null; > > `overlappingReg` is initialized along all branches, so it's not needed to assign `null` here (and then javac will check it is actually assigned before use) Removed. > src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/TypeClass.java line 66: > >> 64: } >> 65: >> 66: static boolean isHomogeneousFloatAggregate(MemoryLayout type, boolean useABIv2) { > > Note that we had to make some changes to this routine on AArch64, since it didn't properly account for nested structs/unions and arrays. See: https://github.com/openjdk/panama-foreign/pull/780 > > Just as a heads up, in case PPC needs changes too. Thanks for the hint! I just added the code since it is known to be needed, small and doesn't break any of the existing tests. I'll need to test and possibly debug the various cases, yet. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 1 06:41:09 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 06:41:09 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Fri, 24 Feb 2023 07:17:30 GMT, Jorn Vernee wrote: >> Some more remarks about other issues: >> - Uploaded my simple reproducer to [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> - Using oversized load / stores is problematic. Don't forget that OpenJDK still supports Big Endian platforms (AIX, s390x). >> - The result of `NativeCallingConvention::calling_convention` is interpreted as size, but it returns the max offset. That's off by one slot. Should I file a bug for that? (PPC64 is not affected because it doesn't use the result.) >> - Since the membar on the return path was mentioned: I think it would be good to enable UseSystemMemoryBarrier by default on operating systems which support it. Maybe we should discuss this with @robehn. > >> * Uploaded my simple reproducer to [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > Thanks! > >> * Using oversized load / stores is problematic. Don't forget that OpenJDK still supports Big Endian platforms (AIX, s390x). > > You're right. I realized that it's also problematic for heap segments, for which we can't do oversized accesses. I am working on another solution that splits up the loads/stores into power-of-two sized chunks: https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:OOB That patch is just a POC at this point though. Also, I don't think it works for BE at the moment (need to flip the offset for BE, I think. Just like we do in Unsafe). > >> * The result of `NativeCallingConvention::calling_convention` is interpreted as size, but it returns the max offset. That's off by one slot. Should I file a bug for that? (PPC64 is not affected because it doesn't use the result.) > > I'm not sure there's an issue there. Note that the 'max offset' is computed as `reg.offset() + reg.stack_size()`, so that should get us the size we need to allocate for the stack arguments. (e.g. 2 ints being passed at offset 0 and 4, would make max offset 4 + 4 = 8, which gives the size needed for the 2 ints). Computing the max offset instead of just summing the sizes of the stack arguments is needed since stack arguments can be sparsely placed in some cases on Mac/AArch64. > >> * Since the membar on the return path was mentioned: I think it would be good to enable UseSystemMemoryBarrier by default on operating systems which support it. Maybe we should discuss this with @robehn. > > ~I don't think we've done that much testing with UseSystemMemoryBarrier since it was added~. I'm a bit nervous about turning it on by default since it's currently also used for JNI. Let's see what Robbin thinks. @JornVernee: Thanks a lot for your detailed review! I have quite a few TODOs which include: - Include my tests for the HFA corner cases. - Try to improve handling of the overlapping registers as you suggested. - Check nesting of HFA. There will surely be more when looking into Big Endian support after merging with your recent work on https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:OOB We should get rid of oversized accesses on PPC64, too. Thanks for sharing your plans to intrisify `linkToNative` in C2 later. I guess we should do more preparation work on all platforms when that gets addressed. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From rkennke at openjdk.org Wed Mar 1 09:48:20 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Wed, 1 Mar 2023 09:48:20 GMT Subject: RFR: Merge openjdk/jdk:master In-Reply-To: References: Message-ID: On Tue, 28 Feb 2023 23:51:13 GMT, William Kemper wrote: > Merge tag jdk-21+11 I would suggest that you enable GHA in your fork, and push it as soon as that is green. It's not very useful to review merges. Thanks! Roman ------------- PR: https://git.openjdk.org/shenandoah/pull/221 From jvernee at openjdk.org Wed Mar 1 13:06:09 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Wed, 1 Mar 2023 13:06:09 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Wed, 1 Mar 2023 06:37:45 GMT, Martin Doerr wrote: > There will surely be more when looking into Big Endian support after merging with your recent work on https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:OOB I will try to move this into the mainline ahead of the JEP PR (process allowing). As well as another testing coverage patch for nested structs/unions: https://github.com/openjdk/panama-foreign/pull/780. > Thanks for sharing your plans to intrisify linkToNative in C2 later. I guess we should do more preparation work on all platforms when that gets addressed. WRT this: we had intrinsification in the past as well (https://github.com/openjdk/panama-foreign/pull/609), but it was removed since we also removed 'trivial' calls at some point (and then the intrinsics were not used). Now that trivial calls are back on the menu, it might be interesting to look at re-adding intrinsics as well. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 1 13:06:13 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 1 Mar 2023 13:06:13 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Tue, 28 Feb 2023 20:51:50 GMT, Jorn Vernee wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove size restriction for structs. Add TODO for Big Endian. > > src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/CallArranger.java line 68: > >> 66: public abstract class CallArranger { >> 67: // Linux PPC64 Little Endian uses ABI v2. >> 68: private static final boolean useABIv2 = ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN; > > Now that I'm here. This could be a potentially interesting case for having 2 subclasses of CallArranger: one for `useABIv2 == true` and one for `false`. Yeah, let's wait until we know what changes we need for AIX (and Big Endian linux). > src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/CallArranger.java line 293: > >> 291: } else { >> 292: overlappingReg = new VMStorage(StorageType.STACK_AND_FLOAT, >> 293: (short) STACK_SLOT_SIZE, (int) stackOffset - 4); > > I think you could remove the mixed VMStorage types here relatively easily by returning a `VMStorage[][]`, where each element is a single element array, but then for the `needOverlapping` case add another element to the array for the extra store (instead of replacing the existing one). > > Then when unboxing a `STRUCT_HFA`, `dup` the result of the `bufferLoad` and then do 2 `vmStore`s (one for each element). > > For boxing, you could just ignore the extra storage, and just `vmLoad` the first one (or, whichever one you like :)) Thanks! I need to find extra time for this. Sounds like a good idea and I may be able to get rid of some nasty code. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From wkemper at openjdk.org Wed Mar 1 18:38:02 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 1 Mar 2023 18:38:02 GMT Subject: RFR: Merge openjdk/jdk:master In-Reply-To: References: Message-ID: On Tue, 28 Feb 2023 23:51:13 GMT, William Kemper wrote: > Merge tag jdk-21+11 I do have GHA enabled in my fork. There is a test failure, tracked here: https://bugs.openjdk.org/browse/JDK-8303105. There are also cross-compilation failures because of: Reading package lists... E: The repository 'https://packages.microsoft.com/ubuntu/22.04/prod jammy Release' is no longer signed. Error: Process completed with exit code 100. I'm really not sure how to address that. I don't think I have permissions to just push to this repo, but I am able to `/integrate` merge commits without review approval. ------------- PR: https://git.openjdk.org/shenandoah/pull/221 From wkemper at openjdk.org Wed Mar 1 18:42:25 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 1 Mar 2023 18:42:25 GMT Subject: RFR: Merge openjdk/jdk:master [v2] In-Reply-To: References: Message-ID: > Merge tag jdk-21+11 William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 229 commits: - Merge tag 'jdk-21+11' into merge-jdk-21-11 Added tag jdk-21+11 for changeset fcaf8714 - Merge openjdk/jdk:master - Remove unused visualizer option Reviewed-by: kdnilsen - Merge openjdk/jdk:master Reviewed-by: rkennke - Merge openjdk/jdk:master - 8299703: GenShen: improvements in card scanning Reviewed-by: kdnilsen, wkemper - Update vestigial comment Reviewed-by: ysr, kdnilsen - Bootstrap old gc should honor aging cycle Reviewed-by: wkemper - Following abbreviated cycle, only increment region age if aging cycle Reviewed-by: wkemper - Fullgc should honor aging cycle Reviewed-by: wkemper - ... and 219 more: https://git.openjdk.org/shenandoah/compare/fcaf8714...9be58b2d ------------- Changes: https://git.openjdk.org/shenandoah/pull/221/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=221&range=01 Stats: 16622 lines in 155 files changed: 15274 ins; 507 del; 841 mod Patch: https://git.openjdk.org/shenandoah/pull/221.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/221/head:pull/221 PR: https://git.openjdk.org/shenandoah/pull/221 From wkemper at openjdk.org Wed Mar 1 18:42:27 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 1 Mar 2023 18:42:27 GMT Subject: Integrated: Merge openjdk/jdk:master In-Reply-To: References: Message-ID: On Tue, 28 Feb 2023 23:51:13 GMT, William Kemper wrote: > Merge tag jdk-21+11 This pull request has now been integrated. Changeset: ebc1fd1d Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/ebc1fd1d19f9cd5d3f4f14f7221fb3a4d739688b Stats: 13641 lines in 534 files changed: 8720 ins; 2537 del; 2384 mod Merge openjdk/jdk:master ------------- PR: https://git.openjdk.org/shenandoah/pull/221 From mdoerr at openjdk.org Thu Mar 2 06:43:15 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 2 Mar 2023 06:43:15 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v6] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Add test for HFA corner cases. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/75b5c78f..c96e1120 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=05 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=04-05 Stats: 256 lines in 2 files changed: 256 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From duke at openjdk.org Thu Mar 2 10:14:00 2023 From: duke at openjdk.org (Amit Kumar) Date: Thu, 2 Mar 2023 10:14:00 GMT Subject: RFR: 8303147: [s390x] fast & slow debug builds are broken Message-ID: <_ZdU0STu6HC-2e2UuIXJPe7RslDoKVDPLYXSGJFAwkA=.7639982c-26d7-4a93-8e6f-2d0171b71477@github.com> This PR fixes broken fast debug and slow debug build for s390x-arch. tier1 test are completed and results are not affect after this patch. ------------- Commit messages: - build fix for s390x Changes: https://git.openjdk.org/jdk/pull/12825/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12825&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303147 Stats: 53 lines in 12 files changed: 35 ins; 13 del; 5 mod Patch: https://git.openjdk.org/jdk/pull/12825.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12825/head:pull/12825 PR: https://git.openjdk.org/jdk/pull/12825 From iwalulya at openjdk.org Thu Mar 2 13:56:04 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Thu, 2 Mar 2023 13:56:04 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects Message-ID: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Hi All, Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. Testing: Tier 1-3 ------------- Commit messages: - restructure - add test - breakout large functioons - working prototype Changes: https://git.openjdk.org/jdk/pull/12830/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12830&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8191565 Stats: 333 lines in 13 files changed: 318 ins; 9 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/12830.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12830/head:pull/12830 PR: https://git.openjdk.org/jdk/pull/12830 From mdoerr at openjdk.org Fri Mar 3 04:04:06 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 3 Mar 2023 04:04:06 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v7] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: - Merge branch 'master' into PPC64_Panama - Add test for HFA corner cases. - Minor cleanup. - HFA: Add support for nested structures. See JDK-8300294. - Remove size restriction for structs. Add TODO for Big Endian. - Clean fix for NativeMemorySegmentImpl issue with byteSize 0. - Initial Panama implementation. ------------- Changes: https://git.openjdk.org/jdk/pull/12708/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=06 Stats: 2247 lines in 59 files changed: 2139 ins; 1 del; 107 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Fri Mar 3 04:43:40 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 3 Mar 2023 04:43:40 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v8] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Fix merge bug. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/d71f0f0a..f23edd8e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=07 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=06-07 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Fri Mar 3 10:55:57 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 3 Mar 2023 10:55:57 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v9] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Handle HFA corner cases with overlapping registers in Java. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/f23edd8e..98e242c2 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=08 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=07-08 Stats: 53 lines in 4 files changed: 16 ins; 20 del; 17 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Fri Mar 3 11:02:18 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Fri, 3 Mar 2023 11:02:18 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Wed, 1 Mar 2023 06:26:20 GMT, Martin Doerr wrote: >> src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/CallArranger.java line 293: >> >>> 291: } else { >>> 292: overlappingReg = new VMStorage(StorageType.STACK_AND_FLOAT, >>> 293: (short) STACK_SLOT_SIZE, (int) stackOffset - 4); >> >> I think you could remove the mixed VMStorage types here relatively easily by returning a `VMStorage[][]`, where each element is a single element array, but then for the `needOverlapping` case add another element to the array for the extra store (instead of replacing the existing one). >> >> Then when unboxing a `STRUCT_HFA`, `dup` the result of the `bufferLoad` and then do 2 `vmStore`s (one for each element). >> >> For boxing, you could just ignore the extra storage, and just `vmLoad` the first one (or, whichever one you like :)) > > Thanks! I need to find extra time for this. Sounds like a good idea and I may be able to get rid of some nasty code. Done by https://github.com/openjdk/jdk/pull/12708/commits/98e242c24c07ea977b7709b9f8d0c10ce87e84c0 (using a record instead of a `VMStorage[][]` because I think this is better readable). Note that it's a bit more complicated. I couldn't use your `dup` trick, because I need to put the value into a GP reg and one half of it to a FP reg. The Panama code doesn't support that (IllegalArgumentException: Invalid operand type: interface java.lang.foreign.MemorySegment. float expected). ------------- PR: https://git.openjdk.org/jdk/pull/12708 From tschatzl at openjdk.org Fri Mar 3 14:58:12 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Fri, 3 Mar 2023 14:58:12 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects In-Reply-To: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: On Thu, 2 Mar 2023 13:48:10 GMT, Ivan Walulya wrote: > Hi All, > > Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. > > Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. > > We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. > > Testing: Tier 1-3 Changes requested by tschatzl (Reviewer). src/hotspot/share/gc/g1/g1FullCollector.hpp line 87: > 85: G1FullGCCompactionPoint _serial_compaction_point; > 86: G1FullGCCompactionPoint _humongous_compaction_point; > 87: GrowableArray* _humongous_compaction_regions; This could be a `GrowableArrayCHeap _humongous_compaction_region` to avoid the necessity for explicit new/delete. Afaics there is no reassignment of that variable ever. src/hotspot/share/gc/g1/g1FullCollector.hpp line 150: > 148: > 149: inline void add_humongous_region(HeapRegion* hr) { _humongous_compaction_regions->append(hr); } > 150: GrowableArray* humongous_compaction_regions() { return _humongous_compaction_regions; } Please move to the `.inline.hpp` file. src/hotspot/share/gc/g1/g1FullGCCompactTask.cpp line 59: > 57: > 58: size_t size = obj->size(); > 59: // copy object and reinit its mark Suggestion: // Copy object and reinit its mark. src/hotspot/share/gc/g1/g1FullGCCompactTask.cpp line 117: > 115: } > 116: > 117: void G1FullGCCompactTask::reset_humongous_metadata(HeapRegion* start_hr, uint num_regions, size_t word_size) { Maybe this could somehow be refactored with `G1CollectedHeap::humongous_obj_allocate_initialize_regions`; probably not easily. src/hotspot/share/gc/g1/g1FullGCCompactTask.cpp line 174: > 172: size_t word_size = obj->size(); > 173: > 174: uint num_regions = (uint) G1CollectedHeap::humongous_obj_size_in_regions(word_size); Suggestion: uint num_regions = (uint)G1CollectedHeap::humongous_obj_size_in_regions(word_size); src/hotspot/share/gc/g1/g1FullGCCompactionPoint.cpp line 153: > 151: oop obj = cast_to_oop(hr->bottom()); > 152: size_t obj_size = obj->size(); > 153: int num_regions = (int) G1CollectedHeap::humongous_obj_size_in_regions(obj_size); Please use signed int (or why not `size_t`?) src/hotspot/share/gc/g1/g1FullGCCompactionPoint.cpp line 159: > 157: } > 158: > 159: // Find contiguous compaction target regions for the humongous object Suggestion: // Find contiguous compaction target regions for the humongous object. src/hotspot/share/gc/g1/g1FullGCCompactionPoint.cpp line 160: > 158: > 159: // Find contiguous compaction target regions for the humongous object > 160: Pair range = find_contiguous_before(hr, num_regions); (I'm suprised that MSVC does not complain here passing an int to the uint parameter) src/hotspot/share/gc/g1/g1FullGCCompactionPoint.cpp line 165: > 163: > 164: if (range_begin == range_end) { > 165: // No contiguous compaction target regions found, so the object cannot be moved Suggestion: // No contiguous compaction target regions found, so the object cannot be moved. src/hotspot/share/gc/g1/g1FullGCCompactionPoint.cpp line 172: > 170: _collector->marker(0)->preserved_stack()->push_if_necessary(obj, obj->mark()); > 171: > 172: HeapRegion* destn_hr = _compaction_regions->at(range_begin); I would prefer `dest_hr` instead of `destn_hr` as abbreviation for "destination". It seems unusual. src/hotspot/share/gc/g1/g1FullGCCompactionPoint.cpp line 176: > 174: assert(obj->is_forwarded(), "Object must be forwarded!"); > 175: > 176: // Add the humongous object regions to the compaction point Suggestion: // Add the humongous object regions to the compaction point. src/hotspot/share/gc/g1/g1FullGCCompactionPoint.cpp line 205: > 203: // Check if the current region and the previous region are contiguous. > 204: bool regions_are_contiguous = (_compaction_regions->at(range_end)->hrm_index() - _compaction_regions->at(range_end - 1)->hrm_index()) == 1; > 205: contiguous_region_count = regions_are_contiguous ? contiguous_region_count + 1 : 1; Suggestion: contiguous_region_count = regions_are_contiguous ? contiguous_region_count + 1 : 1; src/hotspot/share/gc/g1/heapRegion.inline.hpp line 192: > 190: } > 191: > 192: inline void HeapRegion::reset_compacted_humongous_after_full_gc(HeapWord* new_top) { This method could probably just call `reset_compacted_after_full_gc` as they are identical. (I am not suggestion to remove this method, although I'm not completely sure it's useful). src/hotspot/share/utilities/growableArray.hpp line 261: > 259: > 260: // Remove all elements in the range [start - end). The order is preserved. > 261: void erase(int start, int end) { I'd probably name this new method `remove_range` instead of using something completely unrelated. ------------- PR: https://git.openjdk.org/jdk/pull/12830 From wkemper at openjdk.org Fri Mar 3 16:37:49 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 3 Mar 2023 16:37:49 GMT Subject: RFR: Performance improvements for non-generational modes Message-ID: There are a basket of changes here: * Make heuristics a bit less aggressive: * Revert threshold triggers to use `available` rather than adjusted head room. * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically * Skip over more generational mode constraints on the allocation path * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. * More detailed debugging messages under the `gc+free` tags. ------------- Commit messages: - Backout some of the more excessive instrumentation - Merge branch 'master' into regression-hunt - Merge branch 'shenandoah-master' into regression-hunt - More instrumentation, optimize path for non-generational modes - More instrumentation for free set and collection set - Use global generation for usage account in non-generational modes - Assertions that generational constraints aren't being misapplied - Only check if region is in generation when in generational mode - Use available, not headroom for threshold triggers - Set region logging to trace level - ... and 2 more: https://git.openjdk.org/shenandoah/compare/ebc1fd1d...58bd7b36 Changes: https://git.openjdk.org/shenandoah/pull/220/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=220&range=00 Stats: 216 lines in 12 files changed: 74 ins; 69 del; 73 mod Patch: https://git.openjdk.org/shenandoah/pull/220.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/220/head:pull/220 PR: https://git.openjdk.org/shenandoah/pull/220 From wkemper at openjdk.org Fri Mar 3 16:37:50 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 3 Mar 2023 16:37:50 GMT Subject: RFR: Performance improvements for non-generational modes In-Reply-To: References: Message-ID: On Sat, 18 Feb 2023 01:56:10 GMT, William Kemper wrote: > There are a basket of changes here: > * Make heuristics a bit less aggressive: > * Revert threshold triggers to use `available` rather than adjusted head room. > * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically > * Skip over more generational mode constraints on the allocation path > * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. > * More detailed debugging messages under the `gc+free` tags. Converted to draft form pending the results of performance testing. I also intend to factor out new heuristic classes for generational mode, and also untangle the allocation paths for generational mode. ------------- PR: https://git.openjdk.org/shenandoah/pull/220 From kdnilsen at openjdk.org Fri Mar 3 18:02:14 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 3 Mar 2023 18:02:14 GMT Subject: RFR: Performance improvements for non-generational modes In-Reply-To: References: Message-ID: On Sat, 18 Feb 2023 01:56:10 GMT, William Kemper wrote: > There are a basket of changes here: > * Make heuristics a bit less aggressive: > * Revert threshold triggers to use `available` rather than adjusted head room. > * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically > * Skip over more generational mode constraints on the allocation path > * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. > * More detailed debugging messages under the `gc+free` tags. I'll need to be careful not to overwrite these improvements with my next merge. Some of the same code is touched. Thanks. src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 314: > 312: range(1.0,100.0) \ > 313: \ > 314: product(uintx, ShenandoahMaxEvacLABRatio, 0, EXPERIMENTAL, \ Surprised this would have an effect on single-generation performance. Have we measured the costs/benefits of setting this default differently? If 0 works best, maybe we should remove this entirely? ------------- Marked as reviewed by kdnilsen (Committer). PR: https://git.openjdk.org/shenandoah/pull/220 From wkemper at openjdk.org Fri Mar 3 19:11:59 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 3 Mar 2023 19:11:59 GMT Subject: RFR: Performance improvements for non-generational modes In-Reply-To: References: Message-ID: <7fZ-6lUINo4FqBHyYfGOKey2EBLcaXGe089_sW8dBtA=.d457f943-714f-4e2b-92c6-ab5d5a5ee2c3@github.com> On Fri, 3 Mar 2023 17:58:09 GMT, Kelvin Nilsen wrote: >> There are a basket of changes here: >> * Make heuristics a bit less aggressive: >> * Revert threshold triggers to use `available` rather than adjusted head room. >> * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically >> * Skip over more generational mode constraints on the allocation path >> * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. >> * More detailed debugging messages under the `gc+free` tags. > > src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 314: > >> 312: range(1.0,100.0) \ >> 313: \ >> 314: product(uintx, ShenandoahMaxEvacLABRatio, 0, EXPERIMENTAL, \ > > Surprised this would have an effect on single-generation performance. Have we measured the costs/benefits of setting this default differently? If 0 works best, maybe we should remove this entirely? Yes, it was counter intuitive. This setting causes small chunks of memory to remain free in regions that have otherwise been consumed by gc lab allocations. When the evacuation reserve is exhausted, the collector will steal regions from the mutator. Because of the free memory in these regions, they are retained in the mutator's view of the freeset after the collection is complete. This in turn increases fragmentation in the mutator's view and reduces the amount of available memory to the mutators, causing the triggers to run more often. I only tested the default setting (16) and turning it off completely. I will test additional values. ------------- PR: https://git.openjdk.org/shenandoah/pull/220 From iwalulya at openjdk.org Mon Mar 6 10:22:27 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Mon, 6 Mar 2023 10:22:27 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects [v2] In-Reply-To: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: > Hi All, > > Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. > > Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. > > We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. > > Testing: Tier 1-3 Ivan Walulya has updated the pull request incrementally with two additional commits since the last revision: - Refactor resetting humongous metadata - Thomas review ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12830/files - new: https://git.openjdk.org/jdk/pull/12830/files/f4951d37..6f8dd514 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12830&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12830&range=00-01 Stats: 219 lines in 12 files changed: 71 ins; 103 del; 45 mod Patch: https://git.openjdk.org/jdk/pull/12830.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12830/head:pull/12830 PR: https://git.openjdk.org/jdk/pull/12830 From jvernee at openjdk.org Mon Mar 6 16:42:10 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Mon, 6 Mar 2023 16:42:10 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. ---------------------------------------------------------------------- On Wed, 1 Mar 2023 06:37:45 GMT, Martin Doerr wrote: >>> * Uploaded my simple reproducer to [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> Thanks! >> >>> * Using oversized load / stores is problematic. Don't forget that OpenJDK still supports Big Endian platforms (AIX, s390x). >> >> You're right. I realized that it's also problematic for heap segments, for which we can't do oversized accesses. I am working on another solution that splits up the loads/stores into power-of-two sized chunks: https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:OOB That patch is just a POC at this point though. Also, I don't think it works for BE at the moment (need to flip the offset for BE, I think. Just like we do in Unsafe). >> >>> * The result of `NativeCallingConvention::calling_convention` is interpreted as size, but it returns the max offset. That's off by one slot. Should I file a bug for that? (PPC64 is not affected because it doesn't use the result.) >> >> I'm not sure there's an issue there. Note that the 'max offset' is computed as `reg.offset() + reg.stack_size()`, so that should get us the size we need to allocate for the stack arguments. (e.g. 2 ints being passed at offset 0 and 4, would make max offset 4 + 4 = 8, which gives the size needed for the 2 ints). Computing the max offset instead of just summing the sizes of the stack arguments is needed since stack arguments can be sparsely placed in some cases on Mac/AArch64. >> >>> * Since the membar on the return path was mentioned: I think it would be good to enable UseSystemMemoryBarrier by default on operating systems which support it. Maybe we should discuss this with @robehn. >> >> ~I don't think we've done that much testing with UseSystemMemoryBarrier since it was added~. I'm a bit nervous about turning it on by default since it's currently also used for JNI. Let's see what Robbin thinks. > > @JornVernee: Thanks a lot for your detailed review! I have quite a few TODOs which include: > - Include my tests for the HFA corner cases. > - Try to improve handling of the overlapping registers as you suggested. > - Check nesting of HFA. > > There will surely be more when looking into Big Endian support after merging with your recent work on https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:OOB > We should get rid of oversized accesses on PPC64, too. > Thanks for sharing your plans to intrisify `linkToNative` in C2 later. I guess we should do more preparation work on all platforms when that gets addressed. @TheRealMDoerr I've moved the support for structs/unions that are not a power of 2 in size to this repo, so you should be able to merge the master branch to get it now. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From wkemper at openjdk.org Mon Mar 6 17:55:32 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 6 Mar 2023 17:55:32 GMT Subject: Integrated: Performance improvements for non-generational modes In-Reply-To: References: Message-ID: On Sat, 18 Feb 2023 01:56:10 GMT, William Kemper wrote: > There are a basket of changes here: > * Make heuristics a bit less aggressive: > * Revert threshold triggers to use `available` rather than adjusted head room. > * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically > * Skip over more generational mode constraints on the allocation path > * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. > * More detailed debugging messages under the `gc+free` tags. This pull request has now been integrated. Changeset: a1270d7e Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/a1270d7edbedcd111755e2ed6da6260683d1eeaf Stats: 216 lines in 12 files changed: 74 ins; 69 del; 73 mod Performance improvements for non-generational modes Reviewed-by: kdnilsen ------------- PR: https://git.openjdk.org/shenandoah/pull/220 From wkemper at openjdk.org Mon Mar 6 17:55:31 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 6 Mar 2023 17:55:31 GMT Subject: RFR: Performance improvements for non-generational modes In-Reply-To: <7fZ-6lUINo4FqBHyYfGOKey2EBLcaXGe089_sW8dBtA=.d457f943-714f-4e2b-92c6-ab5d5a5ee2c3@github.com> References: <7fZ-6lUINo4FqBHyYfGOKey2EBLcaXGe089_sW8dBtA=.d457f943-714f-4e2b-92c6-ab5d5a5ee2c3@github.com> Message-ID: On Fri, 3 Mar 2023 19:09:18 GMT, William Kemper wrote: >> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 314: >> >>> 312: range(1.0,100.0) \ >>> 313: \ >>> 314: product(uintx, ShenandoahMaxEvacLABRatio, 0, EXPERIMENTAL, \ >> >> Surprised this would have an effect on single-generation performance. Have we measured the costs/benefits of setting this default differently? If 0 works best, maybe we should remove this entirely? > > Yes, it was counter intuitive. This setting causes small chunks of memory to remain free in regions that have otherwise been consumed by gc lab allocations. When the evacuation reserve is exhausted, the collector will steal regions from the mutator. Because of the free memory in these regions, they are retained in the mutator's view of the freeset after the collection is complete. This in turn increases fragmentation in the mutator's view and reduces the amount of available memory to the mutators, causing the triggers to run more often. > > I only tested the default setting (16) and turning it off completely. I will test additional values. Additional tests with ShenandoahMaxEvacLABRatio=0, 8, 16, 32 shows the best performance with 0. All other values resulted in nearly full survivor regions being stuck in the mutator's free set. ------------- PR: https://git.openjdk.org/shenandoah/pull/220 From ysr at openjdk.org Mon Mar 6 20:23:03 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Mon, 6 Mar 2023 20:23:03 GMT Subject: RFR: Performance improvements for non-generational modes In-Reply-To: References: Message-ID: The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. ---------------------------------------------------------------------- On Sat, 18 Feb 2023 01:56:10 GMT, William Kemper wrote: > There are a basket of changes here: > * Make heuristics a bit less aggressive: > * Revert threshold triggers to use `available` rather than adjusted head room. > * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically > * Skip over more generational mode constraints on the allocation path > * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. > * More detailed debugging messages under the `gc+free` tags. Sorry to not get these comments back in time. I didn't understand some of the changes, although the majority seemed reasonable. I just had a few comments where the logging, control flow, and comments seemed to have become inconsistent (or at least confusing to the naive reader) on account of the changes. src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 371: > 369: size_t min_threshold = min_free_threshold(); > 370: > 371: if (available < min_threshold) { In general the changes to this method look somewhat ad-hoc to me. Is the intention to continue using `allocation_headroom` in the future, and this was a "quick-and-dirty" fix? The discrepancy between decision-flow and logging can create confusion. I see that the calculated `allocation_headroom` is being used further down (line 448) for control-flow. If possible, I'd define it closer to where it's going to be used, and move the related comments as well so that readers of this code & the comments above will not end up confused. src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 374: > 372: log_info(gc)("Trigger (%s): Free (" SIZE_FORMAT "%s) is below minimum threshold (" SIZE_FORMAT "%s)", > 373: _generation->name(), > 374: byte_size_in_proper_unit(allocation_headroom), proper_unit_for_byte_size(allocation_headroom), shouldn't this be `available` instead of `allocation_headroom` in the logs? src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 386: > 384: log_info(gc)("Trigger (%s): Learning " SIZE_FORMAT " of " SIZE_FORMAT ". Free (" SIZE_FORMAT "%s) is below initial threshold (" SIZE_FORMAT "%s)", > 385: _generation->name(), _gc_times_learned + 1, max_learn, > 386: byte_size_in_proper_unit(allocation_headroom), proper_unit_for_byte_size(allocation_headroom), same. src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 432: > 430: > 431: double avg_cycle_time = _gc_cycle_time_history->davg() + (_margin_of_error_sd * _gc_cycle_time_history->dsd()); > 432: Here again, we seem to have ended up with a discrepancy between the comments and the control flow that those comments applied to, but which has now disappeared? I'd remove the comments in case they are no longer applicable. If the idea is to revisit this in the future, might it be better to capture in a ticket rather than leaving in obsolete comments? In any case, it might make sense to go over these comments and make sure they reflect current reality and clearly call out future plans. ------------- PR: https://git.openjdk.org/shenandoah/pull/220 From wkemper at openjdk.org Mon Mar 6 22:03:01 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 6 Mar 2023 22:03:01 GMT Subject: RFR: Performance improvements for non-generational modes In-Reply-To: References: Message-ID: On Mon, 6 Mar 2023 19:34:54 GMT, Y. Srinivas Ramakrishna wrote: >> There are a basket of changes here: >> * Make heuristics a bit less aggressive: >> * Revert threshold triggers to use `available` rather than adjusted head room. >> * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically >> * Skip over more generational mode constraints on the allocation path >> * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. >> * More detailed debugging messages under the `gc+free` tags. > > src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 371: > >> 369: size_t min_threshold = min_free_threshold(); >> 370: >> 371: if (available < min_threshold) { > > In general the changes to this method look somewhat ad-hoc to me. Is the intention to continue using `allocation_headroom` in the future, and this was a "quick-and-dirty" fix? The discrepancy between decision-flow and logging can create confusion. > > I see that the calculated `allocation_headroom` is being used further down (line 448) for control-flow. If possible, I'd define it closer to where it's going to be used, and move the related comments as well so that readers of this code & the comments above will not end up confused. This is essentially reverting a change we made to have the `free` and `learning` triggers use the adjusted head room. > src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 374: > >> 372: log_info(gc)("Trigger (%s): Free (" SIZE_FORMAT "%s) is below minimum threshold (" SIZE_FORMAT "%s)", >> 373: _generation->name(), >> 374: byte_size_in_proper_unit(allocation_headroom), proper_unit_for_byte_size(allocation_headroom), > > shouldn't this be `available` instead of `allocation_headroom` in the logs? Using `available` here is as the code is on `openjdk/jdk:master`. ------------- PR: https://git.openjdk.org/shenandoah/pull/220 From wkemper at openjdk.org Mon Mar 6 22:08:48 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 6 Mar 2023 22:08:48 GMT Subject: RFR: Performance improvements for non-generational modes In-Reply-To: References: Message-ID: <7sTkQ4f7QmBbAuwG9v0ahw1sgn1ZVUl5eHmThkKcduU=.c81be199-e1fd-4d17-8ba7-a5cc1b83a347@github.com> On Mon, 6 Mar 2023 20:15:42 GMT, Y. Srinivas Ramakrishna wrote: >> There are a basket of changes here: >> * Make heuristics a bit less aggressive: >> * Revert threshold triggers to use `available` rather than adjusted head room. >> * Remove code that adjusted average gc time, it was causing wild fluctuations and making heuristic behave erratically >> * Skip over more generational mode constraints on the allocation path >> * Disable `ShenadoahEvacLABRatio` by default as it causes unused bytes in survivors regions, which in turn leads to these regions to being included in the mutator view of free memory. This contributed to higher fragmentation rates. >> * More detailed debugging messages under the `gc+free` tags. > > src/hotspot/share/gc/shenandoah/heuristics/shenandoahAdaptiveHeuristics.cpp line 432: > >> 430: >> 431: double avg_cycle_time = _gc_cycle_time_history->davg() + (_margin_of_error_sd * _gc_cycle_time_history->dsd()); >> 432: > > Here again, we seem to have ended up with a discrepancy between the comments and the control flow that those comments applied to, but which has now disappeared? I'd remove the comments in case they are no longer applicable. If the idea is to revisit this in the future, might it be better to capture in a ticket rather than leaving in obsolete comments? In any case, it might make sense to go over these comments and make sure they reflect current reality and clearly call out future plans. I'm not sure why you think the comment doesn't match the control flow. I left in the `TODO` intentionally. The preceding comment does describe how the heuristic operates - maintaining a model of an average allocation rate, but also looking for 'spikes' that are outside of what the model expects. ------------- PR: https://git.openjdk.org/shenandoah/pull/220 From tschatzl at openjdk.org Tue Mar 7 09:57:19 2023 From: tschatzl at openjdk.org (Thomas Schatzl) Date: Tue, 7 Mar 2023 09:57:19 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects [v2] In-Reply-To: References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: On Mon, 6 Mar 2023 10:22:27 GMT, Ivan Walulya wrote: >> Hi All, >> >> Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. >> >> Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. >> >> We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. >> >> Testing: Tier 1-3 > > Ivan Walulya has updated the pull request incrementally with two additional commits since the last revision: > > - Refactor resetting humongous metadata > - Thomas review Looks good. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.org/jdk/pull/12830 From mdoerr at openjdk.org Tue Mar 7 14:33:22 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 7 Mar 2023 14:33:22 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v10] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 10 commits: - Merge remote-tracking branch 'origin' into PPC64_Panama - Handle HFA corner cases with overlapping registers in Java. - Fix merge bug. - Merge branch 'master' into PPC64_Panama - Add test for HFA corner cases. - Minor cleanup. - HFA: Add support for nested structures. See JDK-8300294. - Remove size restriction for structs. Add TODO for Big Endian. - Clean fix for NativeMemorySegmentImpl issue with byteSize 0. - Initial Panama implementation. ------------- Changes: https://git.openjdk.org/jdk/pull/12708/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=09 Stats: 2287 lines in 59 files changed: 2157 ins; 16 del; 114 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 8 13:22:50 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 8 Mar 2023 13:22:50 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v11] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Remove STRUCT_REFERENCE which was incorrectly taken from aarch64. Pass size to bufferLoad/Store. Enable TestNested.java. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/3a541d00..f75a240d Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=10 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=09-10 Stats: 25 lines in 3 files changed: 0 ins; 21 del; 4 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From wkemper at openjdk.org Wed Mar 8 22:10:58 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 8 Mar 2023 22:10:58 GMT Subject: RFR: Use relevant variables in log messages, remove unused import Message-ID: Addresses feedback in PR#220. ------------- Commit messages: - Use relevant variables in log messages, remove unused import Changes: https://git.openjdk.org/shenandoah/pull/222/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=222&range=00 Stats: 35 lines in 1 file changed: 10 ins; 23 del; 2 mod Patch: https://git.openjdk.org/shenandoah/pull/222.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/222/head:pull/222 PR: https://git.openjdk.org/shenandoah/pull/222 From ysr at openjdk.org Wed Mar 8 22:42:40 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 8 Mar 2023 22:42:40 GMT Subject: RFR: Use relevant variables in log messages, remove unused import In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 22:03:01 GMT, William Kemper wrote: > Addresses feedback in PR#220. LGTM; ship it! ------------- Marked as reviewed by ysr (Author). PR: https://git.openjdk.org/shenandoah/pull/222 From mdoerr at openjdk.org Thu Mar 9 14:02:35 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 9 Mar 2023 14:02:35 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v12] In-Reply-To: References: Message-ID: The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. ---------------------------------------------------------------------- > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: The merge change has messed up some includes in the tests. Revert. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/f75a240d..a2eed058 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=11 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=10-11 Stats: 44 lines in 6 files changed: 15 ins; 21 del; 8 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From kbarrett at openjdk.org Thu Mar 9 15:43:06 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Thu, 9 Mar 2023 15:43:06 GMT Subject: RFR: 8303900: Rename BitMap search functions Message-ID: Please review this renaming of the following functions in BitMap: get_next_one_offset => find_first_set_bit get_next_zero_offset => find_first_clear_bit get_next_one_offset_aligned_right => find_first_set_bit_aligned_right Note that ShenandoahMarkBitMap::get_next_one_offset is not being renamed. For some reason that class contains a copy of a sizable chunk of the code from some version of BitMap. (Not sure why it doesn't use an internal BitMapView rather than code copying.) Testing: mach5 tier1-5 ------------- Commit messages: - update gtests - update shenandoah uses - update zgc uses - update gc/shared uses - update Parallel uses - update G1 uses - rename functions Changes: https://git.openjdk.org/jdk/pull/12951/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12951&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8303900 Stats: 75 lines in 12 files changed: 0 ins; 0 del; 75 mod Patch: https://git.openjdk.org/jdk/pull/12951.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12951/head:pull/12951 PR: https://git.openjdk.org/jdk/pull/12951 From mdoerr at openjdk.org Thu Mar 9 15:51:56 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 9 Mar 2023 15:51:56 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v13] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Remove LinuxPPC64CallArranger.java because it doesn't contain anything. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/a2eed058..fb87284c Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=12 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=11-12 Stats: 44 lines in 3 files changed: 0 ins; 38 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Thu Mar 9 15:56:19 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 9 Mar 2023 15:56:19 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Wed, 1 Mar 2023 06:27:19 GMT, Martin Doerr wrote: >> src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/CallArranger.java line 68: >> >>> 66: public abstract class CallArranger { >>> 67: // Linux PPC64 Little Endian uses ABI v2. >>> 68: private static final boolean useABIv2 = ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN; >> >> Now that I'm here. This could be a potentially interesting case for having 2 subclasses of CallArranger: one for `useABIv2 == true` and one for `false`. > > Yeah, let's wait until we know what changes we need for AIX (and Big Endian linux). I think having a ABIV1CallArranger (for Big Endian linux and AIX) and a ABIV2CallArranger (linux ppc64le) will make sense. That's why I have removed the LinuxPPC64CallArranger.java with https://github.com/openjdk/jdk/pull/12708/commits/fb87284c1d3df946db378d196d7f48cd3acbab01. I guess a good time for such cleanup will be after the code for all variants is available (not part of this PR). ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Thu Mar 9 17:29:37 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 9 Mar 2023 17:29:37 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v14] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Introduce ABIv2CallArranger for linux ppc64le. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/fb87284c..2e4e269e Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=13 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=12-13 Stats: 178 lines in 8 files changed: 99 ins; 58 del; 21 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Thu Mar 9 17:42:50 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 9 Mar 2023 17:42:50 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Mon, 6 Mar 2023 16:38:37 GMT, Jorn Vernee wrote: >> @JornVernee: Thanks a lot for your detailed review! I have quite a few TODOs which include: >> - Include my tests for the HFA corner cases. >> - Try to improve handling of the overlapping registers as you suggested. >> - Check nesting of HFA. >> >> There will surely be more when looking into Big Endian support after merging with your recent work on https://github.com/openjdk/panama-foreign/compare/foreign-memaccess+abi...JornVernee:panama-foreign:OOB >> We should get rid of oversized accesses on PPC64, too. >> Thanks for sharing your plans to intrisify `linkToNative` in C2 later. I guess we should do more preparation work on all platforms when that gets addressed. > > @TheRealMDoerr I've moved the support for structs/unions that are not a power of 2 in size to this repo, so you should be able to merge the master branch to get it now. @JornVernee: Thanks! I've merged in your changes. TestArrayStructs is not yet completely working. I will need to investigate. I think I've done most other things you had requested. You may want to take a look at my recent commits. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From stefank at openjdk.org Thu Mar 9 18:04:40 2023 From: stefank at openjdk.org (Stefan Karlsson) Date: Thu, 9 Mar 2023 18:04:40 GMT Subject: RFR: 8303900: Rename BitMap search functions In-Reply-To: References: Message-ID: <4GIB9OwfDN2SPR-vA9cUIN2eNhWjqxV47tBLlsoEWuM=.278dc6e2-93f6-438d-8938-8bf0f35c5599@github.com> On Thu, 9 Mar 2023 15:34:05 GMT, Kim Barrett wrote: > Please review this renaming of the following functions in BitMap: > > get_next_one_offset => find_first_set_bit > get_next_zero_offset => find_first_clear_bit > get_next_one_offset_aligned_right => find_first_set_bit_aligned_right > > Note that ShenandoahMarkBitMap::get_next_one_offset is not being renamed. For > some reason that class contains a copy of a sizable chunk of the code from > some version of BitMap. (Not sure why it doesn't use an internal BitMapView > rather than code copying.) > > Testing: > mach5 tier1-5 Looks good. ------------- Marked as reviewed by stefank (Reviewer). PR: https://git.openjdk.org/jdk/pull/12951 From kdnilsen at openjdk.org Thu Mar 9 21:05:01 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 9 Mar 2023 21:05:01 GMT Subject: RFR: Use relevant variables in log messages, remove unused import In-Reply-To: References: Message-ID: <4Abs-Mkovo3iIMU0g6AYNR0TBB6LbPQ0dOqsz9WTDDQ=.31dc4ce3-29ae-4361-80cd-a52b2845a0fb@github.com> On Wed, 8 Mar 2023 22:03:01 GMT, William Kemper wrote: > Addresses feedback in PR#220. Marked as reviewed by kdnilsen (Committer). ------------- PR: https://git.openjdk.org/shenandoah/pull/222 From wkemper at openjdk.org Fri Mar 10 00:49:52 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 10 Mar 2023 00:49:52 GMT Subject: Integrated: Use relevant variables in log messages, remove unused import In-Reply-To: References: Message-ID: On Wed, 8 Mar 2023 22:03:01 GMT, William Kemper wrote: > Addresses feedback in PR#220. This pull request has now been integrated. Changeset: 87bf68f3 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/87bf68f35c05c063c8a5865b22e1b9704f99e7e5 Stats: 35 lines in 1 file changed: 10 ins; 23 del; 2 mod Use relevant variables in log messages, remove unused import Reviewed-by: ysr, kdnilsen ------------- PR: https://git.openjdk.org/shenandoah/pull/222 From aboldtch at openjdk.org Fri Mar 10 12:39:03 2023 From: aboldtch at openjdk.org (Axel Boldt-Christmas) Date: Fri, 10 Mar 2023 12:39:03 GMT Subject: RFR: 8303900: Rename BitMap search functions In-Reply-To: References: Message-ID: <1EhzQNYr_PTQylNyTm4u4VPII9VuSt26QJmm7pORNFs=.02f76a7f-7cd4-45d5-8602-ac8613fe8c81@github.com> On Thu, 9 Mar 2023 15:34:05 GMT, Kim Barrett wrote: > Please review this renaming of the following functions in BitMap: > > get_next_one_offset => find_first_set_bit > get_next_zero_offset => find_first_clear_bit > get_next_one_offset_aligned_right => find_first_set_bit_aligned_right > > Note that ShenandoahMarkBitMap::get_next_one_offset is not being renamed. For > some reason that class contains a copy of a sizable chunk of the code from > some version of BitMap. (Not sure why it doesn't use an internal BitMapView > rather than code copying.) > > Testing: > mach5 tier1-5 lgtm ------------- Marked as reviewed by aboldtch (Committer). PR: https://git.openjdk.org/jdk/pull/12951 From kbarrett at openjdk.org Fri Mar 10 21:10:30 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 10 Mar 2023 21:10:30 GMT Subject: RFR: 8303900: Rename BitMap search functions [v2] In-Reply-To: References: Message-ID: > Please review this renaming of the following functions in BitMap: > > get_next_one_offset => find_first_set_bit > get_next_zero_offset => find_first_clear_bit > get_next_one_offset_aligned_right => find_first_set_bit_aligned_right > > Note that ShenandoahMarkBitMap::get_next_one_offset is not being renamed. For > some reason that class contains a copy of a sizable chunk of the code from > some version of BitMap. (Not sure why it doesn't use an internal BitMapView > rather than code copying.) > > Testing: > mach5 tier1-5 Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: - Merge branch 'master' into find-rename - update gtests - update shenandoah uses - update zgc uses - update gc/shared uses - update Parallel uses - update G1 uses - rename functions ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12951/files - new: https://git.openjdk.org/jdk/pull/12951/files/94fa65a5..c503e584 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12951&range=01 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12951&range=00-01 Stats: 63148 lines in 577 files changed: 54630 ins; 2410 del; 6108 mod Patch: https://git.openjdk.org/jdk/pull/12951.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12951/head:pull/12951 PR: https://git.openjdk.org/jdk/pull/12951 From kbarrett at openjdk.org Fri Mar 10 21:10:33 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 10 Mar 2023 21:10:33 GMT Subject: RFR: 8303900: Rename BitMap search functions [v2] In-Reply-To: <4GIB9OwfDN2SPR-vA9cUIN2eNhWjqxV47tBLlsoEWuM=.278dc6e2-93f6-438d-8938-8bf0f35c5599@github.com> References: <4GIB9OwfDN2SPR-vA9cUIN2eNhWjqxV47tBLlsoEWuM=.278dc6e2-93f6-438d-8938-8bf0f35c5599@github.com> Message-ID: On Thu, 9 Mar 2023 18:01:57 GMT, Stefan Karlsson wrote: >> Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: >> >> - Merge branch 'master' into find-rename >> - update gtests >> - update shenandoah uses >> - update zgc uses >> - update gc/shared uses >> - update Parallel uses >> - update G1 uses >> - rename functions > > Looks good. Thanks for reviews @stefank and @xmas92 ------------- PR: https://git.openjdk.org/jdk/pull/12951 From kbarrett at openjdk.org Fri Mar 10 21:20:56 2023 From: kbarrett at openjdk.org (Kim Barrett) Date: Fri, 10 Mar 2023 21:20:56 GMT Subject: Integrated: 8303900: Rename BitMap search functions In-Reply-To: References: Message-ID: On Thu, 9 Mar 2023 15:34:05 GMT, Kim Barrett wrote: > Please review this renaming of the following functions in BitMap: > > get_next_one_offset => find_first_set_bit > get_next_zero_offset => find_first_clear_bit > get_next_one_offset_aligned_right => find_first_set_bit_aligned_right > > Note that ShenandoahMarkBitMap::get_next_one_offset is not being renamed. For > some reason that class contains a copy of a sizable chunk of the code from > some version of BitMap. (Not sure why it doesn't use an internal BitMapView > rather than code copying.) > > Testing: > mach5 tier1-5 This pull request has now been integrated. Changeset: 21169285 Author: Kim Barrett URL: https://git.openjdk.org/jdk/commit/2116928528c0554b2ba0171bd7968ab693972804 Stats: 75 lines in 12 files changed: 0 ins; 0 del; 75 mod 8303900: Rename BitMap search functions Reviewed-by: stefank, aboldtch ------------- PR: https://git.openjdk.org/jdk/pull/12951 From jvernee at openjdk.org Sat Mar 11 02:23:24 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Sat, 11 Mar 2023 02:23:24 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Thu, 9 Mar 2023 17:40:02 GMT, Martin Doerr wrote: >> @TheRealMDoerr I've moved the support for structs/unions that are not a power of 2 in size to this repo, so you should be able to merge the master branch to get it now. > > @JornVernee: Thanks! I've merged in your changes. TestArrayStructs is not yet completely working. I will need to investigate. I think I've done most other things you had requested. You may want to take a look at my recent commits. @TheRealMDoerr I've been keeping up with the changes you're making. I just have to look at the new test for HFA's you've added you added (next week). Besides fixing the TestArrayStructs test, do you have anything else you still want to add to this PR? ------------- PR: https://git.openjdk.org/jdk/pull/12708 From jvernee at openjdk.org Sat Mar 11 02:23:25 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Sat, 11 Mar 2023 02:23:25 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: <0RrqDfVTh0EmUK3xCInQa1gXbZwdLpd_jY-3TYgdqwo=.e1ca03f4-8e66-4d3d-97f2-59f5bbd7287b@github.com> On Fri, 3 Mar 2023 10:59:32 GMT, Martin Doerr wrote: >> Thanks! I need to find extra time for this. Sounds like a good idea and I may be able to get rid of some nasty code. > > Done by https://github.com/openjdk/jdk/pull/12708/commits/98e242c24c07ea977b7709b9f8d0c10ce87e84c0 (using a record instead of a `VMStorage[][]` because I think this is better readable). Note that it's a bit more complicated. I couldn't use your `dup` trick, because I need to put the value into a GP reg and one half of it to a FP reg. The Panama code doesn't support that (IllegalArgumentException: Invalid operand type: interface java.lang.foreign.MemorySegment. float expected). Thanks, your solution looks good. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Mon Mar 13 16:23:21 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 13 Mar 2023 16:23:21 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: On Sat, 11 Mar 2023 02:20:31 GMT, Jorn Vernee wrote: >> @JornVernee: Thanks! I've merged in your changes. TestArrayStructs is not yet completely working. I will need to investigate. I think I've done most other things you had requested. You may want to take a look at my recent commits. > > @TheRealMDoerr I've been keeping up with the changes you're making. I just have to look at the new test for HFA's you've added you added (next week). > > Besides fixing the TestArrayStructs test, do you have anything else you still want to add to this PR? @JornVernee: I guess I should add a couple of Upcalls to my new test. Otherwise, I have only planned to fix `TestArrayStructs`. Further changes (and maybe new tests) can still get done when working on Big Endian / AIX or when there is a demand. I'm currently wondering about the `TestArrayStructs` failures. Passing arrays with up to 7 elements seems to work fine. When I pass 8 elements, the last element of `capturedArgs` gets observed as 0. When I pass more than 8 elements, element 5 and 6 of `capturedArgs` get observed as 0. `DowncallLinker.invokeInterpBindings` has the correct `args`, but `UpcallLinker.invokeInterpBindings` doesn't receive the correct values as `lowLevelArgs`. They contain the wrong zeros. The remaining elements look correct. Do you have an idea what could be going wrong? Otherwise, I'll have to continue debugging. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From jvernee at openjdk.org Mon Mar 13 16:40:18 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Mon, 13 Mar 2023 16:40:18 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> Message-ID: <1OXigMWnfCHCkxCI3D29NpgIpgG36Ltlnry1eytOPow=.5f423637-0a12-454e-a31c-57b2fc11123f@github.com> On Mon, 13 Mar 2023 16:20:22 GMT, Martin Doerr wrote: > I guess I should add a couple of Upcalls to my new test. Otherwise, I have only planned to fix TestArrayStructs. Further changes (and maybe new tests) can still get done when working on Big Endian / AIX or when there is a demand. Okay. The reason I ask is because we are looking to get started with the JEP PR, but it might be nice to wrap up this PR first, to avoid any merge conflicts. I also have https://github.com/openjdk/jdk/pull/12883 which would need changes to the PPC call arranger/linker as well (though, it's a pretty simple refactor). I'm trying to figure out what the best order to do things in is. > I'm currently wondering about the TestArrayStructs failures. Passing arrays with up to 7 elements seems to work fine. When I pass 8 elements, the last element of capturedArgs gets observed as 0. When I pass more than 8 elements, element 5 and 6 of capturedArgs get observed as 0. DowncallLinker.invokeInterpBindings has the correct args, but UpcallLinker.invokeInterpBindings doesn't receive the correct values as lowLevelArgs. They contain the wrong zeros. The remaining elements look correct. Do you have an idea what could be going wrong? Otherwise, I'll have to continue debugging. This sounds like there might be a mismatch between the Java and native side. I suggest looking at the assembly generated for the native function for the failing case, and seeing if it matches what is generated by CallArranger. Here is also where adding a CallArranger test can be useful (in test/jdk/java/foreign/callarranger), to test whether the resulting bindings match your expectation for that function descriptor. Also, you might want to check the layout the native compiler uses for the particular struct, and verify that it matches the Java side. (i.e. there's no weird padding or something, it's just a struct of 8 bytes). ------------- PR: https://git.openjdk.org/jdk/pull/12708 From jvernee at openjdk.org Mon Mar 13 16:57:43 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Mon, 13 Mar 2023 16:57:43 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v14] In-Reply-To: References: Message-ID: On Thu, 9 Mar 2023 17:29:37 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: > > Introduce ABIv2CallArranger for linux ppc64le. test/jdk/java/foreign/TestHFA.java line 32: > 30: * @enablePreview > 31: * @requires ((os.arch == "amd64" | os.arch == "x86_64") & sun.arch.data.model == "64") | os.arch == "aarch64" | os.arch == "ppc64le" | os.arch == "riscv64" > 32: * @requires !vm.musl Not sure if this test should be disabled on musl? ------------- PR: https://git.openjdk.org/jdk/pull/12708 From wkemper at openjdk.org Mon Mar 13 18:47:10 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 13 Mar 2023 18:47:10 GMT Subject: RFR: Fix incorrect assert logic Message-ID: Fix incorrect assert logic ------------- Commit messages: - Fix incorrect assert logic Changes: https://git.openjdk.org/shenandoah/pull/224/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=224&range=00 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/shenandoah/pull/224.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/224/head:pull/224 PR: https://git.openjdk.org/shenandoah/pull/224 From kdnilsen at openjdk.org Mon Mar 13 18:47:12 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 13 Mar 2023 18:47:12 GMT Subject: RFR: Fix incorrect assert logic In-Reply-To: References: Message-ID: On Mon, 13 Mar 2023 18:24:37 GMT, William Kemper wrote: > Fix incorrect assert logic Marked as reviewed by kdnilsen (Committer). ------------- PR: https://git.openjdk.org/shenandoah/pull/224 From wkemper at openjdk.org Mon Mar 13 19:03:53 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 13 Mar 2023 19:03:53 GMT Subject: Integrated: Fix incorrect assert logic In-Reply-To: References: Message-ID: <5yq-Gp_T2uqudakWifzg0aBeq8If0WlOPc7A6IWzvEA=.43353058-fb7b-4b18-b415-0a37e6bf591a@github.com> On Mon, 13 Mar 2023 18:24:37 GMT, William Kemper wrote: > Fix incorrect assert logic This pull request has now been integrated. Changeset: b4ee7316 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/b4ee731658e100c8375099cbcb9496b6747c1d7e Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Fix incorrect assert logic Reviewed-by: kdnilsen ------------- PR: https://git.openjdk.org/shenandoah/pull/224 From wkemper at openjdk.org Mon Mar 13 20:23:35 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 13 Mar 2023 20:23:35 GMT Subject: RFR: Merge openjdk/jdk:master Message-ID: Merge jdk-21+13 ------------- Commit messages: - Fix use of CardTableRS - Merge tag 'jdk-21+13' into merge-jdk-21-13 - 8171156: Class java.util.LocaleISOData has outdated information for country Code NP - 8299570: [JVMCI] Insufficient error handling when CodeBuffer is exhausted - 8303833: java.util.LocaleISOData has wrong comments for 'Norwegian Bokm?l' and 'Volap?k' - 8303839: [BACKOUT] JDK-8302799 and JDK-8302189 - 8282201: Consider removal of expiry check in VerifyCACerts.java test - 8302508: Add timestamp to the output TraceCompilerThreads - 8302161: Upgrade jQuery UI to version 1.13.2 - 8303617: update for deprecated sprintf for jdk.jdwp.agent - ... and 168 more: https://git.openjdk.org/shenandoah/compare/87bf68f3...2eaac783 The webrevs contain the adjustments done while merging with regards to each parent branch: - master: https://webrevs.openjdk.org/?repo=shenandoah&pr=225&range=00.0 - openjdk/jdk:master: https://webrevs.openjdk.org/?repo=shenandoah&pr=225&range=00.1 Changes: https://git.openjdk.org/shenandoah/pull/225/files Stats: 26934 lines in 958 files changed: 17048 ins; 5328 del; 4558 mod Patch: https://git.openjdk.org/shenandoah/pull/225.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/225/head:pull/225 PR: https://git.openjdk.org/shenandoah/pull/225 From mdoerr at openjdk.org Mon Mar 13 21:00:31 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Mon, 13 Mar 2023 21:00:31 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: <1OXigMWnfCHCkxCI3D29NpgIpgG36Ltlnry1eytOPow=.5f423637-0a12-454e-a31c-57b2fc11123f@github.com> References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> <1OXigMWnfCHCkxCI3D29NpgIpgG36Ltlnry1eytOPow=.5f423637-0a12-454e-a31c-57b2fc11123f@github.com> Message-ID: <8I_sDu55v8ThrFyF4mMWbKOOKeGkfYvqo66JLNnULUY=.830fe48d-b65f-437c-82fa-9e7dac76c7ab@github.com> On Mon, 13 Mar 2023 16:37:18 GMT, Jorn Vernee wrote: > > I'm currently wondering about the TestArrayStructs failures. Passing arrays with up to 7 elements seems to work fine. When I pass 8 elements, the last element of capturedArgs gets observed as 0. When I pass more than 8 elements, element 5 and 6 of capturedArgs get observed as 0. > > DowncallLinker.invokeInterpBindings has the correct args, but UpcallLinker.invokeInterpBindings doesn't receive the correct values as lowLevelArgs. They contain the wrong zeros. The remaining elements look correct. > > Do you have an idea what could be going wrong? Otherwise, I'll have to continue debugging. > > This sounds like there might be a mismatch between the Java and native side. I suggest looking at the assembly generated for the native function for the failing case, and seeing if it matches what is generated by CallArranger. Here is also where adding a CallArranger test can be useful (in test/jdk/java/foreign/callarranger), to test whether the resulting bindings match your expectation for that function descriptor. > > Also, you might want to check the layout the native compiler uses for the particular struct, and verify that it matches the Java side. (i.e. there's no weird padding or something, it's just a struct of 8 bytes). Note that argument and return value passing works. I'm getting all values back. So, the native side seems to be ok. Only (one or two) values in `returnBox` are broken. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From wkemper at openjdk.org Mon Mar 13 22:17:45 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 13 Mar 2023 22:17:45 GMT Subject: Integrated: Merge openjdk/jdk:master In-Reply-To: References: Message-ID: On Mon, 13 Mar 2023 20:15:52 GMT, William Kemper wrote: > Merge jdk-21+13 This pull request has now been integrated. Changeset: b909ec39 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/b909ec398f7140ff7975050b96f130003ef17f0c Stats: 26934 lines in 958 files changed: 17048 ins; 5328 del; 4558 mod Merge openjdk/jdk:master ------------- PR: https://git.openjdk.org/shenandoah/pull/225 From jvernee at openjdk.org Tue Mar 14 03:05:41 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Tue, 14 Mar 2023 03:05:41 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v3] In-Reply-To: <8I_sDu55v8ThrFyF4mMWbKOOKeGkfYvqo66JLNnULUY=.830fe48d-b65f-437c-82fa-9e7dac76c7ab@github.com> References: <8b3vVrV22RuhdRoRYacXV0ZeghFGgKkC8S_z-iMrzAQ=.dd84b743-8b51-4281-8f5f-f9eff6207bc7@github.com> <1OXigMWnfCHCkxCI3D29NpgIpgG36Ltlnry1eytOPow=.5f423637-0a12-454e-a31c-57b2fc11123f@github.com> <8I_sDu55v8ThrFyF4mMWbKOOKeGkfYvqo66 JLNnULUY=.830fe48d-b65f-437c-82fa-9e7dac76c7ab@github.com> Message-ID: On Mon, 13 Mar 2023 20:57:22 GMT, Martin Doerr wrote: > Note that argument and return value passing works. I'm getting all values back. So, the native side seems to be ok. Only (one or two) values in `returnBox` are broken. You mean you tested by returning each element of the array one by one? If so, that rules out an issue with the downcall, or the struct not being in the right format (in the register), I think. I suggest checking the bindings generated for the upcall, and seeing if they match what the native code does. This could be done in `jshell` for instance: $ .\build\windows-fastdebug\images\jdk\bin\jshell.exe --enable-preview '--add-exports=java.base/jdk.internal.foreign.abi.x64.windows=ALL-UNNAMED' '--add-exports=java.base/jdk.internal.foreign.abi=ALL-UNNAMED' jshell> import java.lang.foreign.* jshell> import java.lang.invoke.* jshell> import static java.lang.foreign.ValueLayout.* jshell> import jdk.internal.foreign.abi.x64.windows.CallArranger jshell> MemoryLayout.structLayout(MemoryLayout.sequenceLayout(8, JAVA_BYTE)) $5 ==> [[8:b8]] jshell> FunctionDescriptor.of($5, $5, JAVA_BYTE, JAVA_BYTE, JAVA_BYTE, JAVA_BYTE, JAVA_BYTE, JAVA_BYTE, JAVA_BYTE, JAVA_BYTE) $6 ==> ([[8:b8]]b8b8b8b8b8b8b8b8)[[8:b8]] jshell> CallArranger.getBindings($6.toMethodType(), $6, true) $7 ==> Bindings[callingSequence=jdk.internal.foreign.abi.CallingSequence at 396e2f39, isInMemoryReturn=false] jshell> System.out.println($7.callingSequence().asString()) CallingSequence: { callerMethodType: (long,int,int,int,int,int,int,int,int)long calleeMethodType: (MemorySegment,byte,byte,byte,byte,byte,byte,byte,byte)MemorySegment FunctionDescriptor: ([[8:b8]]b8b8b8b8b8b8b8b8)[[8:b8]] Argument Bindings: 0: [Allocate[size=8, alignment=1], Dup[], VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=15, indexOrOffset=1, debugName=rcx], type=long], BufferStore[offset=0, type=long, byteWidth=8]] 1: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=15, indexOrOffset=2, debugName=rdx], type=int], INT_TO_BYTE] 2: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=15, indexOrOffset=8, debugName=r8], type=int], INT_TO_BYTE] 3: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=15, indexOrOffset=9, debugName=r9], type=int], INT_TO_BYTE] 4: [VMLoad[storage=VMStorage[type=3, segmentMaskOrSize=8, indexOrOffset=0, debugName=Stack at 0], type=int], INT_TO_BYTE] 5: [VMLoad[storage=VMStorage[type=3, segmentMaskOrSize=8, indexOrOffset=8, debugName=Stack at 8], type=int], INT_TO_BYTE] 6: [VMLoad[storage=VMStorage[type=3, segmentMaskOrSize=8, indexOrOffset=16, debugName=Stack at 16], type=int], INT_TO_BYTE] 7: [VMLoad[storage=VMStorage[type=3, segmentMaskOrSize=8, indexOrOffset=24, debugName=Stack at 24], type=int], INT_TO_BYTE] 8: [VMLoad[storage=VMStorage[type=3, segmentMaskOrSize=8, indexOrOffset=32, debugName=Stack at 32], type=int], INT_TO_BYTE] Return bindings: [BufferLoad[offset=0, type=long, byteWidth=8], VMStore[storage=VMStorage[type=0, segmentMaskOrSize=15, indexOrOffset=0, debugName=rax], type=long]] } ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Tue Mar 14 20:33:34 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 14 Mar 2023 20:33:34 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v15] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Allow TestHFA to run on musl. Add Upcalls. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/2e4e269e..364a84ed Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=14 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=13-14 Stats: 208 lines in 2 files changed: 204 ins; 1 del; 3 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Tue Mar 14 20:33:40 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 14 Mar 2023 20:33:40 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v14] In-Reply-To: References: Message-ID: On Mon, 13 Mar 2023 14:31:19 GMT, Jorn Vernee wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Introduce ABIv2CallArranger for linux ppc64le. > > test/jdk/java/foreign/TestHFA.java line 32: > >> 30: * @enablePreview >> 31: * @requires ((os.arch == "amd64" | os.arch == "x86_64") & sun.arch.data.model == "64") | os.arch == "aarch64" | os.arch == "ppc64le" | os.arch == "riscv64" >> 32: * @requires !vm.musl > > Not sure if this test should be disabled on musl? Changed with https://github.com/openjdk/jdk/pull/12708/commits/364a84edc416abd5f1318f78057c92720fe96990. Plus added Upcalls. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Tue Mar 14 22:30:22 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 14 Mar 2023 22:30:22 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v16] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/364a84ed..9173af20 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=15 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=14-15 Stats: 14 lines in 2 files changed: 5 ins; 3 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Tue Mar 14 22:34:47 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 14 Mar 2023 22:34:47 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v16] In-Reply-To: References: Message-ID: On Tue, 14 Mar 2023 22:30:22 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: > > Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. Thanks for the hint. Using `jshell` is a good idea. { callerMethodType: (long,int,int,int,int,int,int,int,int)long calleeMethodType: (MemorySegment,byte,byte,byte,byte,byte,byte,byte,byte)MemorySegment FunctionDescriptor: ([[8:b8]]b8b8b8b8b8b8b8b8)[[8:b8]] Argument Bindings: 0: [Allocate[size=8, alignment=1], Dup[], VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=3, debugName=r3], type=long], BufferStore[offset=0, type=long, byteWidth=8]] 1: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=4, debugName=r4], type=int], INT_TO_BYTE] 2: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=5, debugName=r5], type=int], INT_TO_BYTE] 3: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=6, debugName=r6], type=int], INT_TO_BYTE] 4: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=7, debugName=r7], type=int], INT_TO_BYTE] 5: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=8, debugName=r8], type=int], INT_TO_BYTE] 6: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=9, debugName=r9], type=int], INT_TO_BYTE] 7: [VMLoad[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=10, debugName=r10], type=int], INT_TO_BYTE] 8: [VMLoad[storage=VMStorage[type=2, segmentMaskOrSize=8, indexOrOffset=64, debugName=Stack at 64], type=int], INT_TO_BYTE] Return: [BufferLoad[offset=0, type=long, byteWidth=8], VMStore[storage=VMStorage[type=0, segmentMaskOrSize=3, indexOrOffset=3, debugName=r3], type=long]] } I have fixed the issue and enabled the test. My code still contained an old assumption which doesn't hold for the cases I had implemented more recently. So, this PR is probably complete. I'll rerun tests and wait for 2 reviews. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Tue Mar 14 22:55:51 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Tue, 14 Mar 2023 22:55:51 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v16] In-Reply-To: References: Message-ID: On Tue, 14 Mar 2023 22:30:22 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: > > Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. Btw. the new cases in which we use int and short accesses when byteWidth is not a power of 2 are never unaligned AFAICS. I guess _UNALIGNED is unnecessary in the JAVA_INT_UNALIGNED and JAVA_SHORT_UNALIGNED. They are always aligned wrt. to their size. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From jsjolen at openjdk.org Wed Mar 15 11:09:25 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 15 Mar 2023 11:09:25 GMT Subject: RFR: JDK-8301498: Replace NULL with nullptr in cpu/x86 [v3] In-Reply-To: References: Message-ID: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Fix vnkozlov's suggestions - Merge remote-tracking branch 'origin/master' into JDK-8301498 - Some more fixes - Fixes - Replace NULL with nullptr in cpu/x86 ------------- Changes: https://git.openjdk.org/jdk/pull/12326/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12326&range=02 Stats: 665 lines in 54 files changed: 0 ins; 0 del; 665 mod Patch: https://git.openjdk.org/jdk/pull/12326.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12326/head:pull/12326 PR: https://git.openjdk.org/jdk/pull/12326 From jsjolen at openjdk.org Wed Mar 15 11:09:27 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 15 Mar 2023 11:09:27 GMT Subject: RFR: JDK-8301498: Replace NULL with nullptr in cpu/x86 [v2] In-Reply-To: References: Message-ID: On Mon, 13 Feb 2023 09:26:07 GMT, Johan Sj?len wrote: >> Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we >> need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. >> >> Here are some typical things to look out for: >> >> 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). >> 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. >> 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. >> >> An example of this: >> >> ```c++ >> // This function returns null >> void* ret_null(); >> // This function returns true if *x == nullptr >> bool is_nullptr(void** x); >> >> >> Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. >> >> Thanks! > > Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision: > > Some more fixes Hi Vladimir! I applied your changes, thanks for the review. ------------- PR: https://git.openjdk.org/jdk/pull/12326 From jsjolen at openjdk.org Wed Mar 15 13:13:23 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 15 Mar 2023 13:13:23 GMT Subject: RFR: JDK-8301495: Replace NULL with nullptr in cpu/ppc Message-ID: Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/ppc. Unfortunately the script that does the change isn't perfect, and so we need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. Here are some typical things to look out for: 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. An example of this: ```c++ // This function returns null void* ret_null(); // This function returns true if *x == nullptr bool is_nullptr(void** x); Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. Thanks! ------------- Commit messages: - Merge remote-tracking branch 'origin/master' into JDK-8301495 - Revert change in file - Fixes - Merge remote-tracking branch 'origin/master' into JDK-8301495 - reinrich suggestions - Replace NULL with nullptr in cpu/ppc Changes: https://git.openjdk.org/jdk/pull/12323/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12323&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8301495 Stats: 384 lines in 51 files changed: 0 ins; 0 del; 384 mod Patch: https://git.openjdk.org/jdk/pull/12323.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12323/head:pull/12323 PR: https://git.openjdk.org/jdk/pull/12323 From rrich at openjdk.org Wed Mar 15 13:13:37 2023 From: rrich at openjdk.org (Richard Reingruber) Date: Wed, 15 Mar 2023 13:13:37 GMT Subject: RFR: JDK-8301495: Replace NULL with nullptr in cpu/ppc In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:39:48 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/ppc. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! Looks good, just a few changes I'd like to suggest. Thanks, Richard. src/hotspot/cpu/ppc/abstractInterpreter_ppc.cpp line 97: > 95: // Parameters: > 96: // > 97: // interpreter_frame != null: Suggestion: // interpreter_frame != nullptr: src/hotspot/cpu/ppc/frame_ppc.cpp line 114: > 112: > 113: // At this point, there still is a chance that fp_safe is false. > 114: // In particular, (fp == null) might be true. So let's check and Suggestion: // In particular, (fp == nullptr) might be true. So let's check and src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp line 920: > 918: // } else if (THREAD->is_lock_owned((address)displaced_header)) > 919: // // Simple recursive case. > 920: // monitor->lock()->set_displaced_header(null); Suggestion: // monitor->lock()->set_displaced_header(nullptr); src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp line 981: > 979: // } else if (THREAD->is_lock_owned((address)displaced_header)) > 980: // // Simple recursive case. > 981: // monitor->lock()->set_displaced_header(null); Suggestion: // monitor->lock()->set_displaced_header(nullptr); src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp line 1031: > 1029: // template code: > 1030: // > 1031: // if ((displaced_header = monitor->displaced_header()) == null) { Suggestion: // if ((displaced_header = monitor->displaced_header()) == nullptr) { src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp line 1033: > 1031: // if ((displaced_header = monitor->displaced_header()) == null) { > 1032: // // Recursive unlock. Mark the monitor unlocked by setting the object field to null. > 1033: // monitor->set_obj(null); Suggestion: // monitor->set_obj(nullptr); src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp line 1036: > 1034: // } else if (Atomic::cmpxchg(obj->mark_addr(), monitor, displaced_header) == monitor) { > 1035: // // We swapped the unlocked mark in displaced_header into the object's mark word. > 1036: // monitor->set_obj(null); Suggestion: // monitor->set_obj(nullptr); src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp line 1062: > 1060: // } else if (Atomic::cmpxchg(obj->mark_addr(), monitor, displaced_header) == monitor) { > 1061: // // We swapped the unlocked mark in displaced_header into the object's mark word. > 1062: // monitor->set_obj(null); Suggestion: // monitor->set_obj(nullptr); src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp line 1097: > 1095: b(done); // Monitor register may be overwritten! Runtime has already freed the slot. > 1096: > 1097: // Exchange worked, do monitor->set_obj(null); Suggestion: // Exchange worked, do monitor->set_obj(nullptr); src/hotspot/cpu/ppc/interpreterRT_ppc.cpp line 102: > 100: > 101: // The handle for a receiver will never be null. > 102: bool do_nullptr_check = offset() != 0 || is_static(); Suggestion: bool do_null_check = offset() != 0 || is_static(); src/hotspot/cpu/ppc/interpreterRT_ppc.cpp line 105: > 103: > 104: Label do_null; > 105: if (do_nullptr_check) { Suggestion: if (do_null_check) { src/hotspot/cpu/ppc/stubGenerator_ppc.cpp line 2421: > 2419: BLOCK_COMMENT("arraycopy initial argument checks"); > 2420: > 2421: __ cmpdi(CCR1, src, 0); // if (src == null) return -1; Suggestion: __ cmpdi(CCR1, src, 0); // if (src == nullptr) return -1; src/hotspot/cpu/ppc/stubGenerator_ppc.cpp line 2423: > 2421: __ cmpdi(CCR1, src, 0); // if (src == null) return -1; > 2422: __ extsw_(src_pos, src_pos); // if (src_pos < 0) return -1; > 2423: __ cmpdi(CCR5, dst, 0); // if (dst == null) return -1; Suggestion: __ cmpdi(CCR5, dst, 0); // if (dst == nullptr) return -1; src/hotspot/cpu/ppc/templateInterpreterGenerator_ppc.cpp line 525: > 523: __ ld(R3_RET, Interpreter::stackElementSize, R15_esp); // get receiver > 524: > 525: // Check if receiver == null and go the slow path. Suggestion: // Check if receiver == nullptr and go the slow path. ------------- Changes requested by rrich (Reviewer). PR: https://git.openjdk.org/jdk/pull/12323 From jsjolen at openjdk.org Wed Mar 15 13:13:42 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 15 Mar 2023 13:13:42 GMT Subject: RFR: JDK-8301495: Replace NULL with nullptr in cpu/ppc In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:39:48 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/ppc. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! Found a few minor issues. src/hotspot/cpu/ppc/frame_ppc.cpp line 114: > 112: > 113: // At this point, there still is a chance that fp_safe is false. > 114: // In particular, (fp == nullptr) might be true. So let's check and "fp might be null" src/hotspot/cpu/ppc/macroAssembler_ppc.cpp line 1819: > 1817: } > 1818: > 1819: // for (scan = klass->itable(); scan->interface() != null; scan += scan_step) { nullptr src/hotspot/cpu/ppc/macroAssembler_ppc.cpp line 2710: > 2708: // Handle existing monitor. > 2709: bind(object_has_monitor); > 2710: // The object's monitor m is unlocked iff m->owner == null, is null src/hotspot/os/posix/os_posix.cpp line 293: > 291: ((MemTracker ::tracking_level() == NMT_detail) > 292: ? NativeCallStack(1) > 293: : NativeCallStack(NativeCallStack ::FakeMarker ::its_fake))); Accidentally committed this, will revert to original. ------------- PR: https://git.openjdk.org/jdk/pull/12323 From jsjolen at openjdk.org Wed Mar 15 13:13:43 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 15 Mar 2023 13:13:43 GMT Subject: RFR: JDK-8301495: Replace NULL with nullptr in cpu/ppc In-Reply-To: References: Message-ID: On Wed, 1 Feb 2023 08:37:33 GMT, Richard Reingruber wrote: >> Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/ppc. Unfortunately the script that does the change isn't perfect, and so we >> need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. >> >> Here are some typical things to look out for: >> >> 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). >> 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. >> 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. >> >> An example of this: >> >> ```c++ >> // This function returns null >> void* ret_null(); >> // This function returns true if *x == nullptr >> bool is_nullptr(void** x); >> >> >> Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. >> >> Thanks! > > Looks good, just a few changes I'd like to suggest. > > Thanks, Richard. Thanks @reinrich for your review. I've fixed the issues you found and fixed some that I found also, we will see what GHA says about this commit. ------------- PR: https://git.openjdk.org/jdk/pull/12323 From sjohanss at openjdk.org Wed Mar 15 13:23:25 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Wed, 15 Mar 2023 13:23:25 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects [v2] In-Reply-To: References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: On Mon, 6 Mar 2023 10:22:27 GMT, Ivan Walulya wrote: >> Hi All, >> >> Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. >> >> Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. >> >> We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. >> >> Testing: Tier 1-3 > > Ivan Walulya has updated the pull request incrementally with two additional commits since the last revision: > > - Refactor resetting humongous metadata > - Thomas review Overall it looks good, just a couple of small comments below. Many thanks for taking on this change. src/hotspot/share/gc/g1/g1CollectedHeap.hpp line 609: > 607: inline void register_humongous_candidate_region_with_region_attr(uint index); > 608: > 609: void reset_humongous_metadata(HeapRegion* first_hr, I would go with just `set_humongous_metadata(...)` src/hotspot/share/gc/g1/g1FullGCCompactionPoint.hpp line 47: > 45: void switch_region(); > 46: HeapRegion* next_region(); > 47: Pair find_contiguous_before(HeapRegion* hr, uint num_regions); I think we can skip returning a `Pair` here. The second value is only used to check if we found any range and instead we could use something like `UINT_MAX` or the max number of regions to mark that. ------------- Changes requested by sjohanss (Reviewer). PR: https://git.openjdk.org/jdk/pull/12830 From kvn at openjdk.org Wed Mar 15 13:51:30 2023 From: kvn at openjdk.org (Vladimir Kozlov) Date: Wed, 15 Mar 2023 13:51:30 GMT Subject: RFR: JDK-8301498: Replace NULL with nullptr in cpu/x86 [v3] In-Reply-To: References: Message-ID: On Wed, 15 Mar 2023 11:09:25 GMT, Johan Sj?len wrote: >> Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we >> need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. >> >> Here are some typical things to look out for: >> >> 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). >> 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. >> 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. >> >> An example of this: >> >> ```c++ >> // This function returns null >> void* ret_null(); >> // This function returns true if *x == nullptr >> bool is_nullptr(void** x); >> >> >> Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. >> >> Thanks! > > Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Fix vnkozlov's suggestions > - Merge remote-tracking branch 'origin/master' into JDK-8301498 > - Some more fixes > - Fixes > - Replace NULL with nullptr in cpu/x86 Looks good. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.org/jdk/pull/12326 From iwalulya at openjdk.org Wed Mar 15 17:00:38 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Wed, 15 Mar 2023 17:00:38 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects [v3] In-Reply-To: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: > Hi All, > > Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. > > Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. > > We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. > > Testing: Tier 1-3 Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: StefanJ review ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12830/files - new: https://git.openjdk.org/jdk/pull/12830/files/6f8dd514..ac6bb065 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12830&range=02 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12830&range=01-02 Stats: 23 lines in 5 files changed: 0 ins; 2 del; 21 mod Patch: https://git.openjdk.org/jdk/pull/12830.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12830/head:pull/12830 PR: https://git.openjdk.org/jdk/pull/12830 From jvernee at openjdk.org Wed Mar 15 17:17:51 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Wed, 15 Mar 2023 17:17:51 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v16] In-Reply-To: References: Message-ID: On Tue, 14 Mar 2023 22:30:22 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: > > Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. src/java.base/share/classes/jdk/internal/foreign/abi/ppc64/TypeClass.java line 2: > 1: /* > 2: * Copyright (c) 2022, 2023 Oracle and/or its affiliates. All rights reserved. The copyright header here is missing a comma after the second year: Suggestion: * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From jvernee at openjdk.org Wed Mar 15 17:27:32 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Wed, 15 Mar 2023 17:27:32 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v16] In-Reply-To: References: Message-ID: On Tue, 14 Mar 2023 22:52:44 GMT, Martin Doerr wrote: > Btw. the new cases in which we use int and short accesses when byteWidth is not a power of 2 are never unaligned AFAICS. I guess _UNALIGNED is unnecessary in the JAVA_INT_UNALIGNED and JAVA_SHORT_UNALIGNED. They are always aligned wrt. to their size. They are not necessarily aligned, for instance of the struct we are given is not aligned itself (at least in the reading case). Though we currently reject that case in AbstractLinker, it is something we might want to allow, and we have been looking at that as of late, for instance to pass packed structs to native code. FWIW, using the unaligned variant effectively just turns off the alignment checks we do. So we have less safety, in theory. There should be no performance difference though just from using the _UNALIGNED layout. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Wed Mar 15 18:53:08 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Wed, 15 Mar 2023 18:53:08 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v17] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Fix Copyright format. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/9173af20..5320f895 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=16 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=15-16 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From jvernee at openjdk.org Wed Mar 15 19:12:27 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Wed, 15 Mar 2023 19:12:27 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v17] In-Reply-To: References: Message-ID: On Wed, 15 Mar 2023 18:53:08 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: > > Fix Copyright format. Marked as reviewed by jvernee (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/12708 From jvernee at openjdk.org Wed Mar 15 20:55:31 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Wed, 15 Mar 2023 20:55:31 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v16] In-Reply-To: References: Message-ID: On Tue, 14 Mar 2023 22:52:44 GMT, Martin Doerr wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. > > Btw. the new cases in which we use int and short accesses when byteWidth is not a power of 2 are never unaligned AFAICS. I guess _UNALIGNED is unnecessary in the JAVA_INT_UNALIGNED and JAVA_SHORT_UNALIGNED. They are always aligned wrt. to their size. @TheRealMDoerr I've approved the PR. I suggest for a second reviewer to try and get someone who knows the PPC port. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From wkemper at openjdk.org Wed Mar 15 23:20:09 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 15 Mar 2023 23:20:09 GMT Subject: RFR: Fix off-by-one error when handling pinned regions in a mixed collection Message-ID: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> This fixes an off-by-one error that manifested when the first mixed collection candidate was pinned. ------------- Commit messages: - Stronger unit test assertions - Update comments and assertion checks - Fix off-by-one error when all regions are pinned - Remove unfinished unit test - Merge branch 'shenandoah-master' into fix-old-pinned - WIP: Fix off by one error, improve unit test Changes: https://git.openjdk.org/shenandoah/pull/226/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=226&range=00 Stats: 97 lines in 2 files changed: 45 ins; 4 del; 48 mod Patch: https://git.openjdk.org/shenandoah/pull/226.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/226/head:pull/226 PR: https://git.openjdk.org/shenandoah/pull/226 From kdnilsen at openjdk.org Thu Mar 16 00:57:14 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 16 Mar 2023 00:57:14 GMT Subject: RFR: Fix off-by-one error when handling pinned regions in a mixed collection In-Reply-To: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> References: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> Message-ID: On Wed, 15 Mar 2023 23:13:38 GMT, William Kemper wrote: > This fixes an off-by-one error that manifested when the first mixed collection candidate was pinned. src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 156: > 154: > 155: // Find pinned regions to the left and move their pointer into a slot > 156: // that was pointing at a region that has been added to the cset. We (maybe augment this comment: "or was holding another pinned region that has already been copied to an unused slot found further to the right of its original position" src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 166: > 164: RegionData& skipped = _region_data[search]; > 165: if (skipped._region->is_pinned()) { > 166: RegionData& added_to_cset = _region_data[write_index]; And this variable might be better named "unused_available_slot" rather than "added_to_cset". In the case that we are overwriting a pinned region pointer, it was not "added_to_cset". src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 186: > 184: // | Start index maintains the pointer to the beginning of the regions we scanned on this cycle. > 185: #ifdef ASSERT > 186: for (size_t check = write_index; check != _start_candidate; --check) { I see that _start_candidate is initialized to zero when ShenandoahOldHeuristics is instantiated. What I do not understand is how _start_candidate is reset to zero each time we rebuild _region_data with a new set of mixed evacuation candidates. Can you provide a comment to explain this? ------------- PR: https://git.openjdk.org/shenandoah/pull/226 From wkemper at openjdk.org Thu Mar 16 03:27:53 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 16 Mar 2023 03:27:53 GMT Subject: RFR: Fix off-by-one error when handling pinned regions in a mixed collection [v2] In-Reply-To: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> References: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> Message-ID: > This fixes an off-by-one error that manifested when the first mixed collection candidate was pinned. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Remove over aggressive asserts, improve readability ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/226/files - new: https://git.openjdk.org/shenandoah/pull/226/files/f6dad77b..cf7040fd Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=226&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=226&range=00-01 Stats: 28 lines in 2 files changed: 1 ins; 22 del; 5 mod Patch: https://git.openjdk.org/shenandoah/pull/226.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/226/head:pull/226 PR: https://git.openjdk.org/shenandoah/pull/226 From wkemper at openjdk.org Thu Mar 16 03:27:54 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 16 Mar 2023 03:27:54 GMT Subject: RFR: Fix off-by-one error when handling pinned regions in a mixed collection [v2] In-Reply-To: References: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> Message-ID: On Wed, 15 Mar 2023 23:39:50 GMT, Kelvin Nilsen wrote: >> William Kemper has updated the pull request incrementally with one additional commit since the last revision: >> >> Remove over aggressive asserts, improve readability > > src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 156: > >> 154: >> 155: // Find pinned regions to the left and move their pointer into a slot >> 156: // that was pointing at a region that has been added to the cset. We > > (maybe augment this comment: "or was holding another pinned region that has already been copied to an unused slot found further to the right of its original position" Done. > src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 166: > >> 164: RegionData& skipped = _region_data[search]; >> 165: if (skipped._region->is_pinned()) { >> 166: RegionData& added_to_cset = _region_data[write_index]; > > And this variable might be better named "unused_available_slot" rather than "added_to_cset". In the case that we are overwriting a pinned region pointer, it was not "added_to_cset". Yes, that's a good point. I shall also remove the assert for the same reason (it may encounter a slot for a pinned region that has been moved up and such a region should _not_ be in the cset). > src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 186: > >> 184: // | Start index maintains the pointer to the beginning of the regions we scanned on this cycle. >> 185: #ifdef ASSERT >> 186: for (size_t check = write_index; check != _start_candidate; --check) { > > I see that _start_candidate is initialized to zero when ShenandoahOldHeuristics is instantiated. What I do not understand is how _start_candidate is reset to zero each time we rebuild _region_data with a new set of mixed evacuation candidates. Can you provide a comment to explain this? That was a bug. I've removed this assertion because we may encounter a slot that refers to a pinned region that was moved up and that should not be in the cset. ------------- PR: https://git.openjdk.org/shenandoah/pull/226 From dholmes at openjdk.org Thu Mar 16 07:26:27 2023 From: dholmes at openjdk.org (David Holmes) Date: Thu, 16 Mar 2023 07:26:27 GMT Subject: RFR: JDK-8301498: Replace NULL with nullptr in cpu/x86 [v3] In-Reply-To: References: Message-ID: On Wed, 15 Mar 2023 11:09:25 GMT, Johan Sj?len wrote: >> Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we >> need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. >> >> Here are some typical things to look out for: >> >> 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). >> 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. >> 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. >> >> An example of this: >> >> ```c++ >> // This function returns null >> void* ret_null(); >> // This function returns true if *x == nullptr >> bool is_nullptr(void** x); >> >> >> Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. >> >> Thanks! > > Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: > > - Fix vnkozlov's suggestions > - Merge remote-tracking branch 'origin/master' into JDK-8301498 > - Some more fixes > - Fixes > - Replace NULL with nullptr in cpu/x86 Still good. Thanks. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.org/jdk/pull/12326 From mdoerr at openjdk.org Thu Mar 16 08:36:39 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 16 Mar 2023 08:36:39 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v16] In-Reply-To: References: Message-ID: On Tue, 14 Mar 2023 22:52:44 GMT, Martin Doerr wrote: >> Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. > > Btw. the new cases in which we use int and short accesses when byteWidth is not a power of 2 are never unaligned AFAICS. I guess _UNALIGNED is unnecessary in the JAVA_INT_UNALIGNED and JAVA_SHORT_UNALIGNED. They are always aligned wrt. to their size. > @TheRealMDoerr I've approved the PR. I suggest for a second reviewer to try and get someone who knows the PPC port. Thanks for the review! I guess I should also wait for your 2 PRs to be integrated, so I can merge and adapt. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From jvernee at openjdk.org Thu Mar 16 12:37:28 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Thu, 16 Mar 2023 12:37:28 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v17] In-Reply-To: References: Message-ID: On Wed, 15 Mar 2023 18:53:08 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: > > Fix Copyright format. Ok, I've gone ahead and integrated: https://github.com/openjdk/jdk/pull/12883 & https://github.com/openjdk/jdk/pull/12908 That's all I had at the moment. I think the rest will come with the JEP PR. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Thu Mar 16 13:15:18 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 16 Mar 2023 13:15:18 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v18] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 18 commits: - Merge branch 'master' into PPC64_Panama - Fix Copyright format. - Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. - Allow TestHFA to run on musl. Add Upcalls. - Introduce ABIv2CallArranger for linux ppc64le. - Remove LinuxPPC64CallArranger.java because it doesn't contain anything. - The merge change has messed up some includes in the tests. Revert. - Remove STRUCT_REFERENCE which was incorrectly taken from aarch64. Pass size to bufferLoad/Store. Enable TestNested.java. - Merge remote-tracking branch 'origin' into PPC64_Panama - Handle HFA corner cases with overlapping registers in Java. - ... and 8 more: https://git.openjdk.org/jdk/compare/7dbab81d...944742b5 ------------- Changes: https://git.openjdk.org/jdk/pull/12708/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=17 Stats: 2434 lines in 61 files changed: 2324 ins; 1 del; 109 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Thu Mar 16 13:26:20 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 16 Mar 2023 13:26:20 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v19] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 19 commits: - Merge branch 'openjdk:master' into PPC64_Panama - Merge branch 'master' into PPC64_Panama - Fix Copyright format. - Fix storing 32 bit integers into Java frames. Enable TestArrayStructs. - Allow TestHFA to run on musl. Add Upcalls. - Introduce ABIv2CallArranger for linux ppc64le. - Remove LinuxPPC64CallArranger.java because it doesn't contain anything. - The merge change has messed up some includes in the tests. Revert. - Remove STRUCT_REFERENCE which was incorrectly taken from aarch64. Pass size to bufferLoad/Store. Enable TestNested.java. - Merge remote-tracking branch 'origin' into PPC64_Panama - ... and 9 more: https://git.openjdk.org/jdk/compare/f6291520...2f2f41be ------------- Changes: https://git.openjdk.org/jdk/pull/12708/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=18 Stats: 2434 lines in 61 files changed: 2324 ins; 1 del; 109 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From sjohanss at openjdk.org Thu Mar 16 13:37:21 2023 From: sjohanss at openjdk.org (Stefan Johansson) Date: Thu, 16 Mar 2023 13:37:21 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects [v3] In-Reply-To: References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: On Wed, 15 Mar 2023 17:00:38 GMT, Ivan Walulya wrote: >> Hi All, >> >> Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. >> >> Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. >> >> We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. >> >> Testing: Tier 1-3 > > Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: > > StefanJ review Looks good! ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.org/jdk/pull/12830 From mdoerr at openjdk.org Thu Mar 16 14:17:12 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 16 Mar 2023 14:17:12 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v20] In-Reply-To: References: Message-ID: <6zPly5eBW4Y9v5qD0VKBSigBod2OJeqKoymR_TEogO4=.88f854ce-6a35-4022-a363-1e617bbbd823@github.com> > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with two additional commits since the last revision: - Adaptation for JDK-8303022. - Adaptation for JDK-8303684. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/2f2f41be..a21f6cfb Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=19 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=18-19 Stats: 23 lines in 5 files changed: 9 ins; 3 del; 11 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From iwalulya at openjdk.org Thu Mar 16 14:19:36 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Thu, 16 Mar 2023 14:19:36 GMT Subject: RFR: 8191565: Last-ditch Full GC should also move humongous objects [v3] In-Reply-To: References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: On Thu, 16 Mar 2023 13:34:55 GMT, Stefan Johansson wrote: >> Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: >> >> StefanJ review > > Looks good! Thanks @kstefanj and @tschatzl for the reviews! ------------- PR: https://git.openjdk.org/jdk/pull/12830 From iwalulya at openjdk.org Thu Mar 16 14:19:38 2023 From: iwalulya at openjdk.org (Ivan Walulya) Date: Thu, 16 Mar 2023 14:19:38 GMT Subject: Integrated: 8191565: Last-ditch Full GC should also move humongous objects In-Reply-To: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> References: <58l059EvQI6HNQyjUYSGYEWt6x-c1yvtmfX1QWfinH8=.87517ba1-ec81-4b9f-a41b-b05c8d33cf3d@github.com> Message-ID: On Thu, 2 Mar 2023 13:48:10 GMT, Ivan Walulya wrote: > Hi All, > > Please review this change to move humongous regions during the Last-Ditch full gc ( on `do_maximal_compaction`). This change will enable G1 to avoid encountering Out-Of-Memory errors that may occur due to the fragmentation of memory regions caused by the allocation of large memory blocks. > > Here's how it works: At the end of `phase2_prepare_compaction`, G1 performs a serial compaction process for regular objects, which results in the heap being divided into two parts. The first part is a densely populated prefix that contains all the regular objects that have been moved. The second part consists of the remaining heap space, which may contain free regions, uncommitted regions, and regions that are not compacting. By moving/compacting the humongous objects in the second part of the heap closer to the dense prefix, G1 reduces the region fragmentation and avoids running into OOM errors. > > We have enabled for G1 the Jtreg test that was previously used only for Shenandoah to test such workload. > > Testing: Tier 1-3 This pull request has now been integrated. Changeset: 96889bf3 Author: Ivan Walulya URL: https://git.openjdk.org/jdk/commit/96889bf3e4f36fa7f9e9b9989a1bc3ac4719bfeb Stats: 401 lines in 15 files changed: 322 ins; 47 del; 32 mod 8191565: Last-ditch Full GC should also move humongous objects Reviewed-by: tschatzl, sjohanss ------------- PR: https://git.openjdk.org/jdk/pull/12830 From mdoerr at openjdk.org Thu Mar 16 14:42:10 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 16 Mar 2023 14:42:10 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v21] In-Reply-To: References: Message-ID: > Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". > > This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). > > Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. > > There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) > > The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. > > I had to make changes to shared code and code for other platforms: > 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: > - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. > - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. > - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! > 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: Move ABIv2CallArranger out of linux subdirectory. ABIv1/2 does match the AIX/linux separation. ------------- Changes: - all: https://git.openjdk.org/jdk/pull/12708/files - new: https://git.openjdk.org/jdk/pull/12708/files/a21f6cfb..4666aa22 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=20 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=12708&range=19-20 Stats: 81 lines in 3 files changed: 40 ins; 40 del; 1 mod Patch: https://git.openjdk.org/jdk/pull/12708.diff Fetch: git fetch https://git.openjdk.org/jdk pull/12708/head:pull/12708 PR: https://git.openjdk.org/jdk/pull/12708 From mdoerr at openjdk.org Thu Mar 16 14:42:12 2023 From: mdoerr at openjdk.org (Martin Doerr) Date: Thu, 16 Mar 2023 14:42:12 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v20] In-Reply-To: <6zPly5eBW4Y9v5qD0VKBSigBod2OJeqKoymR_TEogO4=.88f854ce-6a35-4022-a363-1e617bbbd823@github.com> References: <6zPly5eBW4Y9v5qD0VKBSigBod2OJeqKoymR_TEogO4=.88f854ce-6a35-4022-a363-1e617bbbd823@github.com> Message-ID: <2ly6-8MdohRIteqaZsBXJd5R6FxhhmsAv4pwdIaZPW4=.77a93182-17bc-4149-99fb-2c6064c297eb@github.com> On Thu, 16 Mar 2023 14:17:12 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with two additional commits since the last revision: > > - Adaptation for JDK-8303022. > - Adaptation for JDK-8303684. Rebasing + minor cleanup done. Tests are passing. ------------- PR: https://git.openjdk.org/jdk/pull/12708 From kdnilsen at openjdk.org Thu Mar 16 15:16:13 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 16 Mar 2023 15:16:13 GMT Subject: RFR: Fix off-by-one error when handling pinned regions in a mixed collection [v2] In-Reply-To: References: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> Message-ID: On Thu, 16 Mar 2023 03:27:53 GMT, William Kemper wrote: >> This fixes an off-by-one error that manifested when the first mixed collection candidate was pinned. > > William Kemper has updated the pull request incrementally with one additional commit since the last revision: > > Remove over aggressive asserts, improve readability Thanks for your quick action in this one. Looks good. ------------- Marked as reviewed by kdnilsen (Committer). PR: https://git.openjdk.org/shenandoah/pull/226 From ysr at openjdk.org Thu Mar 16 21:48:53 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 16 Mar 2023 21:48:53 GMT Subject: RFR: Fix off-by-one error when handling pinned regions in a mixed collection [v2] In-Reply-To: References: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> Message-ID: On Thu, 16 Mar 2023 03:27:53 GMT, William Kemper wrote: >> This fixes an off-by-one error that manifested when the first mixed collection candidate was pinned. > > William Kemper has updated the pull request incrementally with one additional commit since the last revision: > > Remove over aggressive asserts, improve readability Marked as reviewed by ysr (Author). ------------- PR: https://git.openjdk.org/shenandoah/pull/226 From jvernee at openjdk.org Fri Mar 17 00:10:31 2023 From: jvernee at openjdk.org (Jorn Vernee) Date: Fri, 17 Mar 2023 00:10:31 GMT Subject: RFR: 8303040: linux PPC64le: Implementation of Foreign Function & Memory API (Preview) [v21] In-Reply-To: References: Message-ID: <4-GOVkzGTUqypflffoXoeF9zAlBleE_1zxV3LfcVHow=.afc0426b-2b97-4b02-9ecc-e2d11a1f8a7e@github.com> On Thu, 16 Mar 2023 14:42:10 GMT, Martin Doerr wrote: >> Implementation of "Foreign Function & Memory API" for linux on Power (Little Endian) according to "Power Architecture 64-Bit ELF V2 ABI Specification". >> >> This PR does not include code for VaList support because it's supposed to get removed by [JDK-8299736](https://bugs.openjdk.org/browse/JDK-8299736). I've kept the related tests disabled for this platform and throw an exception instead. Note that the ABI doesn't precisely specify variable argument lists. Instead, it refers to `` (2.2.4 Variable Argument Lists). >> >> Big Endian support is implemented to some extend, but not complete. E.g. structs with size not divisible by 8 are not passed correctly (see `useABIv2` in CallArranger.java). Big Endian is excluded by selecting `ARCH.equals("ppc64le")` (CABI.java) only. >> >> There's another limitation: This PR only accepts structures with size divisible by 4. (An `IllegalArgumentException` gets thrown otherwise.) I think arbitrary sizes are not usable on other platforms, either, because `SharedUtils.primitiveCarrierForSize` only accepts powers of 2. Update: Will get addressed separately: [JDK-8303017](https://bugs.openjdk.org/browse/JDK-8303017) >> >> The ABI has some tricky corner cases related to HFA (Homogeneous Float Aggregate). The same argument may need to get passed in both, a FP reg and a GP reg or stack slot (see "no partial DW rule"). This cases are not covered by the existing tests. >> >> I had to make changes to shared code and code for other platforms: >> 1. Pass type information when creating `VMStorage` objects from `VMReg`. This is needed for the following reasons: >> - PPC64 ABI requires integer types to get extended to 64 bit (also see CCallingConventionRequiresIntsAsLongs in existing hotspot code). We need to know the type or at least the bit width for that. >> - Floating point load / store instructions need the correct width to select between the correct IEEE 754 formats. The register representation in single FP registers is always IEEE 754 double precision on PPC64. >> - Big Endian also needs usage of the precise size. Storing 8 Bytes and loading 4 Bytes yields different values than on Little Endian! >> 2. It happens that a `NativeMemorySegmentImpl` is used as a raw pointer (with byteSize() == 0) while running TestUpcallScope. Hence, existing size checks don't work (see MemorySegment.java). As a workaround, I'm just skipping the check in this particular case. Please check if this makes sense or if there's a better fix (possibly as separate RFE). Update: This issue is resolved by 2nd commit. > > Martin Doerr has updated the pull request incrementally with one additional commit since the last revision: > > Move ABIv2CallArranger out of linux subdirectory. ABIv1/2 does match the AIX/linux separation. Marked as reviewed by jvernee (Reviewer). ------------- PR: https://git.openjdk.org/jdk/pull/12708 From wkemper at openjdk.org Fri Mar 17 00:15:23 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 17 Mar 2023 00:15:23 GMT Subject: Integrated: Fix off-by-one error when handling pinned regions in a mixed collection In-Reply-To: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> References: <96vllsPlqwPX1L_6uf_8tQILFxPXkUVtLq0c-AX1rK4=.12941580-d454-4ca2-8620-1df06c34f603@github.com> Message-ID: On Wed, 15 Mar 2023 23:13:38 GMT, William Kemper wrote: > This fixes an off-by-one error that manifested when the first mixed collection candidate was pinned. This pull request has now been integrated. Changeset: bfc07d8a Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/bfc07d8ab767a9b3df16fa2813034ee915fc6202 Stats: 109 lines in 3 files changed: 40 ins; 20 del; 49 mod Fix off-by-one error when handling pinned regions in a mixed collection Reviewed-by: kdnilsen, ysr ------------- PR: https://git.openjdk.org/shenandoah/pull/226 From wkemper at openjdk.org Fri Mar 17 17:14:01 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 17 Mar 2023 17:14:01 GMT Subject: RFR: Merge openjdk/jdk:master Message-ID: Merges tag jdk-21+14 ------------- Commit messages: - Merge tag 'jdk-21+14' into merge-jdk-21-14 - 8303022: "assert(allocates2(pc)) failed: not in CodeBuffer memory" When linking downcall handle - 8304264: Debug messages always show up for NativeGSS - 8304287: Problemlist java/net/SocketOption/OptionsTest.java - 8304267: JDK-8303415 missed change in Zero Interpreter - 8284047: Harmonize/Standardize the SSLSocket/SSLEngine/SSLSocketSSLEngine test templates - 8300317: vmTestbase/nsk/stress/strace/strace* tests fail with "ERROR: wrong lengths of stack traces" - 8302659: Modernize Windows native code for NetworkInterface - 8298935: fix independence bug in create_pack logic in SuperWord::find_adjacent_refs - 8301308: Remove version conditionalization for gcc/clang PRAGMA_DIAG_PUSH/POP - ... and 93 more: https://git.openjdk.org/shenandoah/compare/b909ec39...98dbde54 The webrevs contain the adjustments done while merging with regards to each parent branch: - master: https://webrevs.openjdk.org/?repo=shenandoah&pr=227&range=00.0 - openjdk/jdk:master: https://webrevs.openjdk.org/?repo=shenandoah&pr=227&range=00.1 Changes: https://git.openjdk.org/shenandoah/pull/227/files Stats: 135402 lines in 1145 files changed: 99692 ins; 21378 del; 14332 mod Patch: https://git.openjdk.org/shenandoah/pull/227.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/227/head:pull/227 PR: https://git.openjdk.org/shenandoah/pull/227 From wkemper at openjdk.org Fri Mar 17 20:44:35 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 17 Mar 2023 20:44:35 GMT Subject: RFR: Merge openjdk/jdk:master [v2] In-Reply-To: References: Message-ID: > Merges tag jdk-21+14 William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 234 commits: - Merge tag 'jdk-21+14' into merge-jdk-21-14 Added tag jdk-21+14 for changeset 2b81faeb - Merge openjdk/jdk:master - Fix incorrect assert logic Reviewed-by: kdnilsen - Use relevant variables in log messages, remove unused import Reviewed-by: ysr, kdnilsen - Performance improvements for non-generational modes Reviewed-by: kdnilsen - Merge openjdk/jdk:master - Merge openjdk/jdk:master - Remove unused visualizer option Reviewed-by: kdnilsen - Merge openjdk/jdk:master Reviewed-by: rkennke - Merge openjdk/jdk:master - ... and 224 more: https://git.openjdk.org/shenandoah/compare/2b81faeb...98dbde54 ------------- Changes: https://git.openjdk.org/shenandoah/pull/227/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=227&range=01 Stats: 16616 lines in 155 files changed: 15271 ins; 511 del; 834 mod Patch: https://git.openjdk.org/shenandoah/pull/227.diff Fetch: git fetch https://git.openjdk.org/shenandoah pull/227/head:pull/227 PR: https://git.openjdk.org/shenandoah/pull/227 From wkemper at openjdk.org Fri Mar 17 20:44:38 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 17 Mar 2023 20:44:38 GMT Subject: Integrated: Merge openjdk/jdk:master In-Reply-To: References: Message-ID: <9kcEzr_qsAsjbhzwlBcKba-9_tSvuJnxtcgTL42dPsE=.67b85c75-f6df-4181-8dcd-84f5b74627a1@github.com> On Fri, 17 Mar 2023 17:05:50 GMT, William Kemper wrote: > Merges tag jdk-21+14 This pull request has now been integrated. Changeset: bfa248a8 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/bfa248a867b1b874e81923a2d7136bf476992f43 Stats: 135402 lines in 1145 files changed: 99692 ins; 21378 del; 14332 mod Merge openjdk/jdk:master ------------- PR: https://git.openjdk.org/shenandoah/pull/227 From wkemper at openjdk.org Mon Mar 20 17:29:29 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 20 Mar 2023 17:29:29 GMT Subject: RFR: Do not visit gc threads during shutdown Message-ID: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> Discovered a strange crash during shutdown while working on jtreg tests. ------------- Commit messages: - Do not visit gc threads during shutdown Changes: https://git.openjdk.org/shenandoah/pull/228/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=228&range=00 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Patch: https://git.openjdk.org/shenandoah/pull/228.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/228/head:pull/228 PR: https://git.openjdk.org/shenandoah/pull/228 From kdnilsen at openjdk.org Mon Mar 20 17:50:11 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 20 Mar 2023 17:50:11 GMT Subject: RFR: Do not visit gc threads during shutdown In-Reply-To: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> References: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> Message-ID: On Mon, 20 Mar 2023 17:13:27 GMT, William Kemper wrote: > Discovered a strange crash during shutdown while working on jtreg tests. Thanks for quick solution. ------------- Marked as reviewed by kdnilsen (Committer). PR Review: https://git.openjdk.org/shenandoah/pull/228#pullrequestreview-1349113058 From wkemper at openjdk.org Mon Mar 20 17:55:44 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 20 Mar 2023 17:55:44 GMT Subject: Integrated: Do not visit gc threads during shutdown In-Reply-To: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> References: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> Message-ID: On Mon, 20 Mar 2023 17:13:27 GMT, William Kemper wrote: > Discovered a strange crash during shutdown while working on jtreg tests. This pull request has now been integrated. Changeset: d3679094 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/d3679094907d4b5de326a340d06995d06dd5d9e8 Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod Do not visit gc threads during shutdown Reviewed-by: kdnilsen ------------- PR: https://git.openjdk.org/shenandoah/pull/228 From ysr at openjdk.org Mon Mar 20 19:14:01 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Mon, 20 Mar 2023 19:14:01 GMT Subject: RFR: Do not visit gc threads during shutdown In-Reply-To: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> References: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> Message-ID: On Mon, 20 Mar 2023 17:13:27 GMT, William Kemper wrote: > Discovered a strange crash during shutdown while working on jtreg tests. Changes look reasonable, but would like a couple of clarifications to determine in my mind the robustness of the proposed fix. Thanks! src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1807: > 1805: return; > 1806: } > 1807: Some related questions so I can understand the shutdown sequence a bit better, and what caused the issue that @kdnilsen recorded in the ticket: 1. I suppose we have determined that the shutdown sequence doesn't itself need to iterate over these GC-related threads to do anything. 2. was the original issue because one of the threads gone by the time we started iterating over it with the closure below? 3. what prevents a race between the check above and the exit of the threads subject to iteration below, so that some of the threads may be asynchronously gone between the check above and the iteration over that thread below? Thanks! ------------- PR Review: https://git.openjdk.org/shenandoah/pull/228#pullrequestreview-1349237142 PR Review Comment: https://git.openjdk.org/shenandoah/pull/228#discussion_r1142571950 From wkemper at openjdk.org Mon Mar 20 21:05:22 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 20 Mar 2023 21:05:22 GMT Subject: RFR: Do not visit gc threads during shutdown In-Reply-To: References: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> Message-ID: On Mon, 20 Mar 2023 19:09:46 GMT, Y. Srinivas Ramakrishna wrote: >> Discovered a strange crash during shutdown while working on jtreg tests. > > src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1807: > >> 1805: return; >> 1806: } >> 1807: > > Some related questions so I can understand the shutdown sequence a bit better, and what caused the issue that @kdnilsen recorded in the ticket: > > 1. I suppose we have determined that the shutdown sequence doesn't itself need to iterate over these GC-related threads to do anything. > 2. was the original issue because one of the threads gone by the time we started iterating over it with the closure below? > 3. what prevents a race between the check above and the exit of the threads subject to iteration below, so that some of the threads may be asynchronously gone between the check above and the iteration over that thread below? > > Thanks! 1. The shutdown sequence doesn't need to iterate over the threads. 2. The crashing thread wasn't "gone", but it's termination had been requested. It crashed in a call to get the thread's clock time (pthread_getcpuclockid). The thread's native handle should still have been valid. I didn't fully root cause the crash. 3. In this case, it was the control thread itself that was trying to iterate the threads. The fix here will stop the control thread from starting an iteration of the threads _after_ the shutdown sequence has begun, but it is not so robust to cover a shut down during an iteration of the threads. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/228#discussion_r1142674640 From kdnilsen at openjdk.org Mon Mar 20 23:38:20 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 20 Mar 2023 23:38:20 GMT Subject: RFR: Fix error in alignment of plabs Message-ID: A recent test run reported an assertion failure because of the error corrected in this patch. ------------- Commit messages: - Merge remote-tracking branch 'GitFarmBranch/fix-error-in-alignment-of-plabs' into fix-error-in-alignment-of-plabs - Fix alignment miscalculation for plabs Changes: https://git.openjdk.org/shenandoah/pull/229/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=229&range=00 Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod Patch: https://git.openjdk.org/shenandoah/pull/229.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/229/head:pull/229 PR: https://git.openjdk.org/shenandoah/pull/229 From wkemper at openjdk.org Mon Mar 20 23:46:36 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 20 Mar 2023 23:46:36 GMT Subject: RFR: Fix error in alignment of plabs In-Reply-To: References: Message-ID: On Mon, 20 Mar 2023 23:31:51 GMT, Kelvin Nilsen wrote: > A recent test run reported an assertion failure because of the error corrected in this patch. LGTM ------------- Marked as reviewed by wkemper (Committer). PR Review: https://git.openjdk.org/shenandoah/pull/229#pullrequestreview-1349538980 From kdnilsen at openjdk.org Mon Mar 20 23:50:16 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 20 Mar 2023 23:50:16 GMT Subject: Integrated: Fix error in alignment of plabs In-Reply-To: References: Message-ID: On Mon, 20 Mar 2023 23:31:51 GMT, Kelvin Nilsen wrote: > A recent test run reported an assertion failure because of the error corrected in this patch. This pull request has now been integrated. Changeset: 1b6d6b26 Author: Kelvin Nilsen URL: https://git.openjdk.org/shenandoah/commit/1b6d6b26876f6f456f721c462aab094c57e6ef3a Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod Fix error in alignment of plabs Reviewed-by: wkemper ------------- PR: https://git.openjdk.org/shenandoah/pull/229 From kdnilsen at openjdk.org Mon Mar 20 23:53:14 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 20 Mar 2023 23:53:14 GMT Subject: RFR: Restrict region dump to young Message-ID: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. ------------- Commit messages: - Merge remote-tracking branch 'GitFarmBranch/restrict-region-dump-to-young' into restrict-region-dump-to-young - Only report young regions in age-censue report at end of GC cycle Changes: https://git.openjdk.org/shenandoah/pull/230/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=230&range=00 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/shenandoah/pull/230.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/230/head:pull/230 PR: https://git.openjdk.org/shenandoah/pull/230 From kdnilsen at openjdk.org Mon Mar 20 23:57:15 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Mon, 20 Mar 2023 23:57:15 GMT Subject: RFR: Report young regions only in logged age census at end of each GC In-Reply-To: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> References: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> Message-ID: On Mon, 20 Mar 2023 23:46:27 GMT, Kelvin Nilsen wrote: > We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. Sure. If we want report census of old-gen regions, we should probably report that in a separate table. ------------- PR Comment: https://git.openjdk.org/shenandoah/pull/230#issuecomment-1477098869 From wkemper at openjdk.org Mon Mar 20 23:57:15 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 20 Mar 2023 23:57:15 GMT Subject: RFR: Report young regions only in logged age census at end of each GC In-Reply-To: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> References: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> Message-ID: On Mon, 20 Mar 2023 23:46:27 GMT, Kelvin Nilsen wrote: > We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. I'm okay with this, but I want @ysramakrishna to weigh in on it. He may want to see the distribution of ages in old regions for the automatic tenuring threshold work. ------------- PR Comment: https://git.openjdk.org/shenandoah/pull/230#issuecomment-1477097703 From ysr at openjdk.org Tue Mar 21 00:30:36 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Tue, 21 Mar 2023 00:30:36 GMT Subject: RFR: Report young regions only in logged age census at end of each GC In-Reply-To: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> References: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> Message-ID: On Mon, 20 Mar 2023 23:46:27 GMT, Kelvin Nilsen wrote: > We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. Marked as reviewed by ysr (Author). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/230#pullrequestreview-1349566024 From ysr at openjdk.org Tue Mar 21 00:51:19 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Tue, 21 Mar 2023 00:51:19 GMT Subject: RFR: Do not visit gc threads during shutdown In-Reply-To: References: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> Message-ID: On Mon, 20 Mar 2023 21:02:53 GMT, William Kemper wrote: > 1. The shutdown sequence doesn't need to iterate over the threads. Good! > 2. The crashing thread wasn't "gone", but it's termination had been requested. It crashed in a call to get the thread's clock time (pthread_getcpuclockid). The thread's native handle should still have been valid. I didn't fully root cause the crash. I assume you meant "the _GC thread that the control thread was iterating over_ wasn't gone, but its termination had been requested"? I'd consider that as good as gone, absent any interlocking between the iteration and the termination of the target thread. > 3. In this case, it was the control thread itself that was trying to iterate the threads. The fix here will stop the control thread from starting an iteration of the threads _after_ the shutdown sequence has begun, but it is not so robust to cover a shut down during an iteration of the threads. May be one then needs a handshake/interlock with the control thread during the shutdown sequence if it's the only thread that iterates over these threads. (Or their termination could be delegated to the control thread, which is morally equivalent, and perhaps cleaner.) Is the case then that you have by your change above substantially narrowed the window of vulnerability, but not completely closed it? Or did I misunderstand? ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/228#discussion_r1142803142 From wkemper at openjdk.org Tue Mar 21 00:57:57 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 21 Mar 2023 00:57:57 GMT Subject: RFR: Report young regions only in logged age census at end of each GC In-Reply-To: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> References: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> Message-ID: On Mon, 20 Mar 2023 23:46:27 GMT, Kelvin Nilsen wrote: > We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. Marked as reviewed by wkemper (Committer). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/230#pullrequestreview-1349578926 From wkemper at openjdk.org Tue Mar 21 01:04:41 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 21 Mar 2023 01:04:41 GMT Subject: RFR: Do not visit gc threads during shutdown In-Reply-To: References: <0R70PMjBKAS_dGe62QxbX5uk4mhjoUNKHiF869UFzxE=.09752842-28cd-4fb5-9b3d-d98a83bffa74@github.com> Message-ID: On Tue, 21 Mar 2023 00:49:01 GMT, Y. Srinivas Ramakrishna wrote: >> 1. The shutdown sequence doesn't need to iterate over the threads. >> 2. The crashing thread wasn't "gone", but it's termination had been requested. It crashed in a call to get the thread's clock time (pthread_getcpuclockid). The thread's native handle should still have been valid. I didn't fully root cause the crash. >> 3. In this case, it was the control thread itself that was trying to iterate the threads. The fix here will stop the control thread from starting an iteration of the threads _after_ the shutdown sequence has begun, but it is not so robust to cover a shut down during an iteration of the threads. > >> 1. The shutdown sequence doesn't need to iterate over the threads. > > Good! > >> 2. The crashing thread wasn't "gone", but it's termination had been requested. It crashed in a call to get the thread's clock time (pthread_getcpuclockid). The thread's native handle should still have been valid. I didn't fully root cause the crash. > > I assume you meant "the _GC thread that the control thread was iterating over_ wasn't gone, but its termination had been requested"? I'd consider that as good as gone, absent any interlocking between the iteration and the termination of the target thread. > >> 3. In this case, it was the control thread itself that was trying to iterate the threads. The fix here will stop the control thread from starting an iteration of the threads _after_ the shutdown sequence has begun, but it is not so robust to cover a shut down during an iteration of the threads. > > May be one then needs a handshake/interlock with the control thread during the shutdown sequence if it's the only thread that iterates over these threads. (Or their termination could be delegated to the control thread, which is morally equivalent, and perhaps cleaner.) Is the case then that you have by your change above substantially narrowed the window of vulnerability, but not completely closed it? Or did I misunderstand? The `core` file I had was on `/tmp` and was lost in one of the power outages last week. I meant that the control thread's termination was requested and it was the control thread itself that was iterating the other threads. I don't recall which visited thread caused the crash. As I think on this, I'd like to make another small change to make it more robust. Yes, that's fair. This change narrows the window, but doesn't completely close it. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/228#discussion_r1142807659 From shade at openjdk.org Tue Mar 21 09:26:44 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 21 Mar 2023 09:26:44 GMT Subject: RFR: Report young regions only in logged age census at end of each GC In-Reply-To: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> References: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> Message-ID: On Mon, 20 Mar 2023 23:46:27 GMT, Kelvin Nilsen wrote: > We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. Let's just print "Young regions" and "Old regions" separately? AgeTable young_region_ages(false); AgeTable old_region_ages(false); ShenandoahHeap* heap = ShenandoahHeap::heap(); for (uint i = 0; i < heap->num_regions(); ++i) { ShenandoahHeapRegion* r = heap->get_region(i); if (r->age() > 0 && r->age() < AgeTable::table_size) { if (r->young()) { young_region_ages.add(r->age(), r->get_live_data_words()); } else { old_region_ages.add(r->age(), r->get_live_data_words()); } } } st->print("Young regions: "); young_region_ages.print_on(st, InitialTenuringThreshold); st->cr(); st->print("Old regions: "); old_region_ages.print_on(st, InitialTenuringThreshold); ------------- PR Review: https://git.openjdk.org/shenandoah/pull/230#pullrequestreview-1349996321 From jsjolen at openjdk.org Tue Mar 21 09:56:53 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Tue, 21 Mar 2023 09:56:53 GMT Subject: RFR: JDK-8301493: Replace NULL with nullptr in cpu/aarch64 In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:39:27 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/aarch64. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! Not yet. ------------- PR Comment: https://git.openjdk.org/jdk/pull/12321#issuecomment-1477546797 From jsjolen at openjdk.org Tue Mar 21 10:04:04 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Tue, 21 Mar 2023 10:04:04 GMT Subject: RFR: JDK-8301498: Replace NULL with nullptr in cpu/x86 [v4] In-Reply-To: References: Message-ID: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: - Merge remote-tracking branch 'origin/master' into JDK-8301498 - Fix vnkozlov's suggestions - Merge remote-tracking branch 'origin/master' into JDK-8301498 - Some more fixes - Fixes - Replace NULL with nullptr in cpu/x86 ------------- Changes: https://git.openjdk.org/jdk/pull/12326/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12326&range=03 Stats: 657 lines in 54 files changed: 0 ins; 0 del; 657 mod Patch: https://git.openjdk.org/jdk/pull/12326.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/12326/head:pull/12326 PR: https://git.openjdk.org/jdk/pull/12326 From smonteith at openjdk.org Tue Mar 21 13:34:40 2023 From: smonteith at openjdk.org (Stuart Monteith) Date: Tue, 21 Mar 2023 13:34:40 GMT Subject: RFR: JDK-8301493: Replace NULL with nullptr in cpu/aarch64 In-Reply-To: References: Message-ID: <-ZV05tb2xNWIBcGc7Nj_TZ6qq3BGrsjlKCT48_GTmQU=.6480f4f9-f1a5-47fa-94d9-51d3968ff711@github.com> On Tue, 31 Jan 2023 11:39:27 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/aarch64. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! This looks OK so far. However, is it your intention to also do aarch64.ad? aarch64_ad.m4 and aarch64_vector(.ad|_ad.m4) files look clean. ------------- PR Comment: https://git.openjdk.org/jdk/pull/12321#issuecomment-1477844415 From kdnilsen at openjdk.org Tue Mar 21 14:49:51 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 21 Mar 2023 14:49:51 GMT Subject: RFR: Separate young and old regions in age census at end of each GC [v2] In-Reply-To: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> References: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> Message-ID: > We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Add separate report for old region age table ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/230/files - new: https://git.openjdk.org/shenandoah/pull/230/files/379874e5..eb727d9d Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=230&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=230&range=00-01 Stats: 12 lines in 1 file changed: 7 ins; 0 del; 5 mod Patch: https://git.openjdk.org/shenandoah/pull/230.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/230/head:pull/230 PR: https://git.openjdk.org/shenandoah/pull/230 From kdnilsen at openjdk.org Tue Mar 21 14:49:53 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Tue, 21 Mar 2023 14:49:53 GMT Subject: Integrated: Separate young and old regions in age census at end of each GC In-Reply-To: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> References: <0uiYozIkJRLU7WoLd3A9pz2yu36O5sxFcBdHBD_vc7Q=.f34710dc-48b6-41c9-bd3c-0e9bd43b8361@github.com> Message-ID: On Mon, 20 Mar 2023 23:46:27 GMT, Kelvin Nilsen wrote: > We are primarily interested in the young-gen aged regions. It is confusing for the log report to add the live data pertaining to old-gen regions which happen to have age > 0. This pull request has now been integrated. Changeset: a853a7a4 Author: Kelvin Nilsen URL: https://git.openjdk.org/shenandoah/commit/a853a7a4841fbcebce151e3b0e7081bedc782b56 Stats: 11 lines in 1 file changed: 7 ins; 0 del; 4 mod Separate young and old regions in age census at end of each GC Reviewed-by: ysr, wkemper ------------- PR: https://git.openjdk.org/shenandoah/pull/230 From smonteith at openjdk.org Tue Mar 21 20:47:43 2023 From: smonteith at openjdk.org (Stuart Monteith) Date: Tue, 21 Mar 2023 20:47:43 GMT Subject: RFR: JDK-8301493: Replace NULL with nullptr in cpu/aarch64 In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:39:27 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/aarch64. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! src/hotspot/cpu/aarch64/interpreterRT_aarch64.cpp line 270: > 268: virtual void pass_object() { > 269: intptr_t* addr = single_slot_addr(); > 270: intptr_t value = *addr == 0 ? nullptr : (intptr_t)addr; This doesn't compile - perhaps replace nullptr with zero? Unless casting it is more appropriate. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/12321#discussion_r1143970163 From wkemper at openjdk.org Tue Mar 21 21:54:19 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 21 Mar 2023 21:54:19 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented Message-ID: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> Record that shutdown sequence has begun before we request the regulator thread to stop. ------------- Commit messages: - Put regulator thread stop after we block iteration of gc threads Changes: https://git.openjdk.org/shenandoah/pull/231/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=231&range=00 Stats: 6 lines in 1 file changed: 3 ins; 3 del; 0 mod Patch: https://git.openjdk.org/shenandoah/pull/231.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/231/head:pull/231 PR: https://git.openjdk.org/shenandoah/pull/231 From dholmes at openjdk.org Wed Mar 22 07:33:43 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 22 Mar 2023 07:33:43 GMT Subject: RFR: JDK-8301498: Replace NULL with nullptr in cpu/x86 [v4] In-Reply-To: References: Message-ID: On Tue, 21 Mar 2023 10:04:04 GMT, Johan Sj?len wrote: >> Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we >> need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. >> >> Here are some typical things to look out for: >> >> 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). >> 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. >> 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. >> >> An example of this: >> >> ```c++ >> // This function returns null >> void* ret_null(); >> // This function returns true if *x == nullptr >> bool is_nullptr(void** x); >> >> >> Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. >> >> Thanks! > > Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Merge remote-tracking branch 'origin/master' into JDK-8301498 > - Fix vnkozlov's suggestions > - Merge remote-tracking branch 'origin/master' into JDK-8301498 > - Some more fixes > - Fixes > - Replace NULL with nullptr in cpu/x86 Still good. Thanks. (incremental change looked weird but end result seems fine) ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/12326#pullrequestreview-1351869217 From ysr at openjdk.org Wed Mar 22 08:14:33 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Wed, 22 Mar 2023 08:14:33 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented In-Reply-To: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> Message-ID: On Tue, 21 Mar 2023 21:47:10 GMT, William Kemper wrote: > Record that shutdown sequence has begun before we request the regulator thread to stop. src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2365: > 2363: // Step 0a. Stop requesting collections. > 2364: regulator_thread()->stop(); > 2365: May be rename them Steps 0.a and 0.b in the order in which they are executed? Could you state the advantage of doing steps in this rather than the old order? (I confess I looked at the code but couldn't find an immediate reason why this may be preferable to the previous order, e.g. does it accelerate the shutdown if the actions are in this new order, or something else such as prevention of a potential deadlock or a crash?) ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/231#discussion_r1144372703 From jsjolen at openjdk.org Wed Mar 22 14:22:01 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 22 Mar 2023 14:22:01 GMT Subject: RFR: JDK-8301498: Replace NULL with nullptr in cpu/x86 [v4] In-Reply-To: References: Message-ID: <0A-D-P6RuP8u8dtlXBf5ox_2-15SnHi_ixwC5MZW5OM=.0971e7bf-5380-42ce-a496-eedd8cf2fff8@github.com> On Tue, 21 Mar 2023 10:04:04 GMT, Johan Sj?len wrote: >> Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we >> need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. >> >> Here are some typical things to look out for: >> >> 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). >> 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. >> 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. >> >> An example of this: >> >> ```c++ >> // This function returns null >> void* ret_null(); >> // This function returns true if *x == nullptr >> bool is_nullptr(void** x); >> >> >> Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. >> >> Thanks! > > Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits: > > - Merge remote-tracking branch 'origin/master' into JDK-8301498 > - Fix vnkozlov's suggestions > - Merge remote-tracking branch 'origin/master' into JDK-8301498 > - Some more fixes > - Fixes > - Replace NULL with nullptr in cpu/x86 Thanks. All of tier1 and tier2 is passing. ------------- PR Comment: https://git.openjdk.org/jdk/pull/12326#issuecomment-1479646756 From jsjolen at openjdk.org Wed Mar 22 14:22:02 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Wed, 22 Mar 2023 14:22:02 GMT Subject: Integrated: JDK-8301498: Replace NULL with nullptr in cpu/x86 In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:40:19 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/x86. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! This pull request has now been integrated. Changeset: 4154a980 Author: Johan Sj?len URL: https://git.openjdk.org/jdk/commit/4154a980ca28c1ae56db26e3dce64c07c225de12 Stats: 656 lines in 54 files changed: 0 ins; 0 del; 656 mod 8301498: Replace NULL with nullptr in cpu/x86 Reviewed-by: dholmes, kvn ------------- PR: https://git.openjdk.org/jdk/pull/12326 From wkemper at openjdk.org Wed Mar 22 22:29:30 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 22 Mar 2023 22:29:30 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented In-Reply-To: References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> Message-ID: On Wed, 22 Mar 2023 08:11:56 GMT, Y. Srinivas Ramakrishna wrote: >> Record that shutdown sequence has begun before we request the regulator thread to stop. > > src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2365: > >> 2363: // Step 0a. Stop requesting collections. >> 2364: regulator_thread()->stop(); >> 2365: > > May be rename them Steps 0.a and 0.b in the order in which they are executed? > > Could you state the advantage of doing steps in this rather than the old order? (I confess I looked at the code but couldn't find an immediate reason why this may be preferable to the previous order, e.g. does it accelerate the shutdown if the actions are in this new order, or something else such as prevention of a potential deadlock or a crash?) The previous code would have allowed these events: 1. shutdown thread stops regulator thread 2. control thread begins iteration of gc threads (shutdown not 'recorded' yet) 3. shutdown thread records shutdown (meant to prevent iteration of stopped threads) 4. control thread visits stopped regulator thread (which could be unsafe) This change reduces the chances of the control thread iterating the gc threads after the regulator thread is stopped. Of course, without a robust locking sequence it may _still_ happen, but this code is closer to the intended sequence. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/231#discussion_r1145470359 From wkemper at openjdk.org Wed Mar 22 22:37:16 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 22 Mar 2023 22:37:16 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented [v2] In-Reply-To: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> Message-ID: <0PQpsemxj1G8aMAsmZriXcE8sAwxrQpjL-Eonmuro5A=.28cb0266-36eb-4d22-8b1b-0a95d2a52ede@github.com> > Record that shutdown sequence has begun before we request the regulator thread to stop. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Fix numbering for comments ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/231/files - new: https://git.openjdk.org/shenandoah/pull/231/files/2e174bec..d0138a4c Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=231&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=231&range=00-01 Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.org/shenandoah/pull/231.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/231/head:pull/231 PR: https://git.openjdk.org/shenandoah/pull/231 From ysr at openjdk.org Thu Mar 23 06:29:32 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 23 Mar 2023 06:29:32 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented [v2] In-Reply-To: <0PQpsemxj1G8aMAsmZriXcE8sAwxrQpjL-Eonmuro5A=.28cb0266-36eb-4d22-8b1b-0a95d2a52ede@github.com> References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> <0PQpsemxj1G8aMAsmZriXcE8sAwxrQpjL-Eonmuro5A=.28cb0266-36eb-4d22-8b1b-0a95d2a52ede@github.com> Message-ID: On Wed, 22 Mar 2023 22:37:16 GMT, William Kemper wrote: >> Record that shutdown sequence has begun before we request the regulator thread to stop. > > William Kemper has updated the pull request incrementally with one additional commit since the last revision: > > Fix numbering for comments Marked as reviewed by ysr (Author). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/231#pullrequestreview-1353924426 From ysr at openjdk.org Thu Mar 23 06:29:35 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 23 Mar 2023 06:29:35 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented [v2] In-Reply-To: References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> Message-ID: On Wed, 22 Mar 2023 22:26:38 GMT, William Kemper wrote: >> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2365: >> >>> 2363: // Step 0a. Stop requesting collections. >>> 2364: regulator_thread()->stop(); >>> 2365: >> >> May be rename them Steps 0.a and 0.b in the order in which they are executed? >> >> Could you state the advantage of doing steps in this rather than the old order? (I confess I looked at the code but couldn't find an immediate reason why this may be preferable to the previous order, e.g. does it accelerate the shutdown if the actions are in this new order, or something else such as prevention of a potential deadlock or a crash?) > > The previous code would have allowed these events: > 1. shutdown thread stops regulator thread > 2. control thread begins iteration of gc threads (shutdown not 'recorded' yet) > 3. shutdown thread records shutdown (meant to prevent iteration of stopped threads) > 4. control thread visits stopped regulator thread (which could be unsafe) > > This change reduces the chances of the control thread iterating the gc threads after the regulator thread is stopped. Of course, without a robust locking sequence it may _still_ happen, but this code is closer to the intended sequence. OK that makes sense; thanks! ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/231#discussion_r1145728909 From shade at openjdk.org Thu Mar 23 09:54:39 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 23 Mar 2023 09:54:39 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented [v2] In-Reply-To: <0PQpsemxj1G8aMAsmZriXcE8sAwxrQpjL-Eonmuro5A=.28cb0266-36eb-4d22-8b1b-0a95d2a52ede@github.com> References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> <0PQpsemxj1G8aMAsmZriXcE8sAwxrQpjL-Eonmuro5A=.28cb0266-36eb-4d22-8b1b-0a95d2a52ede@github.com> Message-ID: On Wed, 22 Mar 2023 22:37:16 GMT, William Kemper wrote: >> Record that shutdown sequence has begun before we request the regulator thread to stop. > > William Kemper has updated the pull request incrementally with one additional commit since the last revision: > > Fix numbering for comments Marked as reviewed by shade (Committer). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/231#pullrequestreview-1354217855 From kdnilsen at openjdk.org Thu Mar 23 15:06:00 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 23 Mar 2023 15:06:00 GMT Subject: RFR: Put regulator thread stop after gc thread iteration is prevented [v2] In-Reply-To: <0PQpsemxj1G8aMAsmZriXcE8sAwxrQpjL-Eonmuro5A=.28cb0266-36eb-4d22-8b1b-0a95d2a52ede@github.com> References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> <0PQpsemxj1G8aMAsmZriXcE8sAwxrQpjL-Eonmuro5A=.28cb0266-36eb-4d22-8b1b-0a95d2a52ede@github.com> Message-ID: <4KXo86q_oGA0H2dso8Z6av1XnZYRBkBNUdLNHSYK8Cs=.abd48852-f5ba-4ca7-a4a2-4b120426fe0c@github.com> On Wed, 22 Mar 2023 22:37:16 GMT, William Kemper wrote: >> Record that shutdown sequence has begun before we request the regulator thread to stop. > > William Kemper has updated the pull request incrementally with one additional commit since the last revision: > > Fix numbering for comments Marked as reviewed by kdnilsen (Committer). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/231#pullrequestreview-1354843660 From wkemper at openjdk.org Thu Mar 23 15:55:18 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 23 Mar 2023 15:55:18 GMT Subject: Integrated: Put regulator thread stop after gc thread iteration is prevented In-Reply-To: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> References: <6l1l7_DEg4WFLg36czOLCQoV-Jcg41ug5NcLLAvjjyE=.e74c18bd-acc9-470b-8d7f-11bd7fb163a6@github.com> Message-ID: On Tue, 21 Mar 2023 21:47:10 GMT, William Kemper wrote: > Record that shutdown sequence has begun before we request the regulator thread to stop. This pull request has now been integrated. Changeset: 1a3dcaa6 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/1a3dcaa6b2a6a2e386682defe409657eff6591a4 Stats: 10 lines in 1 file changed: 3 ins; 3 del; 4 mod Put regulator thread stop after gc thread iteration is prevented Reviewed-by: ysr, shade, kdnilsen ------------- PR: https://git.openjdk.org/shenandoah/pull/231 From wkemper at openjdk.org Thu Mar 23 22:55:29 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 23 Mar 2023 22:55:29 GMT Subject: RFR: Lower logging of collection set details to debug and use cset tag Message-ID: This logging was unintentionally left at Info level. ------------- Commit messages: - Set cset info to Debug and use cset tag Changes: https://git.openjdk.org/shenandoah/pull/232/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=232&range=00 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/shenandoah/pull/232.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/232/head:pull/232 PR: https://git.openjdk.org/shenandoah/pull/232 From kdnilsen at openjdk.org Thu Mar 23 22:55:29 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 23 Mar 2023 22:55:29 GMT Subject: RFR: Lower logging of collection set details to debug and use cset tag In-Reply-To: References: Message-ID: On Thu, 23 Mar 2023 22:43:39 GMT, William Kemper wrote: > This logging was unintentionally left at Info level. This looks good. (I did something similar when I merged this into my expand-on-demand.) ------------- Marked as reviewed by kdnilsen (Committer). PR Review: https://git.openjdk.org/shenandoah/pull/232#pullrequestreview-1355771641 From ysr at openjdk.org Thu Mar 23 23:01:21 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 23 Mar 2023 23:01:21 GMT Subject: RFR: Lower logging of collection set details to debug and use cset tag In-Reply-To: References: Message-ID: On Thu, 23 Mar 2023 22:43:39 GMT, William Kemper wrote: > This logging was unintentionally left at Info level. Marked as reviewed by ysr (Author). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/232#pullrequestreview-1355780504 From wkemper at openjdk.org Thu Mar 23 23:22:09 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 23 Mar 2023 23:22:09 GMT Subject: Integrated: Lower logging of collection set details to debug and use cset tag In-Reply-To: References: Message-ID: On Thu, 23 Mar 2023 22:43:39 GMT, William Kemper wrote: > This logging was unintentionally left at Info level. This pull request has now been integrated. Changeset: a5d1a9d8 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/a5d1a9d8ee70828bf5e2ccdc7a3691eb7ae324ea Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Lower logging of collection set details to debug and use cset tag Reviewed-by: kdnilsen, ysr ------------- PR: https://git.openjdk.org/shenandoah/pull/232 From wkemper at openjdk.org Fri Mar 24 19:06:19 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 24 Mar 2023 19:06:19 GMT Subject: RFR: Restrict generational mode to adaptive heuristic Message-ID: This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. ------------- Commit messages: - Merge branch 'shenandoah-master' into restrict-generational-heuristic - WIP: Bug fixes, add generational mode to more tests - WIP: Integrate generational mode with more tests - Restrict young generation adaptive heuristic - WIP: Properly integrate generational mode testing (compiler tests) - WIP: Properly integrate generational mode testing - Do not visit gc threads during shutdown Changes: https://git.openjdk.org/shenandoah/pull/233/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=233&range=00 Stats: 1720 lines in 82 files changed: 1345 ins; 312 del; 63 mod Patch: https://git.openjdk.org/shenandoah/pull/233.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/233/head:pull/233 PR: https://git.openjdk.org/shenandoah/pull/233 From shade at openjdk.org Mon Mar 27 10:12:45 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Mon, 27 Mar 2023 10:12:45 GMT Subject: RFR: Restrict generational mode to adaptive heuristic In-Reply-To: References: Message-ID: On Fri, 24 Mar 2023 18:59:50 GMT, William Kemper wrote: > This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: > * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). > * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. Questions and suggestions below: test/hotspot/jtreg/gc/shenandoah/TestAllocHumongousFragment.java line 61: > 59: * > 60: * @run main/othervm -Xmx1g -Xms1g -Xlog:gc -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -XX:ShenandoahTargetNumRegions=2048 > 61: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=aggressive -XX:ShenandoahGCMode=satb Why add `satb` in this test, but not the others? Other tests seem to just run `aggressive` with default GC mode. test/hotspot/jtreg/gc/shenandoah/TestAllocObjectArrays.java line 121: > 119: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational > 120: * -XX:+ShenandoahOOMDuringEvacALot -XX:+ShenandoahVerify > 121: * TestAllocObjectArrays Please put `-XX:+ShenandoahVerify` on the separate line, like the test config above? test/hotspot/jtreg/gc/shenandoah/TestAllocObjectArrays.java line 167: > 165: * @run main/othervm -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -Xmx1g -Xms1g > 166: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational > 167: * -XX:-UseTLAB -XX:+ShenandoahVerify This `-XX:+ShenandoahVerify` seems excessive. This config is supposed to run without verification. Copy-paste error, it seems. test/hotspot/jtreg/gc/shenandoah/TestParallelRefprocSanity.java line 1: > 1: /* Why is this removed? test/hotspot/jtreg/gc/shenandoah/TestRegionSamplingLogging.java line 31: > 29: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions > 30: * -XX:+ShenandoahRegionSampling -XX:+ShenandoahRegionSampling > 31: * -Xlog:gc+region=trace:region-snapshots-%p.log::filesize=100,filecount=3 So, `trace` is enough here? This looks like something we need to fix upstream, rather than do in a bulk change like this. test/hotspot/jtreg/gc/shenandoah/TestRetainObjects.java line 107: > 105: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions > 106: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational > 107: * -XX:+ShenandoahAllocFailureALot -XX:+ShenandoahVerify I think the intent here is to run without `-XX:+ShenandoahVerify`. test/hotspot/jtreg/gc/shenandoah/TestSieveObjects.java line 107: > 105: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions > 106: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational > 107: * -XX:+ShenandoahAllocFailureALot -XX:+ShenandoahVerify Excessive `-XX:+ShenandoahVerify`? test/hotspot/jtreg/gc/shenandoah/compiler/BarrierInInfiniteLoop.java line 35: > 33: */ > 34: > 35: /** This looks like a compiler regression test, so we don't need to run it in all these different GC modes. test/hotspot/jtreg/gc/shenandoah/compiler/CallMultipleCatchProjs.java line 34: > 32: */ > 33: > 34: /** Again, compiler regression test, probably no reason to run with different GC modes. test/hotspot/jtreg/gc/shenandoah/compiler/FoldIfAfterExpansion.java line 36: > 34: */ > 35: > 36: /** Again, compiler regression test, probably no reason to run with different GC modes. test/hotspot/jtreg/gc/shenandoah/compiler/LRBRightAfterMemBar.java line 35: > 33: */ > 34: > 35: /** Right, `compiler` test again. :) I think most (all?) tests in this folder do not need GC mode variance. test/hotspot/jtreg/gc/shenandoah/compiler/TestClone.java line 208: > 206: */ > 207: > 208: /* This one seems to be the rare exception where GC mode variance is meaningful -- this is not a targeted regression test. test/hotspot/jtreg/gc/shenandoah/generational/TestCLIModeGenerational.java line 34: > 32: * @library /testlibrary /test/lib / > 33: * @build jdk.test.whitebox.WhiteBox > 34: * @run driver jdk.test.lib.helpers.ClassFileInstaller jdk.test.whitebox.WhiteBox Generally, these changes should be done separately, not to clobber the actual "generational"-related test change. It is fine to do them here, if these only touch the files than exist only in GenShen. test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 35: > 33: /** > 34: * @test id=heap > 35: * @summary Test allocation of small object to result OOM, but not to crash JVM The `@summary` in these configs are all the same, copy-paste error. Also, we might want to fix English here a bit: `Test allocation of (small|large|larger-than-heap) object should result in OOM, not in JVM crash`. test/hotspot/jtreg/gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java line 1: > 1: /* Is there a generational variant for this? ------------- PR Review: https://git.openjdk.org/shenandoah/pull/233#pullrequestreview-1358736166 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149058740 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149060202 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149061221 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149065439 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149070147 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149070925 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149071593 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149073540 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149074305 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149074544 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149075454 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149076687 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149077952 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149086518 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149089962 From wkemper at openjdk.org Mon Mar 27 16:37:14 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 27 Mar 2023 16:37:14 GMT Subject: RFR: Restrict generational mode to adaptive heuristic In-Reply-To: References: Message-ID: On Mon, 27 Mar 2023 09:45:05 GMT, Aleksey Shipilev wrote: >> This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: >> * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). >> * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. > > test/hotspot/jtreg/gc/shenandoah/TestAllocObjectArrays.java line 167: > >> 165: * @run main/othervm -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -Xmx1g -Xms1g >> 166: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational >> 167: * -XX:-UseTLAB -XX:+ShenandoahVerify > > This `-XX:+ShenandoahVerify` seems excessive. This config is supposed to run without verification. Copy-paste error, it seems. The first execution in this `no-tlab` configuration runs the default mode and heuristic with verification enabled, so I'm doing the same for the generational mode configuration. > test/hotspot/jtreg/gc/shenandoah/TestParallelRefprocSanity.java line 1: > >> 1: /* > > Why is this removed? After reference processing was made concurrent, Shenandoah no longer reads the `ParallelRefProcEnabled` flag. Reference processing is covered by `TestRefprocSanity.java`. > test/hotspot/jtreg/gc/shenandoah/TestRegionSamplingLogging.java line 31: > >> 29: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions >> 30: * -XX:+ShenandoahRegionSampling -XX:+ShenandoahRegionSampling >> 31: * -Xlog:gc+region=trace:region-snapshots-%p.log::filesize=100,filecount=3 > > So, `trace` is enough here? This looks like something we need to fix upstream, rather than do in a bulk change like this. This feature of writing the region sampling data out to logs is only on the generational branch at this time. It wouldn't be hard to pull it out into a separate upstream PR. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149505379 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149507947 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149509452 From wkemper at openjdk.org Mon Mar 27 16:46:50 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 27 Mar 2023 16:46:50 GMT Subject: RFR: Restrict generational mode to adaptive heuristic In-Reply-To: References: Message-ID: On Mon, 27 Mar 2023 09:52:50 GMT, Aleksey Shipilev wrote: >> This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: >> * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). >> * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. > > test/hotspot/jtreg/gc/shenandoah/TestRetainObjects.java line 107: > >> 105: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions >> 106: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational >> 107: * -XX:+ShenandoahAllocFailureALot -XX:+ShenandoahVerify > > I think the intent here is to run without `-XX:+ShenandoahVerify`. This configuration runs with `ShenandoahAllocFailureALot`, the preceeding configuration runs with `ShenandoahOOMDuringEvacALot`. I enabled verification for both to make the tests more rigorous. Verification may find errors in card marking that wouldn't necessarily fail the test otherwise. > test/hotspot/jtreg/gc/shenandoah/TestSieveObjects.java line 107: > >> 105: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions >> 106: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational >> 107: * -XX:+ShenandoahAllocFailureALot -XX:+ShenandoahVerify > > Excessive `-XX:+ShenandoahVerify`? Same rationale for `TestRetainObjects` - are you concerned about the runtime length of the tests? > test/hotspot/jtreg/gc/shenandoah/compiler/BarrierInInfiniteLoop.java line 35: > >> 33: */ >> 34: >> 35: /** > > This looks like a compiler regression test, so we don't need to run it in all these different GC modes. I added the generational mode to these tests because we've added a "post-write" barrier for card marking and it touches many of the same files. Figured it's better to error on the side of caution. > test/hotspot/jtreg/gc/shenandoah/generational/TestCLIModeGenerational.java line 34: > >> 32: * @library /testlibrary /test/lib / >> 33: * @build jdk.test.whitebox.WhiteBox >> 34: * @run driver jdk.test.lib.helpers.ClassFileInstaller jdk.test.whitebox.WhiteBox > > Generally, these changes should be done separately, not to clobber the actual "generational"-related test change. It is fine to do them here, if these only touch the files than exist only in GenShen. Right - these files were added to the generational branch, so they were not updated when we merged the WhiteBox renaming changes. > test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 35: > >> 33: /** >> 34: * @test id=heap >> 35: * @summary Test allocation of small object to result OOM, but not to crash JVM > > The `@summary` in these configs are all the same, copy-paste error. > Also, we might want to fix English here a bit: > `Test allocation of (small|large|larger-than-heap) object should result in OOM, not in JVM crash`. Yep, good catch. Thanks. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149515600 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149517068 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149519460 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149522555 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149523271 From wkemper at openjdk.org Mon Mar 27 17:01:32 2023 From: wkemper at openjdk.org (William Kemper) Date: Mon, 27 Mar 2023 17:01:32 GMT Subject: RFR: Restrict generational mode to adaptive heuristic In-Reply-To: References: Message-ID: <2aVxBGQyb9_iAXGL7z5Ucpuxlxoe_k_-8C_y9mMqPpU=.b9456a8f-aaec-4f3d-a356-2d35b83a688f@github.com> On Mon, 27 Mar 2023 10:08:13 GMT, Aleksey Shipilev wrote: >> This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: >> * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). >> * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. > > test/hotspot/jtreg/gc/stress/jfr/TestStressBigAllocationGCEventsWithShenandoah.java line 1: > >> 1: /* > > Is there a generational variant for this? I've added one just now. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149550631 From jsjolen at openjdk.org Tue Mar 28 10:41:36 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Tue, 28 Mar 2023 10:41:36 GMT Subject: RFR: JDK-8301496: Replace NULL with nullptr in cpu/riscv Message-ID: Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/riscv. Unfortunately the script that does the change isn't perfect, and so we need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. Here are some typical things to look out for: 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. An example of this: ```c++ // This function returns null void* ret_null(); // This function returns true if *x == nullptr bool is_nullptr(void** x); Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. Thanks! ------------- Commit messages: - Merge remote-tracking branch 'origin/master' into JDK-8301496 - Fixes - Merge remote-tracking branch 'origin/master' into JDK-8301496 - Replace NULL with nullptr in cpu/riscv Changes: https://git.openjdk.org/jdk/pull/12324/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=12324&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8301496 Stats: 573 lines in 45 files changed: 0 ins; 0 del; 573 mod Patch: https://git.openjdk.org/jdk/pull/12324.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/12324/head:pull/12324 PR: https://git.openjdk.org/jdk/pull/12324 From jsjolen at openjdk.org Tue Mar 28 10:41:54 2023 From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=) Date: Tue, 28 Mar 2023 10:41:54 GMT Subject: RFR: JDK-8301496: Replace NULL with nullptr in cpu/riscv In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:39:59 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/riscv. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! Review of changes to be made. Don't close yet. src/hotspot/cpu/riscv/assembler_riscv.hpp line 2174: > 2172: } \ > 2173: void NAME(address dest) { \ > 2174: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/assembler_riscv.hpp line 2202: > 2200: } \ > 2201: void NAME(Register Rs1, address dest) { \ > 2202: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/gc/shenandoah/shenandoahBarrierSetAssembler_riscv.cpp line 161: > 159: // Calling the runtime using the regular call_VM_leaf mechanism generates > 160: // code (generated by InterpreterMacroAssember::call_VM_leaf_base) > 161: // that checks that the *(rfp+frame::interpreter_frame_last_sp) == null. is null src/hotspot/cpu/riscv/interp_masm_riscv.cpp line 159: > 157: Label L; > 158: ld(t0, Address(xthread, JavaThread::jvmti_thread_state_offset())); > 159: beqz(t0, L); // if [thread->jvmti_thread_state() == null] then exit is src/hotspot/cpu/riscv/interp_masm_riscv.cpp line 1335: > 1333: // ] > 1334: // if (row[2].rec != null) then [ > 1335: // count.incr() nullptr src/hotspot/cpu/riscv/macroAssembler_riscv.cpp line 836: > 834: #define INSN(NAME, REGISTER) \ > 835: void MacroAssembler::NAME(const address dest, Register temp) { \ > 836: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/macroAssembler_riscv.cpp line 880: > 878: #define INSN(NAME) \ > 879: void MacroAssembler::NAME(Register Rd, const address dest, Register temp) { \ > 880: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/macroAssembler_riscv.cpp line 2025: > 2023: void MacroAssembler::null_check(Register reg, int offset) { > 2024: if (needs_explicit_null_check(offset)) { > 2025: // provoke OS null exception if reg = null by is src/hotspot/cpu/riscv/macroAssembler_riscv.cpp line 2031: > 2029: } else { > 2030: // nothing to do, (later) access of M[reg + offset] > 2031: // will provoke OS NULL exception if reg = NULL is src/hotspot/cpu/riscv/macroAssembler_riscv.cpp line 2031: > 2029: } else { > 2030: // nothing to do, (later) access of M[reg + offset] > 2031: // will provoke OS null exception if reg = null is src/hotspot/cpu/riscv/macroAssembler_riscv.hpp line 593: > 591: #define INSN(NAME) \ > 592: void NAME(Register Rs1, Register Rs2, const address dest) { \ > 593: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/macroAssembler_riscv.hpp line 783: > 781: #define INSN(NAME) \ > 782: void NAME(Register Rd, address dest) { \ > 783: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/macroAssembler_riscv.hpp line 840: > 838: #define INSN(NAME) \ > 839: void NAME(FloatRegister Rd, address dest, Register temp = t0) { \ > 840: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/macroAssembler_riscv.hpp line 900: > 898: #define INSN(NAME) \ > 899: void NAME(Register Rs, address dest, Register temp = t0) { \ > 900: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/macroAssembler_riscv.hpp line 946: > 944: #define INSN(NAME) \ > 945: void NAME(FloatRegister Rs, address dest, Register temp = t0) { \ > 946: assert_cond(dest != nullptr); \ align src/hotspot/cpu/riscv/relocInfo_riscv.cpp line 44: > 42: case relocInfo::oop_type: { > 43: oop_Relocation *reloc = (oop_Relocation *)this; > 44: // in movoop when BarrierSet::barrier_set()->barrier_set_nmethod() != null isn't src/hotspot/cpu/riscv/stubGenerator_riscv.cpp line 1774: > 1772: // > 1773: > 1774: // if [src == null] then return -1 is src/hotspot/cpu/riscv/stubGenerator_riscv.cpp line 1782: > 1780: __ bnez(t0, L_failed); > 1781: > 1782: // if [dst == null] then return -1 is src/hotspot/cpu/riscv/templateInterpreterGenerator_riscv.cpp line 833: > 831: Label slow_path; > 832: const Register local_0 = c_rarg0; > 833: // Check if local 0 != null isn't src/hotspot/cpu/riscv/templateTable_riscv.cpp line 3605: > 3603: __ bind(done); > 3604: // x10 = 0: obj == NULL or obj is not an instanceof the specified klass > 3605: // x10 = 1: obj != NULL and obj is an instanceof the specified klass is isn't ------------- PR Review: https://git.openjdk.org/jdk/pull/12324#pullrequestreview-1360564245 PR Comment: https://git.openjdk.org/jdk/pull/12324#issuecomment-1477544237 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150268232 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150268320 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150271398 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150271778 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150272066 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150272555 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150272631 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150273041 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150272972 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150273240 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150273666 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150273722 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150273798 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150273929 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150274006 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150275666 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150276157 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150276275 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150276773 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1150277254 From wkemper at openjdk.org Tue Mar 28 18:06:26 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 28 Mar 2023 18:06:26 GMT Subject: Integrated: Merge openjdk/jdk:master In-Reply-To: References: Message-ID: On Tue, 28 Mar 2023 17:57:47 GMT, William Kemper wrote: > Merges tag jdk-21+15 This pull request has now been integrated. Changeset: aa157238 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/aa157238783004b45829273e4cb53c6255d05199 Stats: 11524 lines in 523 files changed: 6474 ins; 2392 del; 2658 mod Merge openjdk/jdk:master ------------- PR: https://git.openjdk.org/shenandoah/pull/234 From wkemper at openjdk.org Tue Mar 28 18:06:24 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 28 Mar 2023 18:06:24 GMT Subject: Integrated: Merge openjdk/jdk:master Message-ID: Merges tag jdk-21+15 ------------- Commit messages: - Remove unused card table clear method too - Merge tag 'jdk-21+15' into merge-jdk-21-15 - 8304134: jib bootstrapper fails to quote filename when checking download filetype - 8301498: Replace NULL with nullptr in cpu/x86 - 8304089: Convert TraceDependencies to UL - 8294972: Convert jdk.jlink internal plugins to use the Classfile API - 8304657: G1: Rename set_state_empty to set_state_untracked - 8304144: G1: Remove unnecessary is_survivor check in G1ClearCardTableTask - 8304286: java/net/SocketOption/OptionsTest.java failing after JDK-8302659 - 8303804: Fix some errors of If-VectorTest and CMove-VectorTest - ... and 65 more: https://git.openjdk.org/shenandoah/compare/a5d1a9d8...a0c67bac The webrevs contain the adjustments done while merging with regards to each parent branch: - master: https://webrevs.openjdk.org/?repo=shenandoah&pr=234&range=00.0 - openjdk/jdk:master: https://webrevs.openjdk.org/?repo=shenandoah&pr=234&range=00.1 Changes: https://git.openjdk.org/shenandoah/pull/234/files Stats: 11524 lines in 523 files changed: 6474 ins; 2392 del; 2658 mod Patch: https://git.openjdk.org/shenandoah/pull/234.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/234/head:pull/234 PR: https://git.openjdk.org/shenandoah/pull/234 From wkemper at openjdk.org Tue Mar 28 18:31:45 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 28 Mar 2023 18:31:45 GMT Subject: RFR: Restrict generational mode to adaptive heuristic [v2] In-Reply-To: References: Message-ID: > This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: > * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). > * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision: - Put verify option on separate line - Add generational modes to the test - Allow more memory for young generation - Update test to use shenandoah young gen pool when appropriate - Add generational mode test case - Don't make default mode explicit for consistency with other tests - Fix errors, improve test summaries - WIP: Bug fixes, add generational mode to more tests - WIP: Integrate generational mode with more tests - Restrict young generation adaptive heuristic - ... and 2 more: https://git.openjdk.org/shenandoah/compare/34d8f0de...28509ca0 ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/233/files - new: https://git.openjdk.org/shenandoah/pull/233/files/50cc2f34..28509ca0 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=233&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=233&range=00-01 Stats: 11565 lines in 529 files changed: 6499 ins; 2392 del; 2674 mod Patch: https://git.openjdk.org/shenandoah/pull/233.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/233/head:pull/233 PR: https://git.openjdk.org/shenandoah/pull/233 From wkemper at openjdk.org Tue Mar 28 18:31:46 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 28 Mar 2023 18:31:46 GMT Subject: RFR: Restrict generational mode to adaptive heuristic In-Reply-To: References: Message-ID: On Fri, 24 Mar 2023 18:59:50 GMT, William Kemper wrote: > This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: > * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). > * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. Converted to draft while I investigate a few failures. ------------- PR Comment: https://git.openjdk.org/shenandoah/pull/233#issuecomment-1485521072 From shade at openjdk.org Tue Mar 28 18:31:51 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Tue, 28 Mar 2023 18:31:51 GMT Subject: RFR: Restrict generational mode to adaptive heuristic [v2] In-Reply-To: References: Message-ID: On Mon, 27 Mar 2023 16:30:17 GMT, William Kemper wrote: >> test/hotspot/jtreg/gc/shenandoah/TestAllocObjectArrays.java line 167: >> >>> 165: * @run main/othervm -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions -Xmx1g -Xms1g >>> 166: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational >>> 167: * -XX:-UseTLAB -XX:+ShenandoahVerify >> >> This `-XX:+ShenandoahVerify` seems excessive. This config is supposed to run without verification. Copy-paste error, it seems. > > The first execution in this `no-tlab` configuration runs the default mode and heuristic with verification enabled, so I'm doing the same for the generational mode configuration. Ah right, it differs from the above by mode. Nevermind! >> test/hotspot/jtreg/gc/shenandoah/TestRegionSamplingLogging.java line 31: >> >>> 29: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions >>> 30: * -XX:+ShenandoahRegionSampling -XX:+ShenandoahRegionSampling >>> 31: * -Xlog:gc+region=trace:region-snapshots-%p.log::filesize=100,filecount=3 >> >> So, `trace` is enough here? This looks like something we need to fix upstream, rather than do in a bulk change like this. > > This feature of writing the region sampling data out to logs is only on the generational branch at this time. It wouldn't be hard to pull it out into a separate upstream PR. `TestRegionSamplingLogging` is only in generational branch? Then disregard this comment :) >> test/hotspot/jtreg/gc/shenandoah/TestRetainObjects.java line 107: >> >>> 105: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions >>> 106: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational >>> 107: * -XX:+ShenandoahAllocFailureALot -XX:+ShenandoahVerify >> >> I think the intent here is to run without `-XX:+ShenandoahVerify`. > > This configuration runs with `ShenandoahAllocFailureALot`, the preceeding configuration runs with `ShenandoahOOMDuringEvacALot`. I enabled verification for both to make the tests more rigorous. Verification may find errors in card marking that wouldn't necessarily fail the test otherwise. True. Apparently, I only read the `*ALot` part, which confused me into believing this is a duplicate block. >> test/hotspot/jtreg/gc/shenandoah/TestSieveObjects.java line 107: >> >>> 105: * @run main/othervm -Xmx1g -Xms1g -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions >>> 106: * -XX:+UseShenandoahGC -XX:ShenandoahGCHeuristics=adaptive -XX:ShenandoahGCMode=generational >>> 107: * -XX:+ShenandoahAllocFailureALot -XX:+ShenandoahVerify >> >> Excessive `-XX:+ShenandoahVerify`? > > Same rationale for `TestRetainObjects` - are you concerned about the runtime length of the tests? Yes, same overlook on my side. This is fine. >> test/hotspot/jtreg/gc/shenandoah/compiler/BarrierInInfiniteLoop.java line 35: >> >>> 33: */ >>> 34: >>> 35: /** >> >> This looks like a compiler regression test, so we don't need to run it in all these different GC modes. > > I added the generational mode to these tests because we've added a "post-write" barrier for card marking and it touches many of the same files. Figured it's better to error on the side of caution. Yes, but this test chases the particular compiler bug. Running in multiple modes with `-Xcomp` and `-XX:CompileOnly` might not be fast, so it is better to keep these regression tests targeted. It is not the issue for compiler tests that are not regression tests, like `TestCloneBarrier`. >> test/hotspot/jtreg/gc/shenandoah/generational/TestCLIModeGenerational.java line 34: >> >>> 32: * @library /testlibrary /test/lib / >>> 33: * @build jdk.test.whitebox.WhiteBox >>> 34: * @run driver jdk.test.lib.helpers.ClassFileInstaller jdk.test.whitebox.WhiteBox >> >> Generally, these changes should be done separately, not to clobber the actual "generational"-related test change. It is fine to do them here, if these only touch the files than exist only in GenShen. > > Right - these files were added to the generational branch, so they were not updated when we merged the WhiteBox renaming changes. Okay, great. Nevermind then. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149607705 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149609301 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149611649 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149612279 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149615359 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1149606273 From wkemper at openjdk.org Tue Mar 28 19:49:19 2023 From: wkemper at openjdk.org (William Kemper) Date: Tue, 28 Mar 2023 19:49:19 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v6] In-Reply-To: References: Message-ID: > During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: > * The age and affiliation of a region are encoded in the border and shape of the region (respectively). > * Phases are encoded with different colors for different generations and whether they have degenerated. > * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). > * Popup windows can be opened for additional detail on regions, as well as their history. > * The legend shows the number of regions in the state described by the legend item. > * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. > > Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. William Kemper has updated the pull request incrementally with 51 additional commits since the last revision: - Fix comment formatting - Update copyrights - WIP: Fix issues for live mode - WIP: Use a fixed size buffer for events - Show initial logFile argument in toolbar - Do not load log files on event loop thread - Factor visualizer frame setup out of main function - Fix weird 3-space indentation - Layout improvements - WIP: Fix warnings - ... and 41 more: https://git.openjdk.org/shenandoah-visualizer/compare/dadfaff6...06abcd90 ------------- Changes: - all: https://git.openjdk.org/shenandoah-visualizer/pull/1/files - new: https://git.openjdk.org/shenandoah-visualizer/pull/1/files/dadfaff6..06abcd90 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=05 - incr: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=04-05 Stats: 3912 lines in 36 files changed: 1713 ins; 1796 del; 403 mod Patch: https://git.openjdk.org/shenandoah-visualizer/pull/1.diff Fetch: git fetch https://git.openjdk.org/shenandoah-visualizer.git pull/1/head:pull/1 PR: https://git.openjdk.org/shenandoah-visualizer/pull/1 From dholmes at openjdk.org Wed Mar 29 02:40:56 2023 From: dholmes at openjdk.org (David Holmes) Date: Wed, 29 Mar 2023 02:40:56 GMT Subject: RFR: JDK-8301496: Replace NULL with nullptr in cpu/riscv In-Reply-To: References: Message-ID: <9daAIvq4DDVLuvsw9JjFK5Ap3KGkCkuoAEoYuVhbBto=.f8a06117-49e4-4a75-80aa-e07338388581@github.com> On Tue, 31 Jan 2023 11:39:59 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/riscv. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! LGTM! One nit. Thanks src/hotspot/cpu/riscv/stubGenerator_riscv.cpp line 1836: > 1834: __ bne(t1, scratch_src_klass, L_failed); > 1835: > 1836: // if [src->is_Array() != null] then return -1 nullptr for code fragment ------------- Marked as reviewed by dholmes (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/12324#pullrequestreview-1362167012 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1151332024 From shade at openjdk.org Wed Mar 29 08:07:13 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Mar 2023 08:07:13 GMT Subject: RFR: Restrict generational mode to adaptive heuristic [v2] In-Reply-To: References: Message-ID: On Mon, 27 Mar 2023 18:01:55 GMT, Aleksey Shipilev wrote: >> I added the generational mode to these tests because we've added a "post-write" barrier for card marking and it touches many of the same files. Figured it's better to error on the side of caution. > > Yes, but this test chases the particular compiler bug. Running in multiple modes with `-Xcomp` and `-XX:CompileOnly` might not be fast, so it is better to keep these regression tests targeted. It is not the issue for compiler tests that are not regression tests, like `TestCloneBarrier`. I still don't think it is a good idea to touch these regression tests. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1151543507 From shade at openjdk.org Wed Mar 29 08:07:19 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Mar 2023 08:07:19 GMT Subject: RFR: Restrict generational mode to adaptive heuristic [v2] In-Reply-To: References: Message-ID: On Tue, 28 Mar 2023 18:31:45 GMT, William Kemper wrote: >> This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: >> * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). >> * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. > > William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision: > > - Put verify option on separate line > - Add generational modes to the test > - Allow more memory for young generation > - Update test to use shenandoah young gen pool when appropriate > - Add generational mode test case > - Don't make default mode explicit for consistency with other tests > - Fix errors, improve test summaries > - WIP: Bug fixes, add generational mode to more tests > - WIP: Integrate generational mode with more tests > - Restrict young generation adaptive heuristic > - ... and 2 more: https://git.openjdk.org/shenandoah/compare/a7ad66ba...28509ca0 test/hotspot/jtreg/gc/shenandoah/oom/TestAllocOutOfMemory.java line 90: > 88: "-XX:+UseShenandoahGC", > 89: TestAllocOutOfMemory.class.getName(), > 90: "test", args[0]); Suggestion: pull `args[0]` into the local variable. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1151542108 From wkemper at openjdk.org Wed Mar 29 15:43:25 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 29 Mar 2023 15:43:25 GMT Subject: RFR: Restrict generational mode to adaptive heuristic [v3] In-Reply-To: References: Message-ID: > This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: > * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). > * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. William Kemper has updated the pull request incrementally with two additional commits since the last revision: - Use named local variables and add comments to improve readability - Revert changes to compiler regression tests ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/233/files - new: https://git.openjdk.org/shenandoah/pull/233/files/28509ca0..12e402ab Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=233&range=02 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=233&range=01-02 Stats: 279 lines in 22 files changed: 7 ins; 247 del; 25 mod Patch: https://git.openjdk.org/shenandoah/pull/233.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/233/head:pull/233 PR: https://git.openjdk.org/shenandoah/pull/233 From shade at openjdk.org Wed Mar 29 15:49:36 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Wed, 29 Mar 2023 15:49:36 GMT Subject: RFR: Restrict generational mode to adaptive heuristic [v3] In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 15:43:25 GMT, William Kemper wrote: >> This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: >> * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). >> * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. > > William Kemper has updated the pull request incrementally with two additional commits since the last revision: > > - Use named local variables and add comments to improve readability > - Revert changes to compiler regression tests Everything is good now, except for `TestReferenceCAS`. test/hotspot/jtreg/gc/shenandoah/compiler/TestReferenceCAS.java line 56: > 54: */ > 55: > 56: import java.lang.reflect.Field; This one is actually not a regression test, it is similar to `TestClone` that verifies the generic support for Shenandoah barriers. we should keep it here. ------------- Marked as reviewed by shade (Committer). PR Review: https://git.openjdk.org/shenandoah/pull/233#pullrequestreview-1363434982 PR Review Comment: https://git.openjdk.org/shenandoah/pull/233#discussion_r1152148102 From kdnilsen at openjdk.org Wed Mar 29 22:09:40 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 29 Mar 2023 22:09:40 GMT Subject: RFR: Recompute generation accounts after fullgc Message-ID: This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. ------------- Commit messages: - Merge remote-tracking branch 'GitFarmBranch/recompute-generation-accounts-after-fullgc' into recompute-generation-accounts-after-fullgc - Recompute generation usage following Full GC Changes: https://git.openjdk.org/shenandoah/pull/235/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=235&range=00 Stats: 72 lines in 4 files changed: 72 ins; 0 del; 0 mod Patch: https://git.openjdk.org/shenandoah/pull/235.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/235/head:pull/235 PR: https://git.openjdk.org/shenandoah/pull/235 From wkemper at openjdk.org Wed Mar 29 22:09:44 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 29 Mar 2023 22:09:44 GMT Subject: RFR: Abandon mixed collections if all candidates are pinned Message-ID: A new `WAITING_FOR_FILL` state has been added. This state is now entered when all the candidates for a mixed collection cycle are found to be pinned (which should be quite rare). This state permits a new old generation collection cycle to begin (which itself begins with making these regions parseable). This change also adds a new diagnostic flag `ShenandoahCoalesceChance` which is the probability (expressed as a percentage) of abandoning all the candidates of a mixed collection. Any abandoned candidate regions will be made parseable. ------------- Commit messages: - Make sure all candidates really are pinned - Abandon mixed collections if all candidates are pinned Changes: https://git.openjdk.org/shenandoah/pull/236/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=236&range=00 Stats: 82 lines in 9 files changed: 68 ins; 5 del; 9 mod Patch: https://git.openjdk.org/shenandoah/pull/236.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/236/head:pull/236 PR: https://git.openjdk.org/shenandoah/pull/236 From wkemper at openjdk.org Wed Mar 29 22:41:04 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 29 Mar 2023 22:41:04 GMT Subject: RFR: Restrict generational mode to adaptive heuristic [v4] In-Reply-To: References: Message-ID: > This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: > * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). > * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Restore reference CAS barrier test for generational mode ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/233/files - new: https://git.openjdk.org/shenandoah/pull/233/files/12e402ab..0a55ba70 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=233&range=03 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=233&range=02-03 Stats: 26 lines in 1 file changed: 26 ins; 0 del; 0 mod Patch: https://git.openjdk.org/shenandoah/pull/233.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/233/head:pull/233 PR: https://git.openjdk.org/shenandoah/pull/233 From wkemper at openjdk.org Wed Mar 29 22:44:27 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 29 Mar 2023 22:44:27 GMT Subject: Integrated: Restrict generational mode to adaptive heuristic In-Reply-To: References: Message-ID: On Fri, 24 Mar 2023 18:59:50 GMT, William Kemper wrote: > This is a small change to Shenandoah, but it prevents us from running the tests with the generational mode enabled via a jtreg vm option. This requires us to properly integrate the generational mode with the jtreg tests. There are many test files changed, but the `generational` mode is now handled in the same fashion as other modes (`iu`, `passive`). There are two additional advantages to these changes: > * We no longer need to run the test suite twice (once in the default mode and again with `-XX:ShenandoahGCMode=generational`). > * Expanded test coverage, as some tests would spawn their own VMs with permutations of mode and heuristic options that ignored the generational mode. This pull request has now been integrated. Changeset: 2785875d Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/2785875de3bf86ab20779c5a24ec7e0f69e452aa Stats: 1503 lines in 63 files changed: 1153 ins; 309 del; 41 mod Restrict generational mode to adaptive heuristic Reviewed-by: shade ------------- PR: https://git.openjdk.org/shenandoah/pull/233 From wkemper at openjdk.org Wed Mar 29 23:08:50 2023 From: wkemper at openjdk.org (William Kemper) Date: Wed, 29 Mar 2023 23:08:50 GMT Subject: RFR: Recompute generation accounts after fullgc In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 21:26:40 GMT, Kelvin Nilsen wrote: > This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. > > We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 1486: > 1484: } > 1485: > 1486: void ShenandoahFullGC::phase5_restore_generation_accounts() { Maybe call this `phase5_restore_generation_usage` for consistency with `establish_usage`? ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/235#discussion_r1152574298 From kdnilsen at openjdk.org Wed Mar 29 23:48:40 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Wed, 29 Mar 2023 23:48:40 GMT Subject: RFR: Abandon mixed collections if all candidates are pinned In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 21:57:43 GMT, William Kemper wrote: > A new `WAITING_FOR_FILL` state has been added to the old generation. This state is now entered when all the candidates for a mixed collection cycle are found to be pinned (which should be quite rare). This state permits a new old generation collection cycle to begin (which itself begins with making these regions parseable). > > This change also adds a new diagnostic flag `ShenandoahCoalesceChance` which is the probability (expressed as a percentage) of abandoning all the candidates of a mixed collection. Any abandoned candidate regions will be made parseable. this is nice. thanks. src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 454: > 452: "Testing: make lots of artificial allocation failures.") \ > 453: \ > 454: product(uintx, ShenandoahCoalesceChance, 0, DIAGNOSTIC, \ Should this read: "Abandon remaining mixed collections with this likelihood. Following each mixed collection, we abandon all remaining mixed collection candidate regions with likelihood ShenandoahCoalesceChance." ? (I don't think this is precisely abandon this percentage of mixed collections".) ------------- Marked as reviewed by kdnilsen (Committer). PR Review: https://git.openjdk.org/shenandoah/pull/236#pullrequestreview-1364120423 PR Review Comment: https://git.openjdk.org/shenandoah/pull/236#discussion_r1152593828 From kdnilsen at openjdk.org Thu Mar 30 00:33:45 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 00:33:45 GMT Subject: RFR: Recompute generation accounts after fullgc [v2] In-Reply-To: References: Message-ID: > This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. > > We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Reviewer feedback: change function name to ...restore_generation_usage() ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/235/files - new: https://git.openjdk.org/shenandoah/pull/235/files/b4c9c2d6..0e58a72e Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=235&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=235&range=00-01 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.org/shenandoah/pull/235.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/235/head:pull/235 PR: https://git.openjdk.org/shenandoah/pull/235 From ysr at openjdk.org Thu Mar 30 01:48:10 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 30 Mar 2023 01:48:10 GMT Subject: RFR: Recompute generation accounts after fullgc [v2] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 00:33:45 GMT, Kelvin Nilsen wrote: >> This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. >> >> We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Reviewer feedback: change function name to ...restore_generation_usage() Two small nits, but can be ignored if this is "stop-gap" code for now anyway. Reviewed! src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 1506: > 1504: young_regions++; > 1505: young_usage += r->used(); > 1506: if (r->is_humongous()) { Two nits: 1. There is code in the two arms here that can be consolidated into a work method that can then be used to update the appropriate young or old metric. 2. It's conceptually simpler to check if is_humongous() and then add size of object if is_humongous_start. You can there also do arithmetic for the waste in the last humongous region for that object. Then you can just skip all the is_humongous_cont() regions, and save some time. Other than that, this looks good to me. ------------- Marked as reviewed by ysr (Author). PR Review: https://git.openjdk.org/shenandoah/pull/235#pullrequestreview-1364206649 PR Review Comment: https://git.openjdk.org/shenandoah/pull/235#discussion_r1152647195 From fyang at openjdk.org Thu Mar 30 03:43:45 2023 From: fyang at openjdk.org (Fei Yang) Date: Thu, 30 Mar 2023 03:43:45 GMT Subject: RFR: JDK-8301496: Replace NULL with nullptr in cpu/riscv In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:39:59 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/riscv. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! LGTM. Two nits. src/hotspot/cpu/riscv/interp_masm_riscv.cpp line 1616: > 1614: beqz(t0, L); > 1615: stop("InterpreterMacroAssembler::call_VM_leaf_base:" > 1616: " last_sp != null"); Maybe: " last_sp isn't null" ? src/hotspot/cpu/riscv/interp_masm_riscv.cpp line 1643: > 1641: beqz(t0, L); > 1642: stop("InterpreterMacroAssembler::call_VM_base:" > 1643: " last_sp != null"); Similar here. Maybe: " last_sp isn't null" ? ------------- Marked as reviewed by fyang (Reviewer). PR Review: https://git.openjdk.org/jdk/pull/12324#pullrequestreview-1364274565 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1152695397 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1152695612 From fyang at openjdk.org Thu Mar 30 04:12:39 2023 From: fyang at openjdk.org (Fei Yang) Date: Thu, 30 Mar 2023 04:12:39 GMT Subject: RFR: JDK-8301496: Replace NULL with nullptr in cpu/riscv In-Reply-To: References: Message-ID: On Tue, 31 Jan 2023 11:39:59 GMT, Johan Sj?len wrote: > Hi, this PR changes all occurrences of NULL to nullptr for the subdirectory cpu/riscv. Unfortunately the script that does the change isn't perfect, and so we > need to comb through these manually to make sure nothing has gone wrong. I also review these changes but things slip past my eyes sometimes. > > Here are some typical things to look out for: > > 1. No changes but copyright header changed (probably because I reverted some changes but forgot the copyright). > 2. Macros having their NULL changed to nullptr, these are added to the script when I find them. They should be NULL. > 3. nullptr in comments and logs. We try to use lower case "null" in these cases as it reads better. An exception is made when code expressions are in a comment. > > An example of this: > > ```c++ > // This function returns null > void* ret_null(); > // This function returns true if *x == nullptr > bool is_nullptr(void** x); > > > Note how `nullptr` participates in a code expression here, we really are talking about the specific value `nullptr`. > > Thanks! Changes requested by fyang (Reviewer). src/hotspot/cpu/riscv/interpreterRT_riscv.cpp line 230: > 228: virtual void pass_object() { > 229: intptr_t* addr = single_slot_addr(); > 230: intptr_t value = *addr == 0 ? nullptr : (intptr_t)addr; PS: I got compile errors when doing a native build with GCC-11.3.0: 474 /home/fyang/openjdk-jdk/src/hotspot/cpu/riscv/interpreterRT_riscv.cpp: In member function 'virtual void SlowSignatureHandler::pass_object()': 475 /home/fyang/openjdk-jdk/src/hotspot/cpu/riscv/interpreterRT_riscv.cpp:230:33: error: operands to '?:' have different types 'std::nullptr_t' and 'intptr_t' {aka 'long int'} 476 230 | intptr_t value = *addr == 0 ? nullptr : (intptr_t)addr; 477 | ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ I think we should change the "nullptr" here into "(intptr_t)nullptr". ------------- PR Review: https://git.openjdk.org/jdk/pull/12324#pullrequestreview-1364292576 PR Review Comment: https://git.openjdk.org/jdk/pull/12324#discussion_r1152708517 From shade at openjdk.org Thu Mar 30 08:12:15 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 30 Mar 2023 08:12:15 GMT Subject: RFR: Recompute generation accounts after fullgc [v2] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 00:33:45 GMT, Kelvin Nilsen wrote: >> This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. >> >> We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Reviewer feedback: change function name to ...restore_generation_usage() I think `phase4_compact_objects` does the "epilog" for the Full GC code, including setting the region and heap properties. Stands to reason the generation properties fixups should be there too? Also, `phase4_compact_objects` does "FullGC done: GLOBAL usage ..." logging, which in this version happens before we establish the usages for generations? I'd propose to have the same `phase5_...` method you have here, but move everything starting from `// Reset complete bitmap...` from `phase4_...` into it. ------------- PR Review: https://git.openjdk.org/shenandoah/pull/235#pullrequestreview-1364563607 From shade at openjdk.org Thu Mar 30 11:26:56 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 30 Mar 2023 11:26:56 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v6] In-Reply-To: References: Message-ID: On Tue, 28 Mar 2023 19:49:19 GMT, William Kemper wrote: >> During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: >> * The age and affiliation of a region are encoded in the border and shape of the region (respectively). >> * Phases are encoded with different colors for different generations and whether they have degenerated. >> * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). >> * Popup windows can be opened for additional detail on regions, as well as their history. >> * The legend shows the number of regions in the state described by the legend item. >> * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. >> >> Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. > > William Kemper has updated the pull request incrementally with 51 additional commits since the last revision: > > - Fix comment formatting > - Update copyrights > - WIP: Fix issues for live mode > - WIP: Use a fixed size buffer for events > - Show initial logFile argument in toolbar > - Do not load log files on event loop thread > - Factor visualizer frame setup out of main function > - Fix weird 3-space indentation > - Layout improvements > - WIP: Fix warnings > - ... and 41 more: https://git.openjdk.org/shenandoah-visualizer/compare/dadfaff6...06abcd90 I did a cursory review for this, and I am thinking what to do next. There are many small things need to be done to improve this code, and going back-and-forth with reviews would probably be inefficient. There are few ways we can proceed: 1. I keep putting all the comments here, and you work on them. 2. We push the PR in current form, knowing it works for you. Then we massage the code in subsequent PRs. 3. I do the series of cleanups, give you the patch for this PR, you test it works, then we push the PR in new form. Tell me which one do you prefer? .github/workflows/pre-integration.yml line 13: > 11: strategy: > 12: matrix: > 13: java: [8, 11, 15, 16-ea] While you are at it, you might want to change `java:` to `[8, 11, 17, 20, 21-ea]` pom.xml line 18: > 16: > 17: UTF-8 > 18: 19 Why do we need the target = 19? This would break GHA tests that run with JDK 8, I think. I would prefer to keep the baseline at JDK 8, if possible. If you need something more recent, then the latest LTS would be better, so target 17? You'll need to drop some JDK configs from GHA testing though. src/main/java/org/openjdk/shenandoah/CircularBuffer.java line 31: > 29: public class CircularBuffer { > 30: > 31: public static final int DEFAULT_SIZE = 8; Why `public`? Do you want it accessible in tests? You can drop `public` then, and rely on package-private visibility for this constant. src/main/java/org/openjdk/shenandoah/CircularBuffer.java line 79: > 77: > 78: public int size() { > 79: return Math.min(count, elements.length); Why not just `return count`? src/main/java/org/openjdk/shenandoah/Colors.java line 71: > 69: static final Color AGE_3 = Color.RED; > 70: static final Color AGE_4 = Color.BLUE; > 71: static final Color AGE_5 = Color.BLACK; Are these used anywhere? You can just inline them into `AGE_COLORS`? src/main/java/org/openjdk/shenandoah/DataConnector.java line 73: > 71: > 72: enum State { > 73: Searching, Connecting, Connected, Disconnecting, Disconnected Style: enum constants should be capitalized. src/main/java/org/openjdk/shenandoah/DataConnector.java line 95: > 93: > 94: @Override > 95: public void run() { This should be a nested class, something like `DataConnector.ConnectTask`. Saves you from some foreign code to accidentally put `this` into another executor. src/main/java/org/openjdk/shenandoah/DataConnector.java line 104: > 102: subscribeToGarbageCollectorNotifications(server); > 103: monitoredVmConsumer.accept(vm); > 104: shouldRun = false; `shouldRun` looks redundant, can just `break` out of the `while(true)` loop here. src/main/java/org/openjdk/shenandoah/DataLogProvider.java line 49: > 47: String metaDataLine = br.readLine(); // timestamp status numRegions regionSize > 48: > 49: while (metaDataLine != null && metaDataLine.trim().length() > 0) { This is better written as: Suggestion: // Metadata line: timestamp status numRegions regionSize String metaDataLine; while ((metaDataLine = br.readLine()) != null) { if (metaDataLine.trim().isEmpty()) continue; src/main/java/org/openjdk/shenandoah/RegionStat.java line 76: > 74: this.showLivenessDetail = Boolean.getBoolean("show.liveness"); > 75: } > 76: // This constructor is for CounterTest Can be package-private then. ------------- PR Review: https://git.openjdk.org/shenandoah-visualizer/pull/1#pullrequestreview-1364770965 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153032651 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153034482 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153036929 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153051320 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153041557 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153042478 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153044701 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153046373 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153058909 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153059852 From kdnilsen at openjdk.org Thu Mar 30 14:37:31 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 14:37:31 GMT Subject: RFR: Recompute generation accounts after fullgc [v3] In-Reply-To: References: Message-ID: > This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. > > We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: More efficiently handle each humongous object in Full GC ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/235/files - new: https://git.openjdk.org/shenandoah/pull/235/files/0e58a72e..a333c50c Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=235&range=02 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=235&range=01-02 Stats: 121 lines in 3 files changed: 59 ins; 60 del; 2 mod Patch: https://git.openjdk.org/shenandoah/pull/235.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/235/head:pull/235 PR: https://git.openjdk.org/shenandoah/pull/235 From wkemper at openjdk.org Thu Mar 30 15:39:18 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 15:39:18 GMT Subject: RFR: Recompute generation accounts after fullgc [v3] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 14:37:31 GMT, Kelvin Nilsen wrote: >> This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. >> >> We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > More efficiently handle each humongous object in Full GC Marked as reviewed by wkemper (Committer). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/235#pullrequestreview-1365410420 From wkemper at openjdk.org Thu Mar 30 16:04:23 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 16:04:23 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v6] In-Reply-To: References: Message-ID: On Tue, 28 Mar 2023 19:49:19 GMT, William Kemper wrote: >> During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: >> * The age and affiliation of a region are encoded in the border and shape of the region (respectively). >> * Phases are encoded with different colors for different generations and whether they have degenerated. >> * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). >> * Popup windows can be opened for additional detail on regions, as well as their history. >> * The legend shows the number of regions in the state described by the legend item. >> * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. >> >> Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. > > William Kemper has updated the pull request incrementally with 51 additional commits since the last revision: > > - Fix comment formatting > - Update copyrights > - WIP: Fix issues for live mode > - WIP: Use a fixed size buffer for events > - Show initial logFile argument in toolbar > - Do not load log files on event loop thread > - Factor visualizer frame setup out of main function > - Fix weird 3-space indentation > - Layout improvements > - WIP: Fix warnings > - ... and 41 more: https://git.openjdk.org/shenandoah-visualizer/compare/dadfaff6...06abcd90 I'll address the feedback so far and take a pass at cleaning up some of the code. I upgraded the version mostly to experiment with new language features - I can settle on Java 17. I'd prefer to get these changes in and then work on any remaining issues in subsequent PRs. ------------- PR Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#issuecomment-1490553540 From wkemper at openjdk.org Thu Mar 30 16:51:39 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 16:51:39 GMT Subject: RFR: Abandon mixed collections if all candidates are pinned [v2] In-Reply-To: References: Message-ID: > A new `WAITING_FOR_FILL` state has been added to the old generation. This state is now entered when all the candidates for a mixed collection cycle are found to be pinned (which should be quite rare). This state permits a new old generation collection cycle to begin (which itself begins with making these regions parseable). > > This change also adds a new diagnostic flag `ShenandoahCoalesceChance` which is the probability (expressed as a percentage) of abandoning all the candidates of a mixed collection. Any abandoned candidate regions will be made parseable. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Improve description of new ShenandoahCoalesceChance diagnostic option ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/236/files - new: https://git.openjdk.org/shenandoah/pull/236/files/7be1a1db..84c8e99a Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=236&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=236&range=00-01 Stats: 6 lines in 1 file changed: 3 ins; 0 del; 3 mod Patch: https://git.openjdk.org/shenandoah/pull/236.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/236/head:pull/236 PR: https://git.openjdk.org/shenandoah/pull/236 From wkemper at openjdk.org Thu Mar 30 16:51:40 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 16:51:40 GMT Subject: RFR: Abandon mixed collections if all candidates are pinned [v2] In-Reply-To: References: Message-ID: <7klDgmOyGMlbS-icLoGtmZkj1P1CoC7YrqPUpOhDfVg=.84fd3c21-3249-4e1a-a821-bccba1db4d8a@github.com> On Wed, 29 Mar 2023 23:44:29 GMT, Kelvin Nilsen wrote: >> William Kemper has updated the pull request incrementally with one additional commit since the last revision: >> >> Improve description of new ShenandoahCoalesceChance diagnostic option > > src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 454: > >> 452: "Testing: make lots of artificial allocation failures.") \ >> 453: \ >> 454: product(uintx, ShenandoahCoalesceChance, 0, DIAGNOSTIC, \ > > Should this read: "Abandon remaining mixed collections with this likelihood. Following each mixed collection, we abandon all remaining mixed collection candidate regions with likelihood ShenandoahCoalesceChance." ? (I don't think this is precisely abandon this percentage of mixed collections".) Yes, that's a more accurate description. I'll update this description. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/236#discussion_r1153525025 From ysr at openjdk.org Thu Mar 30 20:36:58 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 30 Mar 2023 20:36:58 GMT Subject: RFR: JVM-1767: GenShen: Reference Processing Needs a Card-Marking Barrier Message-ID: JVM-1767: GenShen: Reference Processing Needs a Card-Marking Barrier Global collections can create new cross-generational pointers during j.l..r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and have provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the barrier is also redundantly added for young collections too. I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb. Will run it through our internal tests pipeline too before pushing the fix. I have not yet run any performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but am open to any suggestions. Thank you for your reviews. ------------- Commit messages: - Add documentation comment & some assertions. - When (generational) GC discovers j.l.Reference:s in both generations, then subsequent - Shelve for now. May not compile. Changes: https://git.openjdk.org/shenandoah/pull/238/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=238&range=00 Issue: https://bugs.openjdk.org/browse/JDK-1767 Stats: 28 lines in 1 file changed: 28 ins; 0 del; 0 mod Patch: https://git.openjdk.org/shenandoah/pull/238.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/238/head:pull/238 PR: https://git.openjdk.org/shenandoah/pull/238 From ysr at openjdk.org Thu Mar 30 21:01:54 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Thu, 30 Mar 2023 21:01:54 GMT Subject: RFR: Recompute generation accounts after fullgc [v3] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 14:37:31 GMT, Kelvin Nilsen wrote: >> This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. >> >> We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > More efficiently handle each humongous object in Full GC Marked as reviewed by ysr (Author). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/235#pullrequestreview-1365920455 From wkemper at openjdk.org Thu Mar 30 21:48:47 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 21:48:47 GMT Subject: RFR: Abandon mixed collections if all candidates are pinned [v3] In-Reply-To: References: Message-ID: <6gvi0s8focf3xeVa0jY4KPkw5Uq6aITT7bQpjXkPMGg=.03a71b4d-48a5-4268-8b84-47aed1c16669@github.com> > A new `WAITING_FOR_FILL` state has been added to the old generation. This state is now entered when all the candidates for a mixed collection cycle are found to be pinned (which should be quite rare). This state permits a new old generation collection cycle to begin (which itself begins with making these regions parseable). > > This change also adds a new diagnostic flag `ShenandoahCoalesceChance` which is the probability (expressed as a percentage) of abandoning all the candidates of a mixed collection. Any abandoned candidate regions will be made parseable. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Abandon mixed collection candidates after they've been made parseable ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/236/files - new: https://git.openjdk.org/shenandoah/pull/236/files/84c8e99a..22b70629 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=236&range=02 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=236&range=01-02 Stats: 6 lines in 1 file changed: 6 ins; 0 del; 0 mod Patch: https://git.openjdk.org/shenandoah/pull/236.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/236/head:pull/236 PR: https://git.openjdk.org/shenandoah/pull/236 From wkemper at openjdk.org Thu Mar 30 22:09:56 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 22:09:56 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v6] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 10:16:53 GMT, Aleksey Shipilev wrote: >> William Kemper has updated the pull request incrementally with 51 additional commits since the last revision: >> >> - Fix comment formatting >> - Update copyrights >> - WIP: Fix issues for live mode >> - WIP: Use a fixed size buffer for events >> - Show initial logFile argument in toolbar >> - Do not load log files on event loop thread >> - Factor visualizer frame setup out of main function >> - Fix weird 3-space indentation >> - Layout improvements >> - WIP: Fix warnings >> - ... and 41 more: https://git.openjdk.org/shenandoah-visualizer/compare/dadfaff6...06abcd90 > > src/main/java/org/openjdk/shenandoah/DataConnector.java line 104: > >> 102: subscribeToGarbageCollectorNotifications(server); >> 103: monitoredVmConsumer.accept(vm); >> 104: shouldRun = false; > > `shouldRun` looks redundant, can just `break` out of the `while(true)` loop here. `shouldRun` can be changed by other threads (for example, when a recording is loaded we stop looking for monitored VMs). ------------- PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153836251 From wkemper at openjdk.org Thu Mar 30 22:12:46 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 22:12:46 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v6] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 10:21:55 GMT, Aleksey Shipilev wrote: >> William Kemper has updated the pull request incrementally with 51 additional commits since the last revision: >> >> - Fix comment formatting >> - Update copyrights >> - WIP: Fix issues for live mode >> - WIP: Use a fixed size buffer for events >> - Show initial logFile argument in toolbar >> - Do not load log files on event loop thread >> - Factor visualizer frame setup out of main function >> - Fix weird 3-space indentation >> - Layout improvements >> - WIP: Fix warnings >> - ... and 41 more: https://git.openjdk.org/shenandoah-visualizer/compare/dadfaff6...06abcd90 > > src/main/java/org/openjdk/shenandoah/CircularBuffer.java line 79: > >> 77: >> 78: public int size() { >> 79: return Math.min(count, elements.length); > > Why not just `return count`? The code uses `count` to detect when the write index has wrapped (count itself does not wrap). This is mostly to support counted loop iteration. ------------- PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1153837878 From kdnilsen at openjdk.org Thu Mar 30 22:39:59 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 22:39:59 GMT Subject: RFR: Recompute generation accounts after fullgc [v3] In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 23:05:57 GMT, William Kemper wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> More efficiently handle each humongous object in Full GC > > src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 1486: > >> 1484: } >> 1485: >> 1486: void ShenandoahFullGC::phase5_restore_generation_accounts() { > > Maybe call this `phase5_restore_generation_usage` for consistency with `establish_usage`? I made this change, and then @shipilev suggested another change below. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/235#discussion_r1153852406 From kdnilsen at openjdk.org Thu Mar 30 22:40:05 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 22:40:05 GMT Subject: RFR: Recompute generation accounts after fullgc [v2] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 01:44:05 GMT, Y. Srinivas Ramakrishna wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Reviewer feedback: change function name to ...restore_generation_usage() > > src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 1506: > >> 1504: young_regions++; >> 1505: young_usage += r->used(); >> 1506: if (r->is_humongous()) { > > Two nits: > > 1. There is code in the two arms here that can be consolidated into a work method that can then be used to update the appropriate young or old metric. > 2. It's conceptually simpler to check if is_humongous() and then add size of object if is_humongous_start. You can there also do arithmetic for the waste in the last humongous region for that object. Then you can just skip all the is_humongous_cont() regions, and save some time. > > Other than that, this looks good to me. Thanks. I've refactored to reuse this code and reduced the amount of humongous processing effort to only trigger on is_humongous_start(). ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/235#discussion_r1153852833 From kdnilsen at openjdk.org Thu Mar 30 22:40:06 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 22:40:06 GMT Subject: Integrated: Recompute generation accounts after fullgc In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 21:26:40 GMT, Kelvin Nilsen wrote: > This fix addresses a problem found during verification following full gcs. This recomputes the generation usage at the end of full GC as part of a 5th phase of full gc. > > We may be able to remove this 5th phase at a later time after we find and fix whatever errors exist in generation accounting in the existing full gc implementation. This pull request has now been integrated. Changeset: f7cede97 Author: Kelvin Nilsen URL: https://git.openjdk.org/shenandoah/commit/f7cede97c507cf847579f001de311353bd3c7d86 Stats: 71 lines in 5 files changed: 71 ins; 0 del; 0 mod Recompute generation accounts after fullgc Reviewed-by: ysr, wkemper ------------- PR: https://git.openjdk.org/shenandoah/pull/235 From wkemper at openjdk.org Thu Mar 30 22:41:03 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 22:41:03 GMT Subject: RFR: JVM-1767: GenShen: Reference Processing Needs a Card-Marking Barrier In-Reply-To: References: Message-ID: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> On Thu, 30 Mar 2023 20:30:26 GMT, Y. Srinivas Ramakrishna wrote: > JVM-1767: GenShen: Reference Processing Needs a Card-Marking Barrier > > Global collections can create new cross-generational pointers during j.l..r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and have provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. > > I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb. Will run it through our internal tests pipeline too before pushing the fix. > > I have not yet run any performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but am open to any suggestions. > > Thank you for your reviews. src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 62: > 60: } > 61: > 62: template This looks similar to the code we have for dropping a reference. https://github.com/openjdk/shenandoah/blob/2785875de3bf86ab20779c5a24ec7e0f69e452aa/src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp#L397 Could we use this function there? ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/238#discussion_r1153854386 From kdnilsen at openjdk.org Thu Mar 30 23:10:56 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 23:10:56 GMT Subject: RFR: Fix satb barrier for object array copy Message-ID: The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. ------------- Commit messages: - Tidy up branch to make consistent with recompute-generation_usage PR - Integrate improvements to region-usage-accounting after full gc - Change name of phase5_restore_generation_usage - Fix comparison between array address and TAMS for old-gen collection - Merge remote-tracking branch 'GitFarmBranch/fix-satb-barrier-for-objectarray-copy' into fix-satb-barrier-for-object-array-copy - Restore generation budgets following fullgc - Fix implementation of SATB barrier for objectarray copy Changes: https://git.openjdk.org/shenandoah/pull/237/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=237&range=00 Stats: 121 lines in 7 files changed: 114 ins; 0 del; 7 mod Patch: https://git.openjdk.org/shenandoah/pull/237.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/237/head:pull/237 PR: https://git.openjdk.org/shenandoah/pull/237 From shade at openjdk.org Thu Mar 30 23:11:03 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Thu, 30 Mar 2023 23:11:03 GMT Subject: RFR: Fix satb barrier for object array copy In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 23:54:25 GMT, Kelvin Nilsen wrote: > The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. > > During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. I have suggestions :) Edit: Apologies, have not realized this is a draft PR, so I reviewed it as if it was final. Still, consider these suggestions! src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 431: > 429: > 430: template > 431: void ShenandoahBarrierSet::arraycopy_marking(T* src, T* dst, size_t count, bool is_old) { Oh, this `is_old` is not about the object, it is about the marking! Maybe name it `is_old_marking`? src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 458: > 456: if (is_old) { > 457: ShenandoahHeapRegion* r = _heap->heap_region_containing(array); > 458: assert(_heap->mode()->is_generational(), "Only perform old-marking if in generational mode"); Let's say: Suggestion: if (is_old) { // Generational, old marking assert(_heap->mode()->is_generational(), "Invariant"); ShenandoahHeapRegion* r = _heap->heap_region_containing(array); src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 459: > 457: ShenandoahHeapRegion* r = _heap->heap_region_containing(array); > 458: assert(_heap->mode()->is_generational(), "Only perform old-marking if in generational mode"); > 459: if (r->is_old() && (((HeapWord*) array)) < _heap->marking_context()->top_at_mark_start(r)) { There is a missing `reinterpret_cast(array)` here. Probably pull it into the local, so that every branch can just reuse it. Also, this check is `!_heap->marking_context()->allocated_after_mark_start(reinterpret_cast(array))`? src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 463: > 461: } > 462: } else if (_heap->mode()->is_generational()) { > 463: ShenandoahHeapRegion* r = _heap->heap_region_containing(array); Suggestion: } else if (_heap->mode()->is_generational()) { // Generational, young marking ShenandoahHeapRegion* r = _heap->heap_region_containing(array); src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 467: > 465: arraycopy_work(array, count); > 466: } > 467: } else { Suggestion: } else { // Non-generational assert(!_heap->mode()->is_generational(), "Invariant"); src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 325: > 323: > 324: phase4_compact_objects(worker_slices); > 325: This is the part of #235, is it? Should not be in this PR, I think. ------------- PR Review: https://git.openjdk.org/shenandoah/pull/237#pullrequestreview-1364570502 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1152907365 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1152920559 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1152904283 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1152920891 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1152921627 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1152897354 From kdnilsen at openjdk.org Thu Mar 30 23:11:05 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 23:11:05 GMT Subject: RFR: Fix satb barrier for object array copy In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 23:54:25 GMT, Kelvin Nilsen wrote: > The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. > > During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. I want to make a change to the code and retest before removing draft status. ------------- PR Comment: https://git.openjdk.org/shenandoah/pull/237#issuecomment-1489488252 From kdnilsen at openjdk.org Thu Mar 30 23:11:06 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 23:11:06 GMT Subject: RFR: Fix satb barrier for object array copy In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 08:13:51 GMT, Aleksey Shipilev wrote: >> The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. >> >> During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. > > src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 325: > >> 323: >> 324: phase4_compact_objects(worker_slices); >> 325: > > This is the part of #235, is it? Should not be in this PR, I think. I included #235 redundantly here so that I could pass some regression tests. I've integrated #235 already, so the history shows that commit before this one... ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1153869428 From wkemper at openjdk.org Thu Mar 30 23:18:55 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 23:18:55 GMT Subject: RFR: Fix satb barrier for object array copy In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 23:54:25 GMT, Kelvin Nilsen wrote: > The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. > > During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 446: > 444: * > 445: * 1. The overwritten array resides in old-gen and it is below TAMS within its old-gen region > 446: * 2. Do not call arraycopy_work for any array residing young-gen because young-gen collection is idle at this time ... array residing _in_ young-gen ... ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1153875281 From kdnilsen at openjdk.org Thu Mar 30 23:32:31 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Thu, 30 Mar 2023 23:32:31 GMT Subject: RFR: Fix satb barrier for object array copy [v2] In-Reply-To: References: Message-ID: > The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. > > During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Fix typo in comment ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/237/files - new: https://git.openjdk.org/shenandoah/pull/237/files/1ce3bf5e..883ab7c3 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=237&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=237&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.org/shenandoah/pull/237.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/237/head:pull/237 PR: https://git.openjdk.org/shenandoah/pull/237 From wkemper at openjdk.org Thu Mar 30 23:32:32 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 23:32:32 GMT Subject: RFR: Fix satb barrier for object array copy [v2] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 23:27:57 GMT, Kelvin Nilsen wrote: >> The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. >> >> During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Fix typo in comment There is a typo in the comments. Suggest using a consistent test for "array is below TAMS". src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 461: > 459: assert(_heap->mode()->is_generational(), "Invariant"); > 460: ShenandoahHeapRegion* r = _heap->heap_region_containing(array_addr); > 461: if (r->is_old() && (array_addr < _heap->marking_context()->top_at_mark_start(r))) { Why not use `!_heap->marking_context()->allocated_after_mark_start(array_addr)` as other branches do? Or replace other `!_heap->marking_context()->allocated_after_mark_start(array_addr)` with `array_addr < _heap->marking_context()->top_at_mark_start(r))` to avoid a second region look up? src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 476: > 474: } else { > 475: T* array = src; > 476: if (!_heap->marking_context()->allocated_after_mark_start(reinterpret_cast(array))) { Pull `array_addr` up and replace `reinterpret_cast(array)` with `array_addr`? ------------- PR Review: https://git.openjdk.org/shenandoah/pull/237#pullrequestreview-1366064653 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1153877907 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1153877631 From wkemper at openjdk.org Thu Mar 30 23:43:29 2023 From: wkemper at openjdk.org (William Kemper) Date: Thu, 30 Mar 2023 23:43:29 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v7] In-Reply-To: References: Message-ID: > During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: > * The age and affiliation of a region are encoded in the border and shape of the region (respectively). > * Phases are encoded with different colors for different generations and whether they have degenerated. > * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). > * Popup windows can be opened for additional detail on regions, as well as their history. > * The legend shows the number of regions in the state described by the legend item. > * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. > > Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. William Kemper has updated the pull request incrementally with eight additional commits since the last revision: - Remove extra whitespace - Improve handling of interrupted exception - Use more idiomatic pattern for iterating over lines in a file - Improve encapsulation of DataConnector's threading - Use idiomatic case conventions for enum - Inline usages of Color.AGE_N variables - Reduce visibility of members to package-private where possible - Target jdk17, update github actions ------------- Changes: - all: https://git.openjdk.org/shenandoah-visualizer/pull/1/files - new: https://git.openjdk.org/shenandoah-visualizer/pull/1/files/06abcd90..5d380c30 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=06 - incr: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=05-06 Stats: 199 lines in 25 files changed: 5 ins; 13 del; 181 mod Patch: https://git.openjdk.org/shenandoah-visualizer/pull/1.diff Fetch: git fetch https://git.openjdk.org/shenandoah-visualizer.git pull/1/head:pull/1 PR: https://git.openjdk.org/shenandoah-visualizer/pull/1 From ysr at openjdk.org Fri Mar 31 00:05:49 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 00:05:49 GMT Subject: RFR: GenShen: Reference Processing Needs a Card-Marking Barrier In-Reply-To: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> References: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> Message-ID: On Thu, 30 Mar 2023 22:37:59 GMT, William Kemper wrote: >> GenShen: Reference Processing Needs a Card-Marking Barrier >> >> Global collections can create new cross-generational pointers during j.l..r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and have provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. >> >> I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb. Will run it through our internal tests pipeline too before pushing the fix. >> >> I have not yet run any performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but am open to any suggestions. >> >> Thank you for your reviews. > > src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 62: > >> 60: } >> 61: >> 62: template > > This looks similar to the code we have for dropping a reference. > https://github.com/openjdk/shenandoah/blob/2785875de3bf86ab20779c5a24ec7e0f69e452aa/src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp#L397 > > Could we use this function there? Let me look into this a bit more closely, in regards to how the card-scanning code treats the traversal of weak pointers. In other words, it must either be the case that the card-mark done in the code you pointed out above (in `drop`) is not needed or, if it is needed, then at least my comment (and attendant assertion at lines 81 & 82) below is wrong and needs to be suitably adjusted and retested: Those cards will continue to // remain dirty on account of this cross-generational pointer to the referent. I'll update after looking into this more closely. (I'll also then see if the code can be shared like you asked.) ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/238#discussion_r1153896157 From ysr at openjdk.org Fri Mar 31 01:04:53 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 01:04:53 GMT Subject: RFR: GenShen: Reference Processing Needs a Card-Marking Barrier In-Reply-To: References: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> Message-ID: On Fri, 31 Mar 2023 00:02:50 GMT, Y. Srinivas Ramakrishna wrote: >> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 62: >> >>> 60: } >>> 61: >>> 62: template >> >> This looks similar to the code we have for dropping a reference. >> https://github.com/openjdk/shenandoah/blob/2785875de3bf86ab20779c5a24ec7e0f69e452aa/src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp#L397 >> >> Could we use this function there? > > Let me look into this a bit more closely, in regards to how the card-scanning code treats the traversal of weak pointers. > > In other words, it must either be the case that the card-mark done in the code you pointed out above (in `drop`) is not needed or, if it is needed, then at least my comment (and attendant assertion at lines 81 & 82) below is wrong and needs to be suitably adjusted and retested: > > Those cards will continue to > // remain dirty on account of this cross-generational pointer to the referent. > > > I'll update after looking into this more closely. (I'll also then see if the code can be shared like you asked.) A visual examination of the card-scanning code reveals that the card-mark in the portion of code you pointed out is redundant because the card-scanning code always dirties the card containing a cross-generational pointer, irrespective of whether it's a weak or strong pointer. I'll work on eliding that unnecessary check and mark in a separate PR as I want to add some assertions and check it carefully. Thanks for drawing my attention to this code which I had just glossed over before. The diffs in my original PR stay unchanged and I'll not try to do anything about sharing the code, since I expect that piece of card-marking code in `drop()` to go away in the PR I promised above, for which I'll open a ticket momentarily. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/238#discussion_r1153918285 From ysr at openjdk.org Fri Mar 31 02:25:07 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 02:25:07 GMT Subject: RFR: Abandon mixed collections if all candidates are pinned [v3] In-Reply-To: <6gvi0s8focf3xeVa0jY4KPkw5Uq6aITT7bQpjXkPMGg=.03a71b4d-48a5-4268-8b84-47aed1c16669@github.com> References: <6gvi0s8focf3xeVa0jY4KPkw5Uq6aITT7bQpjXkPMGg=.03a71b4d-48a5-4268-8b84-47aed1c16669@github.com> Message-ID: On Thu, 30 Mar 2023 21:48:47 GMT, William Kemper wrote: >> A new `WAITING_FOR_FILL` state has been added to the old generation. This state is now entered when all the candidates for a mixed collection cycle are found to be pinned (which should be quite rare). This state permits a new old generation collection cycle to begin (which itself begins with making these regions parseable). >> >> This change also adds a new diagnostic flag `ShenandoahCoalesceChance` which is the probability (expressed as a percentage) of abandoning all the candidates of a mixed collection. Any abandoned candidate regions will be made parseable. > > William Kemper has updated the pull request incrementally with one additional commit since the last revision: > > Abandon mixed collection candidates after they've been made parseable LGTM! src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 118: > 116: _old_generation->transition_to(ShenandoahOldGeneration::IDLE); > 117: } else if (included_old_regions == 0) { > 118: // We have candidates, but none were included for evacuation - are they all be pinned? are they all pinned? (no be) src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 260: > 258: // This code will see the same set of regions to filled on each resumption as it did > 259: // on the initial run. That's okay because each region keeps track of its own coalesce > 260: // and fill state. Region that were filled on a prior attempt will not try to fill again. Regions src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 391: > 389: // > 390: bool ShenandoahOldGeneration::validate_transition(State new_state) { > 391: ShenandoahHeap* heap = ShenandoahHeap::heap(); Could you update the ascii art of the sate transition diagram to include WAITING_FOR* states instead of the old WAITING state? ------------- Marked as reviewed by ysr (Author). PR Review: https://git.openjdk.org/shenandoah/pull/236#pullrequestreview-1366156669 PR Review Comment: https://git.openjdk.org/shenandoah/pull/236#discussion_r1153946691 PR Review Comment: https://git.openjdk.org/shenandoah/pull/236#discussion_r1153946452 PR Review Comment: https://git.openjdk.org/shenandoah/pull/236#discussion_r1153947896 From ysr at openjdk.org Fri Mar 31 02:33:53 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 02:33:53 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier In-Reply-To: References: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> Message-ID: On Fri, 31 Mar 2023 01:00:18 GMT, Y. Srinivas Ramakrishna wrote: >> Let me look into this a bit more closely, in regards to how the card-scanning code treats the traversal of weak pointers. >> >> In other words, it must either be the case that the card-mark done in the code you pointed out above (in `drop`) is not needed or, if it is needed, then at least my comment (and attendant assertion at lines 81 & 82) below is wrong and needs to be suitably adjusted and retested: >> >> Those cards will continue to >> // remain dirty on account of this cross-generational pointer to the referent. >> >> >> I'll update after looking into this more closely. (I'll also then see if the code can be shared like you asked.) > > A visual examination of the card-scanning code reveals that the card-mark in the portion of code you pointed out is redundant because the card-scanning code always dirties the card containing a cross-generational pointer, irrespective of whether it's a weak or strong pointer. I'll work on eliding that unnecessary check and mark in a separate PR as I want to add some assertions and check it carefully. > > Thanks for drawing my attention to this code which I had just glossed over before. > > The diffs in my original PR stay unchanged and I'll not try to do anything about sharing the code, since I expect that piece of card-marking code in `drop()` to go away in the PR I promised above, for which I'll open a ticket momentarily. I opened https://bugs.openjdk.org/browse/JDK-8305335 to follow up on that. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/238#discussion_r1153952419 From shade at openjdk.org Fri Mar 31 07:55:46 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 31 Mar 2023 07:55:46 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v6] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 22:10:13 GMT, William Kemper wrote: >> src/main/java/org/openjdk/shenandoah/CircularBuffer.java line 79: >> >>> 77: >>> 78: public int size() { >>> 79: return Math.min(count, elements.length); >> >> Why not just `return count`? > > The code uses `count` to detect when the write index has wrapped (count itself does not wrap). This is mostly to support counted loop iteration. All right then! >> src/main/java/org/openjdk/shenandoah/DataConnector.java line 104: >> >>> 102: subscribeToGarbageCollectorNotifications(server); >>> 103: monitoredVmConsumer.accept(vm); >>> 104: shouldRun = false; >> >> `shouldRun` looks redundant, can just `break` out of the `while(true)` loop here. > > `shouldRun` can be changed by other threads (for example, when a recording is loaded we stop looking for monitored VMs). All right, this would be a target for clean up later. ------------- PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1154139229 PR Review Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#discussion_r1154139721 From shade at openjdk.org Fri Mar 31 08:05:51 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 31 Mar 2023 08:05:51 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v7] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 23:43:29 GMT, William Kemper wrote: >> During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: >> * The age and affiliation of a region are encoded in the border and shape of the region (respectively). >> * Phases are encoded with different colors for different generations and whether they have degenerated. >> * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). >> * Popup windows can be opened for additional detail on regions, as well as their history. >> * The legend shows the number of regions in the state described by the legend item. >> * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. >> >> Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. > > William Kemper has updated the pull request incrementally with eight additional commits since the last revision: > > - Remove extra whitespace > - Improve handling of interrupted exception > - Use more idiomatic pattern for iterating over lines in a file > - Improve encapsulation of DataConnector's threading > - Use idiomatic case conventions for enum > - Inline usages of Color.AGE_N variables > - Reduce visibility of members to package-private where possible > - Target jdk17, update github actions Okay, thank you for doing the cleanups! I think we can accept these changes in their current form, with the intent to cleanup more later. I don't see GHA runs for this, could you go to https://github.com/earthling-amzn/shenandoah-visualizer/actions and enable them? You'd probably need to submit the workflow run on your `earthling-amzn:genshen-visualizer` branch manually. Bots should catch up with the results then,. ------------- Marked as reviewed by shade (Reviewer). PR Review: https://git.openjdk.org/shenandoah-visualizer/pull/1#pullrequestreview-1366451940 From rkennke at openjdk.org Fri Mar 31 08:08:52 2023 From: rkennke at openjdk.org (Roman Kennke) Date: Fri, 31 Mar 2023 08:08:52 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v7] In-Reply-To: References: Message-ID: <8V0ia6Ln0kNcT6kpRiVXJa_FMBXYadHsNI78orO0U0Q=.bc33828c-eff6-4a4a-8fc3-3479e5e984f3@github.com> On Thu, 30 Mar 2023 23:43:29 GMT, William Kemper wrote: >> During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: >> * The age and affiliation of a region are encoded in the border and shape of the region (respectively). >> * Phases are encoded with different colors for different generations and whether they have degenerated. >> * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). >> * Popup windows can be opened for additional detail on regions, as well as their history. >> * The legend shows the number of regions in the state described by the legend item. >> * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. >> >> Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. > > William Kemper has updated the pull request incrementally with eight additional commits since the last revision: > > - Remove extra whitespace > - Improve handling of interrupted exception > - Use more idiomatic pattern for iterating over lines in a file > - Improve encapsulation of DataConnector's threading > - Use idiomatic case conventions for enum > - Inline usages of Color.AGE_N variables > - Reduce visibility of members to package-private where possible > - Target jdk17, update github actions Marked as reviewed by rkennke (Reviewer). ------------- PR Review: https://git.openjdk.org/shenandoah-visualizer/pull/1#pullrequestreview-1366456363 From shade at openjdk.org Fri Mar 31 08:19:05 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 31 Mar 2023 08:19:05 GMT Subject: RFR: Fix satb barrier for object array copy [v2] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 23:32:31 GMT, Kelvin Nilsen wrote: >> The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. >> >> During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Fix typo in comment I have no further comments on top of what @earthling-amzn already suggested. I'll stamp the approval once those changes are in. Maybe merge from master to get cleaner PR. ------------- PR Review: https://git.openjdk.org/shenandoah/pull/237#pullrequestreview-1366476049 From shade at openjdk.org Fri Mar 31 08:19:08 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 31 Mar 2023 08:19:08 GMT Subject: RFR: Fix satb barrier for object array copy [v2] In-Reply-To: References: Message-ID: <7bEiGmSwDbm6vBdaIWJEACZ8zQXpc3uBphUkK86XdNE=.623d2c3a-4b4f-4658-9b47-0a648b114c2a@github.com> On Thu, 30 Mar 2023 23:03:46 GMT, Kelvin Nilsen wrote: >> src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 325: >> >>> 323: >>> 324: phase4_compact_objects(worker_slices); >>> 325: >> >> This is the part of #235, is it? Should not be in this PR, I think. > > I included #235 redundantly here so that I could pass some regression tests. I've integrated #235 already, so the history shows that commit before this one... Aha! I think you can merge from master and this would be gone. I am surprised GH does not complain about this, but I guess as long as affected hunks are the same, there is no merge conflict. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1154165124 From kdnilsen at openjdk.org Fri Mar 31 15:43:23 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 15:43:23 GMT Subject: RFR: Fix satb barrier for object array copy [v3] In-Reply-To: References: Message-ID: > The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. > > During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: - Merge remote-tracking branch 'github/master' into fix-satb-barrier-for-object-array-copy - Fix typo in comment - Tidy up branch to make consistent with recompute-generation_usage PR - Integrate improvements to region-usage-accounting after full gc - Change name of phase5_restore_generation_usage - Fix comparison between array address and TAMS for old-gen collection - Merge remote-tracking branch 'GitFarmBranch/fix-satb-barrier-for-objectarray-copy' into fix-satb-barrier-for-object-array-copy - Restore generation budgets following fullgc - Fix implementation of SATB barrier for objectarray copy Original code ommitted SATB barrier for old-gen arrays that were above the old-gen TAMS. That was an error. ------------- Changes: https://git.openjdk.org/shenandoah/pull/237/files Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=237&range=02 Stats: 50 lines in 2 files changed: 43 ins; 0 del; 7 mod Patch: https://git.openjdk.org/shenandoah/pull/237.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/237/head:pull/237 PR: https://git.openjdk.org/shenandoah/pull/237 From kdnilsen at openjdk.org Fri Mar 31 15:43:25 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 15:43:25 GMT Subject: RFR: Fix satb barrier for object array copy [v3] In-Reply-To: References: Message-ID: <_5exWFqmSPgmnuNIb97Shvko8yu0fsY_5WnzCBRPbww=.dff4e069-461d-4a15-b3c6-f5634606fbe6@github.com> On Thu, 30 Mar 2023 23:22:05 GMT, William Kemper wrote: >> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: >> >> - Merge remote-tracking branch 'github/master' into fix-satb-barrier-for-object-array-copy >> - Fix typo in comment >> - Tidy up branch to make consistent with recompute-generation_usage PR >> - Integrate improvements to region-usage-accounting after full gc >> - Change name of phase5_restore_generation_usage >> - Fix comparison between array address and TAMS for old-gen collection >> - Merge remote-tracking branch 'GitFarmBranch/fix-satb-barrier-for-objectarray-copy' into fix-satb-barrier-for-object-array-copy >> - Restore generation budgets following fullgc >> - Fix implementation of SATB barrier for objectarray copy >> >> Original code ommitted SATB barrier for old-gen arrays that were above >> the old-gen TAMS. That was an error. > > src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 461: > >> 459: assert(_heap->mode()->is_generational(), "Invariant"); >> 460: ShenandoahHeapRegion* r = _heap->heap_region_containing(array_addr); >> 461: if (r->is_old() && (array_addr < _heap->marking_context()->top_at_mark_start(r))) { > > Why not use `!_heap->marking_context()->allocated_after_mark_start(array_addr)` as other branches do? Or replace other `!_heap->marking_context()->allocated_after_mark_start(array_addr)` with `array_addr < _heap->marking_context()->top_at_mark_start(r))` to avoid a second region look up? Thanks. I'll replace all. (I was initially thinking I should minimize my impact on code unrelated to my change, but agree as written makes it hard to read.) ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1154627791 From kdnilsen at openjdk.org Fri Mar 31 16:23:09 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 16:23:09 GMT Subject: RFR: Fix satb barrier for object array copy [v4] In-Reply-To: References: Message-ID: <2wuZqSMlNxsxzjKFiBec8u42ETJZvc0DlEMPz58AL5Q=.c040069e-2db8-4f64-9650-0ce7ed1f2933@github.com> > The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. > > During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: Refinements requested by reviewers ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/237/files - new: https://git.openjdk.org/shenandoah/pull/237/files/b9c401b1..422372db Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=237&range=03 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=237&range=02-03 Stats: 9 lines in 1 file changed: 4 ins; 2 del; 3 mod Patch: https://git.openjdk.org/shenandoah/pull/237.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/237/head:pull/237 PR: https://git.openjdk.org/shenandoah/pull/237 From kdnilsen at openjdk.org Fri Mar 31 16:23:12 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 16:23:12 GMT Subject: RFR: Fix satb barrier for object array copy [v3] In-Reply-To: References: Message-ID: On Fri, 31 Mar 2023 15:43:23 GMT, Kelvin Nilsen wrote: >> The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. >> >> During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. > > Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: > > - Merge remote-tracking branch 'github/master' into fix-satb-barrier-for-object-array-copy > - Fix typo in comment > - Tidy up branch to make consistent with recompute-generation_usage PR > - Integrate improvements to region-usage-accounting after full gc > - Change name of phase5_restore_generation_usage > - Fix comparison between array address and TAMS for old-gen collection > - Merge remote-tracking branch 'GitFarmBranch/fix-satb-barrier-for-objectarray-copy' into fix-satb-barrier-for-object-array-copy > - Restore generation budgets following fullgc > - Fix implementation of SATB barrier for objectarray copy > > Original code ommitted SATB barrier for old-gen arrays that were above > the old-gen TAMS. That was an error. Have merged from master to clean up redundancy. ------------- PR Comment: https://git.openjdk.org/shenandoah/pull/237#issuecomment-1492216645 From shade at openjdk.org Fri Mar 31 16:23:10 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 31 Mar 2023 16:23:10 GMT Subject: RFR: Fix satb barrier for object array copy [v4] In-Reply-To: <2wuZqSMlNxsxzjKFiBec8u42ETJZvc0DlEMPz58AL5Q=.c040069e-2db8-4f64-9650-0ce7ed1f2933@github.com> References: <2wuZqSMlNxsxzjKFiBec8u42ETJZvc0DlEMPz58AL5Q=.c040069e-2db8-4f64-9650-0ce7ed1f2933@github.com> Message-ID: On Fri, 31 Mar 2023 16:19:04 GMT, Kelvin Nilsen wrote: >> The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. >> >> During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Refinements requested by reviewers Looks fine to me! ------------- Marked as reviewed by shade (Committer). PR Review: https://git.openjdk.org/shenandoah/pull/237#pullrequestreview-1367237464 From kdnilsen at openjdk.org Fri Mar 31 16:23:16 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 16:23:16 GMT Subject: RFR: Fix satb barrier for object array copy [v4] In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 23:16:12 GMT, William Kemper wrote: >> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: >> >> Refinements requested by reviewers > > src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 446: > >> 444: * >> 445: * 1. The overwritten array resides in old-gen and it is below TAMS within its old-gen region >> 446: * 2. Do not call arraycopy_work for any array residing young-gen because young-gen collection is idle at this time > > ... array residing _in_ young-gen ... fixed > src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 476: > >> 474: } else { >> 475: T* array = src; >> 476: if (!_heap->marking_context()->allocated_after_mark_start(reinterpret_cast(array))) { > > Pull `array_addr` up and replace `reinterpret_cast(array)` with `array_addr`? done ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1154664724 PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1154664184 From kdnilsen at openjdk.org Fri Mar 31 16:23:18 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 16:23:18 GMT Subject: RFR: Fix satb barrier for object array copy [v4] In-Reply-To: <_5exWFqmSPgmnuNIb97Shvko8yu0fsY_5WnzCBRPbww=.dff4e069-461d-4a15-b3c6-f5634606fbe6@github.com> References: <_5exWFqmSPgmnuNIb97Shvko8yu0fsY_5WnzCBRPbww=.dff4e069-461d-4a15-b3c6-f5634606fbe6@github.com> Message-ID: On Fri, 31 Mar 2023 15:38:56 GMT, Kelvin Nilsen wrote: >> src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 461: >> >>> 459: assert(_heap->mode()->is_generational(), "Invariant"); >>> 460: ShenandoahHeapRegion* r = _heap->heap_region_containing(array_addr); >>> 461: if (r->is_old() && (array_addr < _heap->marking_context()->top_at_mark_start(r))) { >> >> Why not use `!_heap->marking_context()->allocated_after_mark_start(array_addr)` as other branches do? Or replace other `!_heap->marking_context()->allocated_after_mark_start(array_addr)` with `array_addr < _heap->marking_context()->top_at_mark_start(r))` to avoid a second region look up? > > Thanks. I'll replace all. (I was initially thinking I should minimize my impact on code unrelated to my change, but agree as written makes it hard to read.) done ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/237#discussion_r1154664088 From kdnilsen at openjdk.org Fri Mar 31 17:00:01 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 17:00:01 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier In-Reply-To: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> References: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> Message-ID: On Thu, 30 Mar 2023 22:37:59 GMT, William Kemper wrote: >> Global collections can create new cross-generational pointers during j.l.r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and have provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. >> >> I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb. Will run it through our internal tests pipeline too before pushing the fix. >> >> I have not yet run any performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but am open to any suggestions. >> >> Thank you for your reviews. > > src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 62: > >> 60: } >> 61: >> 62: template > > This looks similar to the code we have for dropping a reference. > https://github.com/openjdk/shenandoah/blob/2785875de3bf86ab20779c5a24ec7e0f69e452aa/src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp#L397 > > Could we use this function there? Maybe that can be done in a separate PR? (I'm not sure the commentary of this method is entirely relevant to that other situation. If it is, feel free to move forward with testing of the change... ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/238#discussion_r1153877827 From kdnilsen at openjdk.org Fri Mar 31 16:59:57 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 16:59:57 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 20:30:26 GMT, Y. Srinivas Ramakrishna wrote: > Global collections can create new cross-generational pointers during j.l.r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and have provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. > > I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb. Will run it through our internal tests pipeline too before pushing the fix. > > I have not yet run any performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but am open to any suggestions. > > Thank you for your reviews. Marked as reviewed by kdnilsen (Committer). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/238#pullrequestreview-1366060254 From kdnilsen at openjdk.org Fri Mar 31 17:50:55 2023 From: kdnilsen at openjdk.org (Kelvin Nilsen) Date: Fri, 31 Mar 2023 17:50:55 GMT Subject: Integrated: Fix satb barrier for object array copy In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 23:54:25 GMT, Kelvin Nilsen wrote: > The satb barrier for overwritten arrays residing in old-gen must be unconditional during young GC. > > During old GC, the satb barrier is only needed for overwritten old-gen arrays if the array is below TAMS within its old-gen region. This pull request has now been integrated. Changeset: ba4161de Author: Kelvin Nilsen URL: https://git.openjdk.org/shenandoah/commit/ba4161de7687db8ab225a0bbff52e56b1dbefbec Stats: 52 lines in 2 files changed: 45 ins; 0 del; 7 mod Fix satb barrier for object array copy Reviewed-by: shade ------------- PR: https://git.openjdk.org/shenandoah/pull/237 From wkemper at openjdk.org Fri Mar 31 18:13:58 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 18:13:58 GMT Subject: RFR: Abandon mixed collections if all candidates are pinned [v4] In-Reply-To: References: Message-ID: > A new `WAITING_FOR_FILL` state has been added to the old generation. This state is now entered when all the candidates for a mixed collection cycle are found to be pinned (which should be quite rare). This state permits a new old generation collection cycle to begin (which itself begins with making these regions parseable). > > This change also adds a new diagnostic flag `ShenandoahCoalesceChance` which is the probability (expressed as a percentage) of abandoning all the candidates of a mixed collection. Any abandoned candidate regions will be made parseable. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Fix typos in comments, update ascii state diagram ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/236/files - new: https://git.openjdk.org/shenandoah/pull/236/files/22b70629..7dddc164 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=236&range=03 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=236&range=02-03 Stats: 32 lines in 2 files changed: 7 ins; 0 del; 25 mod Patch: https://git.openjdk.org/shenandoah/pull/236.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/236/head:pull/236 PR: https://git.openjdk.org/shenandoah/pull/236 From wkemper at openjdk.org Fri Mar 31 18:27:45 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 18:27:45 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v8] In-Reply-To: References: Message-ID: > During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: > * The age and affiliation of a region are encoded in the border and shape of the region (respectively). > * Phases are encoded with different colors for different generations and whether they have degenerated. > * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). > * Popup windows can be opened for additional detail on regions, as well as their history. > * The legend shows the number of regions in the state described by the legend item. > * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. > > Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Upgrade actions and os to address deprecation warnings ------------- Changes: - all: https://git.openjdk.org/shenandoah-visualizer/pull/1/files - new: https://git.openjdk.org/shenandoah-visualizer/pull/1/files/5d380c30..2d78f967 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=07 - incr: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=06-07 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.org/shenandoah-visualizer/pull/1.diff Fetch: git fetch https://git.openjdk.org/shenandoah-visualizer.git pull/1/head:pull/1 PR: https://git.openjdk.org/shenandoah-visualizer/pull/1 From wkemper at openjdk.org Fri Mar 31 18:43:49 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 18:43:49 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v9] In-Reply-To: References: Message-ID: <6JO2FEtqvxOsLU8_rUk0AFLvtuu3AaL3iVQnS9GP5Xg=.9dc42548-9af5-43a4-ae61-033c5e2b1b6c@github.com> > During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: > * The age and affiliation of a region are encoded in the border and shape of the region (respectively). > * Phases are encoded with different colors for different generations and whether they have degenerated. > * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). > * Popup windows can be opened for additional detail on regions, as well as their history. > * The legend shows the number of regions in the state described by the legend item. > * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. > > Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Use corretto distribution for java-setup action ------------- Changes: - all: https://git.openjdk.org/shenandoah-visualizer/pull/1/files - new: https://git.openjdk.org/shenandoah-visualizer/pull/1/files/2d78f967..07009d32 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=08 - incr: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=07-08 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.org/shenandoah-visualizer/pull/1.diff Fetch: git fetch https://git.openjdk.org/shenandoah-visualizer.git pull/1/head:pull/1 PR: https://git.openjdk.org/shenandoah-visualizer/pull/1 From wkemper at openjdk.org Fri Mar 31 18:49:46 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 18:49:46 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v10] In-Reply-To: References: Message-ID: > During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: > * The age and affiliation of a region are encoded in the border and shape of the region (respectively). > * Phases are encoded with different colors for different generations and whether they have degenerated. > * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). > * Popup windows can be opened for additional detail on regions, as well as their history. > * The legend shows the number of regions in the state described by the legend item. > * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. > > Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Revert back to java-setup at v1 to test 21-ea ------------- Changes: - all: https://git.openjdk.org/shenandoah-visualizer/pull/1/files - new: https://git.openjdk.org/shenandoah-visualizer/pull/1/files/07009d32..15b65ea7 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=09 - incr: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=08-09 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.org/shenandoah-visualizer/pull/1.diff Fetch: git fetch https://git.openjdk.org/shenandoah-visualizer.git pull/1/head:pull/1 PR: https://git.openjdk.org/shenandoah-visualizer/pull/1 From wkemper at openjdk.org Fri Mar 31 18:59:36 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 18:59:36 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v11] In-Reply-To: References: Message-ID: > During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: > * The age and affiliation of a region are encoded in the border and shape of the region (respectively). > * Phases are encoded with different colors for different generations and whether they have degenerated. > * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). > * Popup windows can be opened for additional detail on regions, as well as their history. > * The legend shows the number of regions in the state described by the legend item. > * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. > > Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. William Kemper has updated the pull request incrementally with one additional commit since the last revision: Okay, version 3 of setup-java using the same distribution as version 1 ------------- Changes: - all: https://git.openjdk.org/shenandoah-visualizer/pull/1/files - new: https://git.openjdk.org/shenandoah-visualizer/pull/1/files/15b65ea7..c753414c Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=10 - incr: https://webrevs.openjdk.org/?repo=shenandoah-visualizer&pr=1&range=09-10 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.org/shenandoah-visualizer/pull/1.diff Fetch: git fetch https://git.openjdk.org/shenandoah-visualizer.git pull/1/head:pull/1 PR: https://git.openjdk.org/shenandoah-visualizer/pull/1 From shade at openjdk.org Fri Mar 31 18:59:38 2023 From: shade at openjdk.org (Aleksey Shipilev) Date: Fri, 31 Mar 2023 18:59:38 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v7] In-Reply-To: References: Message-ID: On Fri, 31 Mar 2023 08:03:18 GMT, Aleksey Shipilev wrote: > I don't see GHA runs for this, could you go to https://github.com/earthling-amzn/shenandoah-visualizer/actions and enable them? You'd probably need to submit the workflow run on your `earthling-amzn:genshen-visualizer` branch manually. Bots should catch up with the results then,. Apparently GHA results would not show up in this PR until this configuration change: https://bugs.openjdk.org/browse/SKARA-1869 Just check it is green in https://github.com/earthling-amzn/shenandoah-visualizer/actions. ------------- PR Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#issuecomment-1492447912 From wkemper at openjdk.org Fri Mar 31 19:05:56 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 19:05:56 GMT Subject: RFR: Enhancements for Shenandoah's generational mode [v11] In-Reply-To: References: Message-ID: On Fri, 31 Mar 2023 18:59:36 GMT, William Kemper wrote: >> During the course of the development of the generational mode for Shenandoah, we have found this tool useful to demonstrate progress and changes in behavior. We have also found this tool useful for troubleshooting performance issues and debugging crashes. There are many changes here, but these are the highlights: >> * The age and affiliation of a region are encoded in the border and shape of the region (respectively). >> * Phases are encoded with different colors for different generations and whether they have degenerated. >> * A mechanism to record and replay session has been added (record feature is implemented in hotspot and is not yet upstream). >> * Popup windows can be opened for additional detail on regions, as well as their history. >> * The legend shows the number of regions in the state described by the legend item. >> * Visualizer can now 'find' VMs running Shenandoah with region sampling enabled. >> >> Many months ago we broke backward compatibility on our branch, but we have recently restored it so the time seems right for a PR. Thank you for looking and sorry for the massive number of changes. > > William Kemper has updated the pull request incrementally with one additional commit since the last revision: > > Okay, version 3 of setup-java using the same distribution as version 1 Yes, I ran the actions and updated the configuration to eliminate GHA deprecation warnings. ------------- PR Comment: https://git.openjdk.org/shenandoah-visualizer/pull/1#issuecomment-1492457554 From wkemper at openjdk.org Fri Mar 31 19:23:04 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 19:23:04 GMT Subject: Integrated: Abandon mixed collections if all candidates are pinned In-Reply-To: References: Message-ID: On Wed, 29 Mar 2023 21:57:43 GMT, William Kemper wrote: > A new `WAITING_FOR_FILL` state has been added to the old generation. This state is now entered when all the candidates for a mixed collection cycle are found to be pinned (which should be quite rare). This state permits a new old generation collection cycle to begin (which itself begins with making these regions parseable). > > This change also adds a new diagnostic flag `ShenandoahCoalesceChance` which is the probability (expressed as a percentage) of abandoning all the candidates of a mixed collection. Any abandoned candidate regions will be made parseable. This pull request has now been integrated. Changeset: f75b50b6 Author: William Kemper URL: https://git.openjdk.org/shenandoah/commit/f75b50b61e5e69e23ce2361ba4e22ef88060fd69 Stats: 119 lines in 9 files changed: 83 ins; 4 del; 32 mod Abandon mixed collections if all candidates are pinned Reviewed-by: kdnilsen, ysr ------------- PR: https://git.openjdk.org/shenandoah/pull/236 From wkemper at openjdk.org Fri Mar 31 19:33:02 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 19:33:02 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier In-Reply-To: References: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> Message-ID: On Fri, 31 Mar 2023 02:31:06 GMT, Y. Srinivas Ramakrishna wrote: >> A visual examination of the card-scanning code reveals that the card-mark in the portion of code you pointed out is redundant because the card-scanning code always dirties the card containing a cross-generational pointer, irrespective of whether it's a weak or strong pointer. I'll work on eliding that unnecessary check and mark in a separate PR as I want to add some assertions and check it carefully. >> >> Thanks for drawing my attention to this code which I had just glossed over before. >> >> The diffs in my original PR stay unchanged and I'll not try to do anything about sharing the code, since I expect that piece of card-marking code in `drop()` to go away in the PR I promised above, for which I'll open a ticket momentarily. > > I opened https://bugs.openjdk.org/browse/JDK-8305335 to follow up on that. Okay, thanks for taking a closer look. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/238#discussion_r1154821675 From wkemper at openjdk.org Fri Mar 31 19:33:00 2023 From: wkemper at openjdk.org (William Kemper) Date: Fri, 31 Mar 2023 19:33:00 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier In-Reply-To: References: Message-ID: <6HHSbhXtcyPVtnwjA03JBC7XkmP97d5EL0aIILIPOcU=.97efaf9b-456d-4714-b79d-875d59a60404@github.com> On Thu, 30 Mar 2023 20:30:26 GMT, Y. Srinivas Ramakrishna wrote: > Global collections can create new cross-generational pointers during j.l.r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and have provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. > > I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb. Will run it through our internal tests pipeline too before pushing the fix. > > I have not yet run any performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but am open to any suggestions. > > Thank you for your reviews. Marked as reviewed by wkemper (Committer). ------------- PR Review: https://git.openjdk.org/shenandoah/pull/238#pullrequestreview-1367481200 From ysr at openjdk.org Fri Mar 31 20:55:53 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 20:55:53 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier [v2] In-Reply-To: References: Message-ID: > Global collections can create new cross-generational pointers during j.l.r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. > > I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb, and through our internal tests pipelines. > > I have not run any specific performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but I do not expect much impact. Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: - Merge branch 'master' into ref_proc_card_mark_fix - Add documentation comment & some assertions. - When (generational) GC discovers j.l.Reference:s in both generations, then subsequent processing of the discovered list may create new old to young pointers which the young collector needs to know about. This fix takes care of marking cards for that case. The issue was revealed during a verification failure and subsequently verified by means of a check in the reference processing code to detect the condition when the new cross generational pointer was being created without the associated card-marking. This typically happens when discovery is in both generations -- for young collections such Reference objects in the old generation are discovered during the card scanning, and for global collections when objects are marked. The situation is typically rare in the case of young collections because the Reference objects typically lie on dirty cards and hold pointers to referents in the young generation. - Shelve for now. May not compile. ------------- Changes: - all: https://git.openjdk.org/shenandoah/pull/238/files - new: https://git.openjdk.org/shenandoah/pull/238/files/cca3d009..4e6fb943 Webrevs: - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=238&range=01 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=238&range=00-01 Stats: 13435 lines in 586 files changed: 7831 ins; 2905 del; 2699 mod Patch: https://git.openjdk.org/shenandoah/pull/238.diff Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/238/head:pull/238 PR: https://git.openjdk.org/shenandoah/pull/238 From ysr at openjdk.org Fri Mar 31 20:55:53 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 20:55:53 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier [v2] In-Reply-To: References: Message-ID: On Fri, 31 Mar 2023 16:57:29 GMT, Kelvin Nilsen wrote: >> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision: >> >> - Merge branch 'master' into ref_proc_card_mark_fix >> - Add documentation comment & some assertions. >> - When (generational) GC discovers j.l.Reference:s in both generations, then subsequent >> processing of the discovered list may create new old to young pointers >> which the young collector needs to know about. This fix takes care of >> marking cards for that case. >> >> The issue was revealed during a verification failure and subsequently >> verified by means of a check in the reference processing code to detect >> the condition when the new cross generational pointer was being created >> without the associated card-marking. This typically happens when >> discovery is in both generations -- for young collections such Reference >> objects in the old generation are discovered during the card scanning, >> and for global collections when objects are marked. The situation is >> typically rare in the case of young collections because the Reference >> objects typically lie on dirty cards and hold pointers to referents in >> the young generation. >> - Shelve for now. May not compile. > > Marked as reviewed by kdnilsen (Committer). Thanks @kdnilsen & @earthling-amzn : Could one of you /sponsor then I can land the change. Thanks! ------------- PR Comment: https://git.openjdk.org/shenandoah/pull/238#issuecomment-1492589524 From ysr at openjdk.org Fri Mar 31 20:55:53 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 20:55:53 GMT Subject: RFR: JDK-8305334: GenShen: reference processing needs a card-marking barrier [v2] In-Reply-To: References: <9inWmzN0JJmvz2xArwjqqRihx0Tmb9RXpTsLA6tFQPg=.a30118cd-edfc-4631-b6f0-c4374ff95369@github.com> Message-ID: <7EFcaKlVEOqpjjyTyzL4NF_zIj9DI6P0qaFAoOX1gKs=.fb2e62be-5411-4f58-a239-cf635b782d65@github.com> On Fri, 31 Mar 2023 19:30:10 GMT, William Kemper wrote: >> I opened https://bugs.openjdk.org/browse/JDK-8305335 to follow up on that. > > Okay, thanks for taking a closer look. > Maybe that can be done in a separate PR? (I'm not sure the commentary of this method is entirely relevant to that other situation. If it is, feel free to move forward with testing of the change... Correct; we'll follow up separately on that in a separate PR & ticket, linked i comment above. ------------- PR Review Comment: https://git.openjdk.org/shenandoah/pull/238#discussion_r1154884652 From ysr at openjdk.org Fri Mar 31 20:58:07 2023 From: ysr at openjdk.org (Y. Srinivas Ramakrishna) Date: Fri, 31 Mar 2023 20:58:07 GMT Subject: Integrated: JDK-8305334: GenShen: reference processing needs a card-marking barrier In-Reply-To: References: Message-ID: On Thu, 30 Mar 2023 20:30:26 GMT, Y. Srinivas Ramakrishna wrote: > Global collections can create new cross-generational pointers during j.l.r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. > > I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb, and through our internal tests pipelines. > > I have not run any specific performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but I do not expect much impact. This pull request has now been integrated. Changeset: d0e1f519 Author: Y. Srinivas Ramakrishna Committer: Kelvin Nilsen URL: https://git.openjdk.org/shenandoah/commit/d0e1f5195315019467419dfd377d440026bbb87e Stats: 28 lines in 1 file changed: 28 ins; 0 del; 0 mod 8305334: GenShen: reference processing needs a card-marking barrier Global collections can create new cross-generational pointers during j.l.r.Reference processing which should be added to the card marking remembered set. The issue was found with dacapo during heap verification and happens somewhat rarely. I added the requisite barrier and provided a comment describing the sole situation in reference processing that should need the barrier. Assertions check this condition, but the card is also redundantly dirtied for young collections too where it's strictly not needed. I have tested with and without heap verification using product and fastdebug builds using dacapo and specjbb, and through our internal tests pipelines. I have not run any specific performance comparisons to assess the impact of the checks for applications that traffic heavily in j.l.r.Refs, but I do not expect much impact. Reviewed-by: kdnilsen, wkemper ------------- PR: https://git.openjdk.org/shenandoah/pull/238