From gnu.andrew at redhat.com Thu Jul 1 05:26:55 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Thu, 1 Jul 2021 06:26:55 +0100 Subject: [RFR] [8u] 8u302-b04 Upstream Sync Message-ID: <20210701052655.GA1922384@rincewind> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/ Merge changesets: http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/corba/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/jaxp/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/jaxws/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/jdk/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/hotspot/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/langtools/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/nashorn/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/root/merge.changeset Changes in aarch64-shenandoah-jdk8u302-b04: - JDK-8032050: Clean up for java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java - JDK-8043264: hsdis library not picked up correctly on expected paths - JDK-8130430: [TEST_BUG] remove unnecessary internal calls from javax/swing/JRadioButton/8075609/bug8075609.java - JDK-8206243: java -XshowSettings fails if memory.limit_in_bytes overflows LONG.max - JDK-8206925: Support the certificate_authorities extension - JDK-8228757: Fail fast if the handshake type is unknown - JDK-8242565: Policy initialization issues when the denyAfter constraint is enabled - JDK-8253375: OSX build fails with Xcode 12.0 (12A7209) - JDK-8257039: [8u] GenericTaskQueue destructor is incorrect - JDK-8262446: DragAndDrop hangs on Windows - JDK-8265666: Enable AIX build platform to make external debug symbols Main issues of note: None, clean merge. diffstat for root b/.hgtags | 1 + b/common/autoconf/generated-configure.sh | 13 ++++--------- b/common/autoconf/jdk-options.m4 | 11 +++-------- b/make/common/NativeCompilation.gmk | 13 ++++++++++++- 4 files changed, 20 insertions(+), 18 deletions(-) diffstat for corba b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxp b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxws b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for langtools b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for nashorn b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jdk b/.hgtags | 1 b/src/linux/classes/jdk/internal/platform/cgroupv1/SubSystem.java | 14 b/src/share/classes/sun/security/jca/Providers.java | 3 b/src/share/classes/sun/security/ssl/CertificateAuthoritiesExtension.java | 406 ++++++++++ b/src/share/classes/sun/security/ssl/CertificateRequest.java | 20 b/src/share/classes/sun/security/ssl/SSLEngineInputRecord.java | 9 b/src/share/classes/sun/security/ssl/SSLExtension.java | 66 + b/src/share/classes/sun/security/ssl/SSLHandshake.java | 10 b/src/share/classes/sun/security/ssl/SSLSocketInputRecord.java | 9 b/src/share/classes/sun/security/ssl/X509Authentication.java | 16 b/src/share/classes/sun/security/tools/KeyStoreUtil.java | 12 b/src/share/classes/sun/security/tools/jarsigner/Main.java | 5 b/src/share/classes/sun/security/tools/keytool/Main.java | 3 b/src/windows/native/sun/windows/awt_DnDDT.cpp | 9 b/test/java/rmi/activation/Activatable/shutdownGracefully/ShutdownGracefully.java | 134 --- b/test/java/rmi/testlibrary/JavaVM.java | 35 b/test/java/security/Policy/SignedJar/SignedJarTest.java | 6 b/test/java/security/Policy/SignedJar/java.security | 3 b/test/javax/net/ssl/templates/SSLContextTemplate.java | 208 +++-- b/test/javax/swing/JRadioButton/8075609/bug8075609.java | 9 b/test/sun/security/ssl/X509KeyManager/CertificateAuthorities.java | 92 ++ b/test/sun/security/ssl/X509TrustManagerImpl/CacertsLimit.java | 87 ++ b/test/sun/security/ssl/X509TrustManagerImpl/TooManyCAs.java | 282 ++++++ 23 files changed, 1218 insertions(+), 221 deletions(-) diffstat for hotspot b/.hgtags | 1 b/make/aix/makefiles/defs.make | 88 ++++++++++++++----------------- b/make/aix/makefiles/jsig.make | 20 +++---- b/make/aix/makefiles/saproc.make | 22 +++---- b/make/aix/makefiles/vm.make | 20 +++---- b/src/share/vm/compiler/disassembler.cpp | 4 - b/src/share/vm/runtime/sharedRuntime.cpp | 4 - b/src/share/vm/utilities/taskqueue.hpp | 4 - 8 files changed, 79 insertions(+), 84 deletions(-) Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero), ppc64, ppc64le, aarch32 (Zero) & aarch64. Ok to push? Thanks, -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From gnu.andrew at redhat.com Thu Jul 1 05:33:47 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Thu, 1 Jul 2021 06:33:47 +0100 Subject: RFR/RFC: Shenandoah 8u integration 2021-06-23 In-Reply-To: <888b18e4-629d-d490-0694-1a734ecad069@redhat.com> References: <7b5710ff-2ca0-7aec-f5f3-fa1f9f2b172b@redhat.com> <20210623153053.GC782669@rincewind> <888b18e4-629d-d490-0694-1a734ecad069@redhat.com> Message-ID: <20210701053347.GA1924403@rincewind> On 17:32 Wed 23 Jun , Aleksey Shipilev wrote: > On 6/23/21 5:30 PM, Andrew Hughes wrote: > > Please merge and I'll rebase the b04 I have on top of this. > > Thanks! Pushed. > > -- > -Aleksey > Thanks. Built fine without any issues on all archs :) -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From shade at redhat.com Thu Jul 1 07:47:28 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 1 Jul 2021 09:47:28 +0200 Subject: [RFR] [8u] 8u302-b04 Upstream Sync In-Reply-To: <20210701052655.GA1922384@rincewind> References: <20210701052655.GA1922384@rincewind> Message-ID: <9b8612e5-3f29-9648-6f83-485d921bfa66@redhat.com> On 7/1/21 7:26 AM, Andrew Hughes wrote: > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/corba/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/jaxp/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/jaxws/merge.changeset Look trivially fine. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/jdk/merge.changeset Looks fine. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/hotspot/merge.changeset I had doubts about GenericTaskQueue change, but I think this is fine for 8u. Commented in the bug itself: https://bugs.openjdk.java.net/browse/JDK-8257039 > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/langtools/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/nashorn/merge.changeset Look trivially fine. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b04/root/merge.changeset Looks fine. > Ok to push? Yes. -- Thanks, -Aleksey From hseigel at openjdk.java.net Thu Jul 1 13:12:16 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Thu, 1 Jul 2021 13:12:16 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE Message-ID: Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. Thanks, Harold ------------- Commit messages: - 8244162: Additional opportunities to use NONCOPYABLE Changes: https://git.openjdk.java.net/jdk/pull/4652/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4652&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8244162 Stats: 21 lines in 6 files changed: 6 ins; 9 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/4652.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4652/head:pull/4652 PR: https://git.openjdk.java.net/jdk/pull/4652 From yyang at openjdk.java.net Fri Jul 2 01:47:17 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Fri, 2 Jul 2021 01:47:17 GMT Subject: RFR: 8269672: C1: Remove unaligned move on all architectures Message-ID: This is a follow up PR for #3917, no functional change. As @TheRealMDoerr said The "unaligned" parameter is unused on all platforms, now. Note that PPC64 and s390 support unaligned accesses, too. I believe it was used by SPARC which has been removed. Thanks? ------------- Commit messages: - remove unaligned Changes: https://git.openjdk.java.net/jdk/pull/4662/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4662&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8269672 Stats: 66 lines in 19 files changed: 0 ins; 27 del; 39 mod Patch: https://git.openjdk.java.net/jdk/pull/4662.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4662/head:pull/4662 PR: https://git.openjdk.java.net/jdk/pull/4662 From shade at openjdk.java.net Fri Jul 2 09:20:35 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 2 Jul 2021 09:20:35 GMT Subject: RFR: 8261492: Shenandoah: reconsider forwardee accesses memory ordering [v6] In-Reply-To: References: Message-ID: <8wTsKKtuOS-OWOMgEvZD8Z1kql-ncYNA7-iYl11pDk8=.e870a1f2-58f6-407b-b9fb-391dffe0c6da@github.com> > Shenandoah carries forwardee information in object's mark word. Installing the new mark word is effectively "releasing" the object copy, and reading from the new mark word is "acquiring" that object copy. > > For the forwardee update side, Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. This seems to be excessive for Shenandoah forwardee updates, and "release" is enough. > > For the forwardee load side, we need to guarantee "acquire". We do not do it now, reading the markword without memory semantics. It does not seem to pose a practical problem today, because GC does not access the object contents in the new copy, and mutators get this from the JRT-called stub that separates the fwdptr access and object contents access by a lot. It still should be cleaner to "acquire" the mark on load to avoid surprises. > > Additional testing: > - [x] Linux x86_64 `hotspot_gc_shenandoah` > - [x] Linux AArch64 `hotspot_gc_shenandoah` > - [x] Linux AArch64 `tier1` with Shenandoah Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: - Merge branch 'master' into JDK-8261492-shenandoah-forwardee-memord - 8261492: Shenandoah: reconsider forwardee accesses memory ordering ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2496/files - new: https://git.openjdk.java.net/jdk/pull/2496/files/36e2da27..4953a6dd Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2496&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2496&range=04-05 Stats: 20377 lines in 568 files changed: 8720 ins; 9594 del; 2063 mod Patch: https://git.openjdk.java.net/jdk/pull/2496.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2496/head:pull/2496 PR: https://git.openjdk.java.net/jdk/pull/2496 From zgu at openjdk.java.net Fri Jul 2 14:25:16 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 2 Jul 2021 14:25:16 GMT Subject: RFR: 8261492: Shenandoah: reconsider forwardee accesses memory ordering [v6] In-Reply-To: <8wTsKKtuOS-OWOMgEvZD8Z1kql-ncYNA7-iYl11pDk8=.e870a1f2-58f6-407b-b9fb-391dffe0c6da@github.com> References: <8wTsKKtuOS-OWOMgEvZD8Z1kql-ncYNA7-iYl11pDk8=.e870a1f2-58f6-407b-b9fb-391dffe0c6da@github.com> Message-ID: On Fri, 2 Jul 2021 09:20:35 GMT, Aleksey Shipilev wrote: >> Shenandoah carries forwardee information in object's mark word. Installing the new mark word is effectively "releasing" the object copy, and reading from the new mark word is "acquiring" that object copy. >> >> For the forwardee update side, Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. This seems to be excessive for Shenandoah forwardee updates, and "release" is enough. >> >> For the forwardee load side, we need to guarantee "acquire". We do not do it now, reading the markword without memory semantics. It does not seem to pose a practical problem today, because GC does not access the object contents in the new copy, and mutators get this from the JRT-called stub that separates the fwdptr access and object contents access by a lot. It still should be cleaner to "acquire" the mark on load to avoid surprises. >> >> Additional testing: >> - [x] Linux x86_64 `hotspot_gc_shenandoah` >> - [x] Linux AArch64 `hotspot_gc_shenandoah` >> - [x] Linux AArch64 `tier1` with Shenandoah > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains two additional commits since the last revision: > > - Merge branch 'master' into JDK-8261492-shenandoah-forwardee-memord > - 8261492: Shenandoah: reconsider forwardee accesses memory ordering Still good. ------------- Marked as reviewed by zgu (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/2496 From dholmes at openjdk.java.net Mon Jul 5 01:54:53 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Mon, 5 Jul 2021 01:54:53 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Thu, 1 Jul 2021 13:06:55 GMT, Harold Seigel wrote: > Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Hi Harold, That all seems quite reasonable. One query below about private constructors. Thanks, David src/hotspot/share/classfile/stackMapTableFormat.hpp line 47: > 45: > 46: protected: > 47: // No constructors - should be 'private', but GCC issues a warning if it is Have we checked this is still the case? Might be a very old gcc issue. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4652 From gnu.andrew at redhat.com Mon Jul 5 04:27:28 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Mon, 5 Jul 2021 05:27:28 +0100 Subject: [RFR] [8u] 8u302-b05 Upstream Sync Message-ID: <20210705042728.GA2523975@rincewind> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/ Merge changesets: http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/corba/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jaxp/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jaxws/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jdk/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/hotspot/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/langtools/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/nashorn/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/root/merge.changeset Changes in aarch64-shenandoah-jdk8u302-b05: - JDK-6990210: [TEST_BUG] EventDispatchThread/HandleExceptionOnEDT/HandleExceptionOnEDT.java fails on gnome - JDK-7106851: Test should not use System.exit - JDK-8028618: [TEST BUG] javax/swing/JScrollBar/bug4202954/bug4202954.java fails - JDK-8035000: clean up ActivationLibrary.DestroyThread - JDK-8035287: gcc warnings compiling various libraries files - JDK-8037825: Fix warnings and enable "warnings as errors" in serviceability native libraries - JDK-8043646: libosxapp.dylib fails to build on Mac OS 10.9 with clang - JDK-8047939: [TESTBUG] Rewrite test/runtime/8001071/Test8001071.sh - JDK-8074835: Resolve disabled warnings for libj2gss - JDK-8074836: Resolve disabled warnings for libosxkrb5 - JDK-8078855: [TEST_BUG] javax/swing/JComboBox/8032878/bug8032878.java fails in WindowsClassicLookAndFeel - JDK-8081764: [TEST_BUG] Test javax/swing/plaf/aqua/CustomComboBoxFocusTest.java fails on Windows, Solaris Sparcv9 and Linux but passes on MacOSX - JDK-8172188: JDI tests fail due to "permission denied" when creating temp file - JDK-8196092: javax/swing/JComboBox/8032878/bug8032878.java fails - JDK-8202299: Java Keystore fails to load PKCS12/PFX certificates created in WindowsServer2016 - JDK-8239053: [8u] clean up undefined-var-template warnings - JDK-8239400: [8u] clean up undefined-var-template warnings - JDK-8249142: java/awt/FontClass/CreateFont/DeleteFont.sh is unstable - JDK-8250876: Fix issues with cross-compile on macos - JDK-8254631: Better support ALPN byte wire values in SunJSSE - JDK-8265462: Handle multiple slots in the NSS Internal Module from SunPKCS11's Secmod - JDK-8266723: JFR periodic events are causing extra allocations - JDK-8266929: Unable to use algorithms from 3p providers - JDK-8267235: [macos_aarch64] InterpreterRuntime::throw_pending_exception messing up LR results in crash - JDK-8267426: MonitorVmStartTerminate test timed out on Embedded VM - JDK-8267689: [aarch64] Crash due to bad shift in indirect addressing mode Main issues of note: None. Just one minor merge conflict due to existing include in aarch64.ad. diffstat for root b/.hgtags | 1 b/common/autoconf/generated-configure.sh | 775 +++++++++++++++++++++++-------- b/common/autoconf/toolchain.m4 | 23 3 files changed, 599 insertions(+), 200 deletions(-) diffstat for corba b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxp b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxws b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for langtools b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for nashorn b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jdk b/.hgtags | 1 b/make/lib/SecurityLibraries.gmk | 2 b/make/lib/ServiceabilityLibraries.gmk | 20 b/src/macosx/native/sun/osxapp/ThreadUtilities.m | 2 b/src/share/back/SDE.c | 8 b/src/share/back/eventHandler.c | 4 b/src/share/back/log_messages.c | 21 b/src/share/classes/jdk/jfr/internal/JVM.java | 10 b/src/share/classes/jdk/jfr/internal/Logger.java | 4 b/src/share/classes/jdk/jfr/internal/RequestEngine.java | 8 b/src/share/classes/sun/security/pkcs11/Secmod.java | 28 b/src/share/classes/sun/security/pkcs12/PKCS12KeyStore.java | 100 +-- b/src/share/classes/sun/security/ssl/AlpnExtension.java | 37 - b/src/share/classes/sun/security/x509/AlgorithmId.java | 9 b/src/share/instrument/InvocationAdapter.c | 2 b/src/share/instrument/PathCharsValidator.c | 10 b/src/share/lib/security/java.security-aix | 10 b/src/share/lib/security/java.security-linux | 10 b/src/share/lib/security/java.security-macosx | 10 b/src/share/lib/security/java.security-solaris | 10 b/src/share/lib/security/java.security-windows | 10 b/src/share/native/com/sun/java/util/jar/pack/bands.cpp | 10 b/src/share/native/sun/java2d/opengl/OGLContext.c | 2 b/src/share/native/sun/security/jgss/wrapper/GSSLibStub.c | 4 b/src/share/native/sun/security/krb5/nativeccache.c | 2 b/src/share/native/sun/security/pkcs11/j2secmod.c | 14 b/src/share/native/sun/security/pkcs11/j2secmod.h | 152 ++++ b/src/share/native/sun/security/pkcs11/wrapper/pkcs11t.h | 2 b/src/solaris/back/util_md.h | 14 b/src/solaris/native/sun/awt/awt_Font.c | 5 b/src/solaris/native/sun/java2d/x11/X11SurfaceData.c | 2 b/src/solaris/native/sun/management/MacosxOperatingSystem.c | 1 b/src/solaris/native/sun/xawt/XToolkit.c | 2 b/src/windows/back/linker_md.c | 1 b/src/windows/back/proc_md.h | 3 b/test/com/sun/jdi/ShellScaffold.sh | 10 b/test/java/awt/EventDispatchThread/HandleExceptionOnEDT/HandleExceptionOnEDT.java | 12 b/test/java/awt/FontClass/CreateFont/DeleteFont.java | 24 b/test/java/awt/FontClass/CreateFont/DeleteFont.sh | 3 b/test/java/rmi/activation/Activatable/checkActivateRef/CheckActivateRef.java | 4 b/test/java/rmi/activation/Activatable/checkAnnotations/CheckAnnotations.java | 4 b/test/java/rmi/activation/Activatable/checkImplClassLoader/CheckImplClassLoader.java | 4 b/test/java/rmi/activation/Activatable/checkRegisterInLog/CheckRegisterInLog.java | 4 b/test/java/rmi/activation/Activatable/createPrivateActivable/CreatePrivateActivatable.java | 4 b/test/java/rmi/activation/Activatable/downloadParameterClass/DownloadParameterClass.java | 4 b/test/java/rmi/activation/Activatable/elucidateNoSuchMethod/ElucidateNoSuchMethod.java | 4 b/test/java/rmi/activation/Activatable/extLoadedImpl/ExtLoadedImplTest.java | 4 b/test/java/rmi/activation/Activatable/forceLogSnapshot/ForceLogSnapshot.java | 4 b/test/java/rmi/activation/Activatable/inactiveGroup/InactiveGroup.java | 4 b/test/java/rmi/activation/Activatable/lookupActivationSystem/LookupActivationSystem.java | 4 b/test/java/rmi/activation/Activatable/nestedActivate/NestedActivate.java | 4 b/test/java/rmi/activation/Activatable/nonExistentActivatable/NonExistentActivatable.java | 4 b/test/java/rmi/activation/Activatable/restartCrashedService/RestartCrashedService.java | 4 b/test/java/rmi/activation/Activatable/restartLatecomer/RestartLatecomer.java | 4 b/test/java/rmi/activation/Activatable/restartService/RestartService.java | 4 b/test/java/rmi/activation/Activatable/unregisterInactive/UnregisterInactive.java | 4 b/test/java/rmi/activation/ActivateFailedException/activateFails/ActivateFails.java | 4 b/test/java/rmi/activation/ActivationGroup/downloadActivationGroup/DownloadActivationGroup.java | 4 b/test/java/rmi/activation/ActivationSystem/activeGroup/IdempotentActiveGroup.java | 4 b/test/java/rmi/activation/ActivationSystem/modifyDescriptor/ModifyDescriptor.java | 4 b/test/java/rmi/activation/ActivationSystem/stubClassesPermitted/StubClassesPermitted.java | 4 b/test/java/rmi/activation/ActivationSystem/unregisterGroup/UnregisterGroup.java | 4 b/test/java/rmi/activation/CommandEnvironment/SetChildEnv.java | 4 b/test/java/rmi/activation/rmidViaInheritedChannel/InheritedChannelNotServerSocket.java | 4 b/test/java/rmi/activation/rmidViaInheritedChannel/RmidViaInheritedChannel.java | 4 b/test/java/rmi/registry/altSecurityManager/AltSecurityManager.java | 7 b/test/java/rmi/server/RMISocketFactory/useSocketFactory/activatable/UseCustomSocketFactory.java | 4 b/test/java/rmi/testlibrary/ActivationLibrary.java | 64 -- b/test/java/rmi/testlibrary/RMID.java | 212 +++--- b/test/javax/accessibility/6192422/bug6192422.java | 99 +++ b/test/javax/swing/JComboBox/8032878/bug8032878.java | 31 b/test/javax/swing/JScrollBar/bug4202954/bug4202954.java | 32 - b/test/javax/swing/plaf/aqua/CustomComboBoxFocusTest.java | 9 b/test/sun/jvmstat/monitor/MonitoredVm/MonitorVmStartTerminate.java | 1 b/test/sun/security/pkcs12/EmptyPassword.java | 66 ++ b/test/sun/security/ssl/ALPN/AlpnGreaseTest.java | 314 ++++++++++ 76 files changed, 1102 insertions(+), 420 deletions(-) diffstat for hotspot a/test/runtime/8001071/Test8001071.java | 45 -------------- a/test/runtime/8001071/Test8001071.sh | 63 -------------------- b/.hgtags | 1 b/make/bsd/makefiles/saproc.make | 6 + b/src/cpu/aarch64/vm/aarch64.ad | 47 +++++++++++++- b/src/cpu/aarch64/vm/macroAssembler_aarch64.cpp | 5 + b/src/os/aix/vm/decoder_aix.hpp | 2 b/src/os/bsd/vm/decoder_machO.hpp | 2 b/src/os/windows/vm/decoder_windows.hpp | 2 b/src/share/vm/classfile/classLoader.hpp | 7 +- b/src/share/vm/jfr/jni/jfrJniMethod.cpp | 4 + b/src/share/vm/jfr/jni/jfrJniMethod.hpp | 2 b/src/share/vm/jfr/jni/jfrJniMethodRegistration.cpp | 1 b/src/share/vm/jfr/utilities/jfrJavaLog.cpp | 4 + b/src/share/vm/jfr/utilities/jfrJavaLog.hpp | 1 b/src/share/vm/memory/binaryTreeDictionary.cpp | 5 - b/src/share/vm/memory/binaryTreeDictionary.hpp | 10 +-- b/src/share/vm/prims/jvmtiImpl.hpp | 1 b/src/share/vm/runtime/perfData.hpp | 2 b/src/share/vm/utilities/decoder.hpp | 3 b/src/share/vm/utilities/decoder_elf.hpp | 2 b/src/share/vm/utilities/hashtable.cpp | 6 - b/src/share/vm/utilities/hashtable.hpp | 7 +- b/test/runtime/Unsafe/RangeCheck.java | 58 ++++++++++++++++++ 24 files changed, 149 insertions(+), 137 deletions(-) Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero), ppc64, ppc64le, aarch32 (Zero) & aarch64. Ok to push? Thanks, -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From shade at redhat.com Mon Jul 5 07:08:23 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 5 Jul 2021 09:08:23 +0200 Subject: [RFR] [8u] 8u302-b05 Upstream Sync In-Reply-To: <20210705042728.GA2523975@rincewind> References: <20210705042728.GA2523975@rincewind> Message-ID: On 7/5/21 6:27 AM, Andrew Hughes wrote: > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/ > > Merge changesets: > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/corba/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jaxp/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jaxws/merge.changeset Look trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jdk/merge.changeset Looks okay. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/hotspot/merge.changeset Looks okay. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/langtools/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/nashorn/merge.changeset Look trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/root/merge.changeset Looks okay. > Ok to push? Yes. -- Thanks, -Aleksey From shade at openjdk.java.net Mon Jul 5 16:13:16 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 5 Jul 2021 16:13:16 GMT Subject: RFR: 8261492: Shenandoah: reconsider forwardee accesses memory ordering [v7] In-Reply-To: References: Message-ID: > Shenandoah carries forwardee information in object's mark word. Installing the new mark word is effectively "releasing" the object copy, and reading from the new mark word is "acquiring" that object copy. > > For the forwardee update side, Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. This seems to be excessive for Shenandoah forwardee updates, and "release" is enough. > > For the forwardee load side, we need to guarantee "acquire". We do not do it now, reading the markword without memory semantics. It does not seem to pose a practical problem today, because GC does not access the object contents in the new copy, and mutators get this from the JRT-called stub that separates the fwdptr access and object contents access by a lot. It still should be cleaner to "acquire" the mark on load to avoid surprises. > > Additional testing: > - [x] Linux x86_64 `hotspot_gc_shenandoah` > - [x] Linux AArch64 `hotspot_gc_shenandoah` > - [x] Linux AArch64 `tier1` with Shenandoah Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: "acquire" is too slow on aarch64, and does not seem neccessary anyway ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2496/files - new: https://git.openjdk.java.net/jdk/pull/2496/files/4953a6dd..0e78dc98 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2496&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2496&range=05-06 Stats: 32 lines in 3 files changed: 13 ins; 7 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/2496.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2496/head:pull/2496 PR: https://git.openjdk.java.net/jdk/pull/2496 From shade at openjdk.java.net Mon Jul 5 16:23:18 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 5 Jul 2021 16:23:18 GMT Subject: RFR: 8261492: Shenandoah: reconsider forwardee accesses memory ordering [v7] In-Reply-To: References: Message-ID: On Mon, 5 Jul 2021 16:13:16 GMT, Aleksey Shipilev wrote: >> Shenandoah carries forwardee information in object's mark word. Installing the new mark word is effectively "releasing" the object copy, and reading from the new mark word is "acquiring" that object copy. >> >> For the forwardee update side, Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. This seems to be excessive for Shenandoah forwardee updates, and "release" is enough. The reader side is much more interesting, because we generally want "consume", but it is not available. We can do "acquire", but it regresses performance all too much. >> >> The close inspection of the code reveals we don't even need "consume", since we don't read the evacuated object contents in GC code, and mutator code goes through runtime interface which provides enough cushion for relaxed reads to work. This must explain why current weaker reader side was never seen to fail. >> >> The relaxation in `try_update_forwardee` improves concurrent evacuation quite visibly. See for example GC cycle times with SPECjvm2008, Compiler.sunflow on AArch64: >> >> Before: >> >> >> [info][gc,stats] Concurrent Evacuation = 3.421 s (a = 21247 us) (n = 161) >> [info][gc,stats] Concurrent Evacuation = 3.584 s (a = 21080 us) (n = 170) >> [info][gc,stats] Concurrent Evacuation = 3.226 s (a = 21088 us) (n = 153) >> [info][gc,stats] Concurrent Evacuation = 3.270 s (a = 20827 us) (n = 157) >> [info][gc,stats] Concurrent Evacuation = 3.339 s (a = 20742 us) (n = 161) >> >> >> After: >> >> [info][gc,stats] Concurrent Evacuation = 3.109 s (a = 18617 us) (n = 167) >> [info][gc,stats] Concurrent Evacuation = 3.027 s (a = 18918 us) (n = 160) >> [info][gc,stats] Concurrent Evacuation = 2.862 s (a = 17669 us) (n = 162) >> [info][gc,stats] Concurrent Evacuation = 2.858 s (a = 17425 us) (n = 164) >> [info][gc,stats] Concurrent Evacuation = 2.883 s (a = 17685 us) (n = 163) >> >> >> Additional testing: >> - [x] Linux x86_64 `hotspot_gc_shenandoah` >> - [x] Linux AArch64 `hotspot_gc_shenandoah` >> - [x] Linux x86_64 `tier1` with Shenandoah >> - [x] Linux AArch64 `tier1` with Shenandoah > > Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: > > "acquire" is too slow on aarch64, and does not seem neccessary anyway Performance tests on AArch64 reveal that doing "acquire" on the `get_forwardee` path is penalizing concurrent update references quite significantly. Therefore, I had to inspect the code and convince myself that current "relaxed" reads are actually fine. Therefore, we only need to relax the fwdptr store path with "release". See the PR body for sample performance data, and the code comments for more discussion. ------------- PR: https://git.openjdk.java.net/jdk/pull/2496 From shade at openjdk.java.net Mon Jul 5 16:23:17 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 5 Jul 2021 16:23:17 GMT Subject: RFR: 8261492: Shenandoah: reconsider forwardee accesses memory ordering [v8] In-Reply-To: References: Message-ID: <_IqJ7u4Vk7jF8E--2RzWfdnxYXDQr86TIsxA7sh_3WI=.4d2c4cd9-63c8-4921-b5a1-e77d66c10325@github.com> > Shenandoah carries forwardee information in object's mark word. Installing the new mark word is effectively "releasing" the object copy, and reading from the new mark word is "acquiring" that object copy. > > For the forwardee update side, Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. This seems to be excessive for Shenandoah forwardee updates, and "release" is enough. The reader side is much more interesting, because we generally want "consume", but it is not available. We can do "acquire", but it regresses performance all too much. > > The close inspection of the code reveals we don't even need "consume", since we don't read the evacuated object contents in GC code, and mutator code goes through runtime interface which provides enough cushion for relaxed reads to work. This must explain why current weaker reader side was never seen to fail. > > The relaxation in `try_update_forwardee` improves concurrent evacuation quite visibly. See for example GC cycle times with SPECjvm2008, Compiler.sunflow on AArch64: > > Before: > > > [info][gc,stats] Concurrent Evacuation = 3.421 s (a = 21247 us) (n = 161) > [info][gc,stats] Concurrent Evacuation = 3.584 s (a = 21080 us) (n = 170) > [info][gc,stats] Concurrent Evacuation = 3.226 s (a = 21088 us) (n = 153) > [info][gc,stats] Concurrent Evacuation = 3.270 s (a = 20827 us) (n = 157) > [info][gc,stats] Concurrent Evacuation = 3.339 s (a = 20742 us) (n = 161) > > > After: > > [info][gc,stats] Concurrent Evacuation = 3.109 s (a = 18617 us) (n = 167) > [info][gc,stats] Concurrent Evacuation = 3.027 s (a = 18918 us) (n = 160) > [info][gc,stats] Concurrent Evacuation = 2.862 s (a = 17669 us) (n = 162) > [info][gc,stats] Concurrent Evacuation = 2.858 s (a = 17425 us) (n = 164) > [info][gc,stats] Concurrent Evacuation = 2.883 s (a = 17685 us) (n = 163) > > > Additional testing: > - [x] Linux x86_64 `hotspot_gc_shenandoah` > - [x] Linux AArch64 `hotspot_gc_shenandoah` > - [x] Linux x86_64 `tier1` with Shenandoah > - [x] Linux AArch64 `tier1` with Shenandoah Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision: Add TODO ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2496/files - new: https://git.openjdk.java.net/jdk/pull/2496/files/0e78dc98..954dfc19 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2496&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2496&range=06-07 Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/2496.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2496/head:pull/2496 PR: https://git.openjdk.java.net/jdk/pull/2496 From gnu.andrew at redhat.com Mon Jul 5 16:29:51 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Mon, 5 Jul 2021 17:29:51 +0100 Subject: [RFR] [8u] 8u302-b05 Upstream Sync In-Reply-To: References: <20210705042728.GA2523975@rincewind> Message-ID: <20210705162951.GA2639780@rincewind> On 09:08 Mon 05 Jul , Aleksey Shipilev wrote: > On 7/5/21 6:27 AM, Andrew Hughes wrote: > > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/ > > > > Merge changesets: > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/corba/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jaxp/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jaxws/merge.changeset > > Look trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/jdk/merge.changeset > > Looks okay. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/hotspot/merge.changeset > > Looks okay. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/langtools/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/nashorn/merge.changeset > > Look trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b05/root/merge.changeset > > Looks okay. > > > Ok to push? > > Yes. > > -- > Thanks, > -Aleksey > Thanks, pushed. Just two minor rampdown promotions to go (b06 & b07). -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From kbarrett at openjdk.java.net Tue Jul 6 07:19:51 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 6 Jul 2021 07:19:51 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Mon, 5 Jul 2021 01:49:41 GMT, David Holmes wrote: >> Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. >> >> Thanks, Harold > > src/hotspot/share/classfile/stackMapTableFormat.hpp line 47: > >> 45: >> 46: protected: >> 47: // No constructors - should be 'private', but GCC issues a warning if it is > > Have we checked this is still the case? Might be a very old gcc issue. I did a little bit of experimenting and couldn't get gcc to warn, though other compilers might do so. I think such a warning is not unreasonable. ------------- PR: https://git.openjdk.java.net/jdk/pull/4652 From kbarrett at openjdk.java.net Tue Jul 6 07:19:51 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 6 Jul 2021 07:19:51 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Thu, 1 Jul 2021 13:06:55 GMT, Harold Seigel wrote: > Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Changes requested by kbarrett (Reviewer). src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.hpp line 85: > 83: Counters _discovered_count; > 84: Counters _enqueued_count; > 85: NONCOPYABLE(ShenandoahRefProcThreadLocal); I think this change should not be made. We Oracle folks don't make casual changes to Shenandoah, since we don't support or test it. The Shenandoah team is free to write it out rather than using the macro if they prefer. ------------- PR: https://git.openjdk.java.net/jdk/pull/4652 From shade at openjdk.java.net Tue Jul 6 07:36:49 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 6 Jul 2021 07:36:49 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Thu, 1 Jul 2021 13:06:55 GMT, Harold Seigel wrote: > Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Shenandoah bit looks good. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4652 From shade at openjdk.java.net Tue Jul 6 07:36:50 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 6 Jul 2021 07:36:50 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: <5Fe2aqTCHFpotxcrCvIfH81EleWfUMY4_EYq50Lba7w=.5c5753e0-4359-4b72-a19b-af3fe6827050@github.com> On Tue, 6 Jul 2021 07:03:33 GMT, Kim Barrett wrote: >> Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. >> >> Thanks, Harold > > src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.hpp line 85: > >> 83: Counters _discovered_count; >> 84: Counters _enqueued_count; >> 85: NONCOPYABLE(ShenandoahRefProcThreadLocal); > > I think this change should not be made. We Oracle folks don't make casual changes to Shenandoah, since we don't support or test it. The Shenandoah team is free to write it out rather than using the macro if they prefer. OTOH, it is fine to suggest minor/cosmetic changes to the entirety of the OpenJDK code, and ask others to review/test the bits that are not built/tested by the submitter. This change is fine (AFAICS, it is semantically the same as the old code), and it passes `hotspot_gc_shenandoah`. ------------- PR: https://git.openjdk.java.net/jdk/pull/4652 From kbarrett at openjdk.java.net Tue Jul 6 07:44:51 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 6 Jul 2021 07:44:51 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Thu, 1 Jul 2021 13:06:55 GMT, Harold Seigel wrote: > Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Looks good. ------------- Marked as reviewed by kbarrett (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4652 From kbarrett at openjdk.java.net Tue Jul 6 07:44:52 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 6 Jul 2021 07:44:52 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: <5Fe2aqTCHFpotxcrCvIfH81EleWfUMY4_EYq50Lba7w=.5c5753e0-4359-4b72-a19b-af3fe6827050@github.com> References: <5Fe2aqTCHFpotxcrCvIfH81EleWfUMY4_EYq50Lba7w=.5c5753e0-4359-4b72-a19b-af3fe6827050@github.com> Message-ID: <3dqApek2b5eDgZqfVuWa26VMxAChgUMtVU4WmF33py8=.b6a00624-08eb-4b26-969f-9743ce893a1e@github.com> On Tue, 6 Jul 2021 07:32:07 GMT, Aleksey Shipilev wrote: >> src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.hpp line 85: >> >>> 83: Counters _discovered_count; >>> 84: Counters _enqueued_count; >>> 85: NONCOPYABLE(ShenandoahRefProcThreadLocal); >> >> I think this change should not be made. We Oracle folks don't make casual changes to Shenandoah, since we don't support or test it. The Shenandoah team is free to write it out rather than using the macro if they prefer. > > OTOH, it is fine to suggest minor/cosmetic changes to the entirety of the OpenJDK code, and ask others to review/test the bits that are not built/tested by the submitter. > > This change is fine (AFAICS, it is semantically the same as the old code), and it passes `hotspot_gc_shenandoah`. Then I'll withdraw my objection. ------------- PR: https://git.openjdk.java.net/jdk/pull/4652 From shade at openjdk.java.net Tue Jul 6 15:21:10 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 6 Jul 2021 15:21:10 GMT Subject: RFR: 8269924: Shenandoah: Introduce weak/strong marking asserts Message-ID: For debugging, it is sometimes important to have not only `shenandoah_assert_marked`, but also `shenandoah_assert_marked_(weak|strong)` flavors. ------------- Commit messages: - 8269924: Shenandoah: Introduce weak/strong marking asserts Changes: https://git.openjdk.java.net/jdk/pull/4691/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4691&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8269924 Stats: 46 lines in 2 files changed: 46 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/4691.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4691/head:pull/4691 PR: https://git.openjdk.java.net/jdk/pull/4691 From rkennke at openjdk.java.net Tue Jul 6 19:17:58 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Tue, 6 Jul 2021 19:17:58 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong Message-ID: We've observed test failures in jcstress, see: https://bugs.openjdk.java.net/browse/JDK-8269897 We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. Testing: - [x] the provided jcstress test - [x] hotspot_gc_shenandoah - [x] tier1 ------------- Commit messages: - 8269897: Shenandoah: Treat UNKNOWN refs access as strong Changes: https://git.openjdk.java.net/jdk17/pull/219/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk17&pr=219&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8269897 Stats: 3 lines in 2 files changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk17/pull/219.diff Fetch: git fetch https://git.openjdk.java.net/jdk17 pull/219/head:pull/219 PR: https://git.openjdk.java.net/jdk17/pull/219 From shade at openjdk.java.net Wed Jul 7 10:44:51 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 7 Jul 2021 10:44:51 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong In-Reply-To: References: Message-ID: On Tue, 6 Jul 2021 19:11:58 GMT, Roman Kennke wrote: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 It looks okay to me, and it fixes the jcstress failures. But I see one of the `tier1` tests failing: $ make run-test TEST=gc/TestReferenceClearDuringReferenceProcessing.java TEST_VM_OPTS="-XX:+UseShenandoahGC" ... TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: ref not enqueued I wonder if that failure indicates that fix is still incomplete or too conservative. ------------- PR: https://git.openjdk.java.net/jdk17/pull/219 From rkennke at openjdk.java.net Wed Jul 7 12:37:16 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Wed, 7 Jul 2021 12:37:16 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v2] In-Reply-To: References: Message-ID: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Treat UNKNOWN as STRONG more selectively only on CAE ------------- Changes: - all: https://git.openjdk.java.net/jdk17/pull/219/files - new: https://git.openjdk.java.net/jdk17/pull/219/files/07d7a9f1..31805dee Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk17&pr=219&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk17&pr=219&range=00-01 Stats: 5 lines in 3 files changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk17/pull/219.diff Fetch: git fetch https://git.openjdk.java.net/jdk17 pull/219/head:pull/219 PR: https://git.openjdk.java.net/jdk17/pull/219 From rkennke at openjdk.java.net Wed Jul 7 12:37:16 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Wed, 7 Jul 2021 12:37:16 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v2] In-Reply-To: References: Message-ID: On Wed, 7 Jul 2021 10:41:25 GMT, Aleksey Shipilev wrote: > It looks okay to me, and it fixes the jcstress failures. But I see one of the `tier1` tests failing: > > ``` > $ make run-test TEST=gc/TestReferenceClearDuringReferenceProcessing.java TEST_VM_OPTS="-XX:+UseShenandoahGC" > ... > TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: ref not enqueued > ``` > > I wonder if that failure indicates that fix is still incomplete or too conservative. Changing behaviour for oop_load_in_heap() seems to break it (although it is not quite obvious to me how - the test does not seem to access Reference.referent itself, especially not via reflection/unsafe/etc. Maybe something internal in Reference queue does, and expects it to return NULL on unreachable.) Let's limit the change to treat UNKNOWN as STRONG only for CAE. ------------- PR: https://git.openjdk.java.net/jdk17/pull/219 From kdnilsen at openjdk.java.net Wed Jul 7 16:55:36 2021 From: kdnilsen at openjdk.java.net (Kelvin Nilsen) Date: Wed, 7 Jul 2021 16:55:36 GMT Subject: RFR: Do old satb during interrupting young Message-ID: This patch includes bug fixes discovered during integration and testing of concurrent remembered set scanning. The bug is that processing of SATB barrier by young-gen collections that preempt a concurrent old-gen concurrent marking effort was ignoring pointers to old-gen. It needs to pass these pointers along to the old-gen concurrent mark effort. Since the bug manifests even outside concurrent remembered set scanning, we propose to upstream it now. The patch has been tested with jtreg:tier1_gc_shenandoah with no regressions. ------------- Commit messages: - Stronger encapsulation of bootstrap gc concept - Remove duplicate asserts, init global generation for bootstrap (rather than old + young, which visits free regions twice). - Remove completed TODO comment - Make old gc bootstrap flag immutable - Preserve access to old gen mark queues for young collects Changes: https://git.openjdk.java.net/shenandoah/pull/51/files Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=51&range=00 Stats: 47 lines in 7 files changed: 20 ins; 12 del; 15 mod Patch: https://git.openjdk.java.net/shenandoah/pull/51.diff Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/51/head:pull/51 PR: https://git.openjdk.java.net/shenandoah/pull/51 From shade at openjdk.java.net Wed Jul 7 18:32:54 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 7 Jul 2021 18:32:54 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v2] In-Reply-To: <22QG4_SdgK-z2XrxRCJ17H_G9Rk8paxapKURHrDD3A4=.1d67309e-3149-4722-92cf-a2b394eeaa54@github.com> References: <22QG4_SdgK-z2XrxRCJ17H_G9Rk8paxapKURHrDD3A4=.1d67309e-3149-4722-92cf-a2b394eeaa54@github.com> Message-ID: On Wed, 7 Jul 2021 17:48:50 GMT, Aleksey Shipilev wrote: >> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: >> >> Treat UNKNOWN as STRONG more selectively only on CAE > > src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 270: > >> 268: // Note: We don't need a keep-alive-barrier here. We already enqueue any loaded reference for SATB anyway, >> 269: // because it must be the previous value. >> 270: res = ShenandoahBarrierSet::barrier_set()->load_reference_barrier(res, NULL); > > Add a comment: > > > // Note: We don't need a keep-alive-barrier here. We already enqueue any loaded reference for SATB anyway, > // because it must be the previous value. Convert ON_UNKNOWN_OOP_REF to strong, so that we don't return > // NULL for weak values, breaking CAS contract. The downside is this method effectively resurrecting > // weak values without GC knowing about it. If Unsafe/JNI users are using this method on weak values > // (i.e. doing unsafe accesses to Reference.referent), they are on their own. I also believe "xchg" needs the same fix and a comment: // Note: We don't need a keep-alive-barrier here. We already enqueue any loaded reference for SATB anyway, // because it must be the previous value. Convert ON_UNKNOWN_OOP_REF to strong, so that we don't return // NULL for weak values. ShenandoahBarrierSet::See oop_atomic_cmpxchg_not_in_heap for explanation. ------------- PR: https://git.openjdk.java.net/jdk17/pull/219 From shade at openjdk.java.net Wed Jul 7 18:32:53 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 7 Jul 2021 18:32:53 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v2] In-Reply-To: References: Message-ID: <22QG4_SdgK-z2XrxRCJ17H_G9Rk8paxapKURHrDD3A4=.1d67309e-3149-4722-92cf-a2b394eeaa54@github.com> On Wed, 7 Jul 2021 12:37:16 GMT, Roman Kennke wrote: >> We've observed test failures in jcstress, see: >> https://bugs.openjdk.java.net/browse/JDK-8269897 >> >> We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. >> >> A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. >> This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. >> >> Testing: >> - [x] the provided jcstress test >> - [x] hotspot_gc_shenandoah >> - [x] tier1 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Treat UNKNOWN as STRONG more selectively only on CAE Need synopsis change after limiting a fix? Need to handle `xchg` case too. This patch passes `tier1` with Shenandoah for me, but fails jcstress bundle when run with C1. A relevant `ShenandoahBarrierSetC1` change is quite probably missing. Also, probably `Unsafe` `get` and `set` require the same handling: under concurrent weak roots, it is safer to return overly strong value, not NULL for strong refs. Let it be users' problem accessing weak values with Unsafe? src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 643: > 641: } > 642: #endif > 643: load_store = kit->gvn().transform(new ShenandoahLoadReferenceBarrierNode(NULL, load_store, access.decorators() & ~ON_UNKNOWN_OOP_REF)); Add a comment: // Convert ON_UNKNOWN_OOP_REF to strong, so that we don't return NULL for weak values. // See ShenandoahBarrierSet::See oop_atomic_cmpxchg_not_in_heap for explanation. src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 270: > 268: // Note: We don't need a keep-alive-barrier here. We already enqueue any loaded reference for SATB anyway, > 269: // because it must be the previous value. > 270: res = ShenandoahBarrierSet::barrier_set()->load_reference_barrier(res, NULL); Add a comment: // Note: We don't need a keep-alive-barrier here. We already enqueue any loaded reference for SATB anyway, // because it must be the previous value. Convert ON_UNKNOWN_OOP_REF to strong, so that we don't return // NULL for weak values, breaking CAS contract. The downside is this method effectively resurrecting // weak values without GC knowing about it. If Unsafe/JNI users are using this method on weak values // (i.e. doing unsafe accesses to Reference.referent), they are on their own. ------------- Changes requested by shade (Reviewer). PR: https://git.openjdk.java.net/jdk17/pull/219 From rkennke at openjdk.java.net Thu Jul 8 12:41:26 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 8 Jul 2021 12:41:26 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v3] In-Reply-To: References: Message-ID: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Properly resolve access strength in load, cmpxchg and xchg ------------- Changes: - all: https://git.openjdk.java.net/jdk17/pull/219/files - new: https://git.openjdk.java.net/jdk17/pull/219/files/31805dee..7647b234 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk17&pr=219&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk17&pr=219&range=01-02 Stats: 78 lines in 2 files changed: 68 ins; 4 del; 6 mod Patch: https://git.openjdk.java.net/jdk17/pull/219.diff Fetch: git fetch https://git.openjdk.java.net/jdk17 pull/219/head:pull/219 PR: https://git.openjdk.java.net/jdk17/pull/219 From rkennke at openjdk.java.net Thu Jul 8 12:47:15 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 8 Jul 2021 12:47:15 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v4] In-Reply-To: References: Message-ID: <5HFkesTKX9967Ne_KryaXvJ8uRO8VO_bIUzx4ektRP4=.67c13b70-706f-449a-aa81-930aaa5cb48a@github.com> > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Fix assert ------------- Changes: - all: https://git.openjdk.java.net/jdk17/pull/219/files - new: https://git.openjdk.java.net/jdk17/pull/219/files/7647b234..8c48c9aa Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk17&pr=219&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk17&pr=219&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk17/pull/219.diff Fetch: git fetch https://git.openjdk.java.net/jdk17 pull/219/head:pull/219 PR: https://git.openjdk.java.net/jdk17/pull/219 From shade at openjdk.java.net Thu Jul 8 13:07:03 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 8 Jul 2021 13:07:03 GMT Subject: RFR: 8270096: Shenandoah: Optimize gc/shenandoah/TestRefprocSanity.java for interpreter mode Message-ID: <4PhHOGTQrjGVKUFDmomEdyifdQpu0WI4r-nw0fvEdJY=.647f9dbc-5f70-4dd2-9203-8b0e6eb41914@github.com> Found this while testing Shenandoah with Zero. This configuration takes a lot of time, and eventually times out: $ time CONF=linux-x86_64-server-fastdebug make run-test TEST=gc/shenandoah/TestRefprocSanity.java TEST_OPTS="JAVA_OPTIONS=-Xint" With these fixes, the run time with `-Xint` is under 1 minute: real 0m51.472s user 3m0.366s sys 0m5.473s In default mode, the run time is now under 20 seconds: real 0m20.584s user 1m8.693s sys 0m5.030s This also passes with Zero (plus some Zero fixes) now. ------------- Commit messages: - Drop the window to 1K too - 8270096: Shenandoah: Optimize gc/shenandoah/TestRefprocSanity.java for interpreter mode Changes: https://git.openjdk.java.net/jdk/pull/4725/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4725&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8270096 Stats: 17 lines in 1 file changed: 7 ins; 0 del; 10 mod Patch: https://git.openjdk.java.net/jdk/pull/4725.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4725/head:pull/4725 PR: https://git.openjdk.java.net/jdk/pull/4725 From shade at openjdk.java.net Thu Jul 8 14:00:14 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 8 Jul 2021 14:00:14 GMT Subject: RFR: 8270094: Shenandoah: Provide human-readable labels for test configurations Message-ID: `hotspot_gc_shenandoah` contains a lot of parallel test configurations, and it would be much nicer to have them named appropriately, rather than just #id1...#id2. Additional testing: - [x] `hotspot_gc_shenandoah`, eyeballing test names ------------- Commit messages: - 8270094: Shenandoah: Provide human-readable labels for test configurations Changes: https://git.openjdk.java.net/jdk/pull/4726/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4726&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8270094 Stats: 227 lines in 81 files changed: 19 ins; 0 del; 208 mod Patch: https://git.openjdk.java.net/jdk/pull/4726.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4726/head:pull/4726 PR: https://git.openjdk.java.net/jdk/pull/4726 From zgu at openjdk.java.net Thu Jul 8 14:01:54 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Thu, 8 Jul 2021 14:01:54 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v4] In-Reply-To: <5HFkesTKX9967Ne_KryaXvJ8uRO8VO_bIUzx4ektRP4=.67c13b70-706f-449a-aa81-930aaa5cb48a@github.com> References: <5HFkesTKX9967Ne_KryaXvJ8uRO8VO_bIUzx4ektRP4=.67c13b70-706f-449a-aa81-930aaa5cb48a@github.com> Message-ID: On Thu, 8 Jul 2021 12:47:15 GMT, Roman Kennke wrote: >> We've observed test failures in jcstress, see: >> https://bugs.openjdk.java.net/browse/JDK-8269897 >> >> We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. >> >> A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. >> This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. >> >> Testing: >> - [x] the provided jcstress test >> - [x] hotspot_gc_shenandoah >> - [x] tier1 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Fix assert src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 643: > 641: } > 642: #endif > 643: load_store = kit->gvn().transform(new ShenandoahLoadReferenceBarrierNode(NULL, load_store, access.decorators() & ~ON_UNKNOWN_OOP_REF)); This load should go through slow path when GC is in progress, right? Should let slow path to resolve the strength? sounds there is possibility to resurrect dead oop. ------------- PR: https://git.openjdk.java.net/jdk17/pull/219 From zgu at openjdk.java.net Thu Jul 8 14:08:51 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Thu, 8 Jul 2021 14:08:51 GMT Subject: RFR: 8270094: Shenandoah: Provide human-readable labels for test configurations In-Reply-To: References: Message-ID: On Thu, 8 Jul 2021 13:54:20 GMT, Aleksey Shipilev wrote: > `hotspot_gc_shenandoah` contains a lot of parallel test configurations, and it would be much nicer to have them named appropriately, rather than just #id1...#id2. > > Additional testing: > - [x] `hotspot_gc_shenandoah`, eyeballing test names Good to me ------------- Marked as reviewed by zgu (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4726 From zgu at openjdk.java.net Thu Jul 8 16:01:53 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Thu, 8 Jul 2021 16:01:53 GMT Subject: RFR: 8270096: Shenandoah: Optimize gc/shenandoah/TestRefprocSanity.java for interpreter mode In-Reply-To: <4PhHOGTQrjGVKUFDmomEdyifdQpu0WI4r-nw0fvEdJY=.647f9dbc-5f70-4dd2-9203-8b0e6eb41914@github.com> References: <4PhHOGTQrjGVKUFDmomEdyifdQpu0WI4r-nw0fvEdJY=.647f9dbc-5f70-4dd2-9203-8b0e6eb41914@github.com> Message-ID: On Thu, 8 Jul 2021 12:59:30 GMT, Aleksey Shipilev wrote: > Found this while testing Shenandoah with Zero. This configuration takes a lot of time, and eventually times out: > > > $ time CONF=linux-x86_64-server-fastdebug make run-test TEST=gc/shenandoah/TestRefprocSanity.java TEST_OPTS="JAVA_OPTIONS=-Xint" > > > > With these fixes, the run time with `-Xint` is under 1 minute: > > > real 0m51.472s > user 3m0.366s > sys 0m5.473s > > > In default mode, the run time is now under 20 seconds: > > > real 0m20.584s > user 1m8.693s > sys 0m5.030s > > > This also passes with Zero (plus some Zero fixes) now. Looks good to me. ------------- Marked as reviewed by zgu (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4725 From shade at openjdk.java.net Thu Jul 8 16:46:02 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 8 Jul 2021 16:46:02 GMT Subject: Integrated: 8270096: Shenandoah: Optimize gc/shenandoah/TestRefprocSanity.java for interpreter mode In-Reply-To: <4PhHOGTQrjGVKUFDmomEdyifdQpu0WI4r-nw0fvEdJY=.647f9dbc-5f70-4dd2-9203-8b0e6eb41914@github.com> References: <4PhHOGTQrjGVKUFDmomEdyifdQpu0WI4r-nw0fvEdJY=.647f9dbc-5f70-4dd2-9203-8b0e6eb41914@github.com> Message-ID: <5-XsEjUXMByJHbgIOywqfPki1-B8TuPD5ootQeqsEgY=.dcc0c0e4-533a-47a3-8333-b20798aa9264@github.com> On Thu, 8 Jul 2021 12:59:30 GMT, Aleksey Shipilev wrote: > Found this while testing Shenandoah with Zero. This configuration takes a lot of time, and eventually times out: > > > $ time CONF=linux-x86_64-server-fastdebug make run-test TEST=gc/shenandoah/TestRefprocSanity.java TEST_OPTS="JAVA_OPTIONS=-Xint" > > > > With these fixes, the run time with `-Xint` is under 1 minute: > > > real 0m51.472s > user 3m0.366s > sys 0m5.473s > > > In default mode, the run time is now under 20 seconds: > > > real 0m20.584s > user 1m8.693s > sys 0m5.030s > > > This also passes with Zero (plus some Zero fixes) now. This pull request has now been integrated. Changeset: 4f322a9b Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/4f322a9b6c9cfdf080df23326daa45733f1b9aad Stats: 17 lines in 1 file changed: 7 ins; 0 del; 10 mod 8270096: Shenandoah: Optimize gc/shenandoah/TestRefprocSanity.java for interpreter mode Reviewed-by: zgu ------------- PR: https://git.openjdk.java.net/jdk/pull/4725 From shade at openjdk.java.net Thu Jul 8 16:46:01 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 8 Jul 2021 16:46:01 GMT Subject: RFR: 8270096: Shenandoah: Optimize gc/shenandoah/TestRefprocSanity.java for interpreter mode In-Reply-To: <4PhHOGTQrjGVKUFDmomEdyifdQpu0WI4r-nw0fvEdJY=.647f9dbc-5f70-4dd2-9203-8b0e6eb41914@github.com> References: <4PhHOGTQrjGVKUFDmomEdyifdQpu0WI4r-nw0fvEdJY=.647f9dbc-5f70-4dd2-9203-8b0e6eb41914@github.com> Message-ID: On Thu, 8 Jul 2021 12:59:30 GMT, Aleksey Shipilev wrote: > Found this while testing Shenandoah with Zero. This configuration takes a lot of time, and eventually times out: > > > $ time CONF=linux-x86_64-server-fastdebug make run-test TEST=gc/shenandoah/TestRefprocSanity.java TEST_OPTS="JAVA_OPTIONS=-Xint" > > > > With these fixes, the run time with `-Xint` is under 1 minute: > > > real 0m51.472s > user 3m0.366s > sys 0m5.473s > > > In default mode, the run time is now under 20 seconds: > > > real 0m20.584s > user 1m8.693s > sys 0m5.030s > > > This also passes with Zero (plus some Zero fixes) now. Thanks! There is only one unrelated failure in GHA testing. ------------- PR: https://git.openjdk.java.net/jdk/pull/4725 From rkennke at openjdk.java.net Thu Jul 8 18:36:10 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 8 Jul 2021 18:36:10 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible Message-ID: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> We've observed test failures in jcstress, see: https://bugs.openjdk.java.net/browse/JDK-8269897 We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. Testing: - [x] the provided jcstress test - [x] hotspot_gc_shenandoah - [x] tier1 ------------- Commit messages: - Determine access strength in all runtime barriers, where possible - Treat UNKNOWN as STRONG more selectively only on CAE - 8269897: Shenandoah: Treat UNKNOWN refs access as strong Changes: https://git.openjdk.java.net/jdk/pull/4697/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4697&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8269897 Stats: 111 lines in 4 files changed: 43 ins; 35 del; 33 mod Patch: https://git.openjdk.java.net/jdk/pull/4697.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4697/head:pull/4697 PR: https://git.openjdk.java.net/jdk/pull/4697 From rkennke at openjdk.java.net Thu Jul 8 18:36:10 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 8 Jul 2021 18:36:10 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible In-Reply-To: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: On Tue, 6 Jul 2021 19:00:59 GMT, Roman Kennke wrote: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 Closing and retargeting to jdk17. ------------- PR: https://git.openjdk.java.net/jdk/pull/4697 From rkennke at openjdk.java.net Thu Jul 8 18:40:33 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 8 Jul 2021 18:40:33 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible [v2] In-Reply-To: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Revert C2 changes ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/4697/files - new: https://git.openjdk.java.net/jdk/pull/4697/files/99e11285..c2e666b9 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4697&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4697&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/4697.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4697/head:pull/4697 PR: https://git.openjdk.java.net/jdk/pull/4697 From rkennke at openjdk.java.net Thu Jul 8 20:02:56 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 8 Jul 2021 20:02:56 GMT Subject: [jdk17] RFR: 8269897: Shenandoah: Treat UNKNOWN refs access as strong [v4] In-Reply-To: <5HFkesTKX9967Ne_KryaXvJ8uRO8VO_bIUzx4ektRP4=.67c13b70-706f-449a-aa81-930aaa5cb48a@github.com> References: <5HFkesTKX9967Ne_KryaXvJ8uRO8VO_bIUzx4ektRP4=.67c13b70-706f-449a-aa81-930aaa5cb48a@github.com> Message-ID: On Thu, 8 Jul 2021 12:47:15 GMT, Roman Kennke wrote: >> We've observed test failures in jcstress, see: >> https://bugs.openjdk.java.net/browse/JDK-8269897 >> >> We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. >> >> A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. >> This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. >> >> Testing: >> - [x] the provided jcstress test >> - [x] hotspot_gc_shenandoah >> - [x] tier1 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Fix assert Fixing this in jdk18 first, should backport to jdk17 later, after some testing in 18. See: https://github.com/openjdk/jdk/pull/4697 ------------- PR: https://git.openjdk.java.net/jdk17/pull/219 From rkennke at openjdk.java.net Thu Jul 8 20:02:57 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 8 Jul 2021 20:02:57 GMT Subject: [jdk17] Withdrawn: 8269897: Shenandoah: Treat UNKNOWN refs access as strong In-Reply-To: References: Message-ID: On Tue, 6 Jul 2021 19:11:58 GMT, Roman Kennke wrote: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk17/pull/219 From gnu.andrew at redhat.com Fri Jul 9 02:52:39 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Fri, 9 Jul 2021 03:52:39 +0100 Subject: [RFR] [8u] 8u302-b06 Upstream Sync Message-ID: <20210709025239.GA3047528@rincewind> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/ Merge changesets: http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/corba/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jaxp/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jaxws/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jdk/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/hotspot/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/langtools/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/nashorn/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/root/merge.changeset Changes in aarch64-shenandoah-jdk8u302-b06: - JDK-8267545: [8u] Enable Xcode 12 builds on macOS - JDK-8268444: keytool -v -list print is incorrect after backport JDK-8141457 Main issues of note: None, clean merge (no HotSpot changes). diffstat for root b/.hgtags | 1 b/common/autoconf/basics.m4 | 2 - b/common/autoconf/flags.m4 | 18 ++++++++++ b/common/autoconf/generated-configure.sh | 52 ++++++++++++++++++++++--------- b/common/autoconf/libraries.m4 | 8 +++- b/common/autoconf/platform.m4 | 2 + b/common/autoconf/toolchain.m4 | 14 ++++---- 7 files changed, 71 insertions(+), 26 deletions(-) diffstat for corba b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxp b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxws b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for langtools b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for nashorn b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jdk b/.hgtags | 1 + b/make/lib/CoreLibraries.gmk | 5 +++-- b/src/macosx/native/apple/applescript/AppleScriptExecutionContext.m | 1 + b/src/macosx/native/sun/osxapp/ThreadUtilities.h | 2 +- b/src/share/classes/sun/security/tools/keytool/Resources.java | 2 +- b/src/share/native/sun/misc/URLClassPath.c | 9 +++++++-- 6 files changed, 14 insertions(+), 6 deletions(-) diffstat for hotspot b/.hgtags | 1 + b/make/bsd/makefiles/gcc.make | 10 +++++++++- b/make/bsd/makefiles/jsig.make | 5 +++++ b/src/os_cpu/bsd_x86/vm/os_bsd_x86.cpp | 2 +- b/src/share/vm/utilities/ostream.cpp | 3 +++ 5 files changed, 19 insertions(+), 2 deletions(-) Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero), ppc64, ppc64le, aarch32 (Zero) & aarch64. Ok to push? Thanks, -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From shade at openjdk.java.net Fri Jul 9 07:57:54 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 9 Jul 2021 07:57:54 GMT Subject: Integrated: 8270094: Shenandoah: Provide human-readable labels for test configurations In-Reply-To: References: Message-ID: <9UMCrzvn0D3vHMMhFULWzN-UkG4oWfG9SpRiZH7WYE0=.87c840d1-4c84-44ca-ad72-7ad8f6c09b3d@github.com> On Thu, 8 Jul 2021 13:54:20 GMT, Aleksey Shipilev wrote: > `hotspot_gc_shenandoah` contains a lot of parallel test configurations, and it would be much nicer to have them named appropriately, rather than just #id1...#id2. > > Additional testing: > - [x] `hotspot_gc_shenandoah`, eyeballing test names This pull request has now been integrated. Changeset: 676f1d76 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/676f1d76cab9108655f2ce462db321b975563eb7 Stats: 227 lines in 81 files changed: 19 ins; 0 del; 208 mod 8270094: Shenandoah: Provide human-readable labels for test configurations Reviewed-by: zgu ------------- PR: https://git.openjdk.java.net/jdk/pull/4726 From shade at redhat.com Fri Jul 9 08:11:34 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 9 Jul 2021 10:11:34 +0200 Subject: [RFR] [8u] 8u302-b06 Upstream Sync In-Reply-To: <20210709025239.GA3047528@rincewind> References: <20210709025239.GA3047528@rincewind> Message-ID: <528074e9-ca65-249f-301a-c370a8f4aaff@redhat.com> On 7/9/21 4:52 AM, Andrew Hughes wrote: > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/ > > Merge changesets: > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/corba/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jaxp/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jaxws/merge.changeset Look trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jdk/merge.changeset Looks good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/hotspot/merge.changeset Looks good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/langtools/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/nashorn/merge.changeset Look trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/root/merge.changeset Looks good. > Ok to push? Yes. -- Thanks, -Aleksey From shade at openjdk.java.net Fri Jul 9 10:48:51 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 9 Jul 2021 10:48:51 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible [v2] In-Reply-To: References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: On Thu, 8 Jul 2021 18:40:33 GMT, Roman Kennke wrote: >> We've observed test failures in jcstress, see: >> https://bugs.openjdk.java.net/browse/JDK-8269897 >> >> We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. >> >> A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. >> This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. >> >> Testing: >> - [x] the provided jcstress test >> - [x] hotspot_gc_shenandoah >> - [x] tier1 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Revert C2 changes Looks okay to me, I am testing. One nit: lets assert no UNKNOWN decorators are passed to AccessBarrier methods that do not do the UNKNOWN resolve? ------------- PR: https://git.openjdk.java.net/jdk/pull/4697 From shade at openjdk.java.net Fri Jul 9 12:20:58 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 9 Jul 2021 12:20:58 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible [v2] In-Reply-To: References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: <_oqNWhYQRm3xeTlTpnARISiGQCz1rfIBT7TyORCsj4Y=.7e621de4-9e81-46a5-b3e0-cd534e1b7926@github.com> On Thu, 8 Jul 2021 18:40:33 GMT, Roman Kennke wrote: >> We've observed test failures in jcstress, see: >> https://bugs.openjdk.java.net/browse/JDK-8269897 >> >> We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. >> >> A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. >> This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. >> >> Testing: >> - [x] the provided jcstress test >> - [x] hotspot_gc_shenandoah >> - [x] tier1 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Revert C2 changes Current patch passes my testing, thanks. Still think we need to do asserts for extra safety. ------------- PR: https://git.openjdk.java.net/jdk/pull/4697 From zgu at openjdk.java.net Fri Jul 9 13:17:08 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 9 Jul 2021 13:17:08 GMT Subject: RFR: 8270110: Shenandoah: Add test for JDK-8269661 Message-ID: I would like to add a test for JDK-8269661. Currently, JDK-8269661 only affects Shenandoah GC, as it is the only GC supports object pinning and string deduplication. Test: - [x] new test passes with JDK-8269661, fails after backout JDK-8269661 fix. ------------- Commit messages: - Disable compact string - Merge branch 'master' into string_critical_dedup - update test - v2 - v0 Changes: https://git.openjdk.java.net/jdk/pull/4740/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4740&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8270110 Stats: 208 lines in 3 files changed: 207 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/4740.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4740/head:pull/4740 PR: https://git.openjdk.java.net/jdk/pull/4740 From zgu at openjdk.java.net Fri Jul 9 13:31:16 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 9 Jul 2021 13:31:16 GMT Subject: RFR: 8270171: Shenandoah: Cleanup TestStringDedup and TestStringDedupStress tests Message-ID: Please review this trivial patch to cleanup TestStringDedup and TestStringDedupStress tests, removes unused Unsafe usages. Test: - [x] Both tests still pass ------------- Commit messages: - v0 Changes: https://git.openjdk.java.net/jdk/pull/4742/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4742&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8270171 Stats: 20 lines in 2 files changed: 0 ins; 19 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/4742.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4742/head:pull/4742 PR: https://git.openjdk.java.net/jdk/pull/4742 From shade at openjdk.java.net Fri Jul 9 13:49:57 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 9 Jul 2021 13:49:57 GMT Subject: RFR: 8270171: Shenandoah: Cleanup TestStringDedup and TestStringDedupStress tests In-Reply-To: References: Message-ID: On Fri, 9 Jul 2021 13:23:56 GMT, Zhengyu Gu wrote: > Please review this trivial patch to cleanup TestStringDedup and TestStringDedupStress tests, removes unused Unsafe usages. > > Test: > - [x] Both tests still pass Ah yes, `java.lang:open` already provides this. Good! ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4742 From rkennke at openjdk.java.net Fri Jul 9 15:31:23 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Fri, 9 Jul 2021 15:31:23 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible [v3] In-Reply-To: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Add asserts for various decorators to runtime barriers ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/4697/files - new: https://git.openjdk.java.net/jdk/pull/4697/files/c2e666b9..307ce30b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4697&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4697&range=01-02 Stats: 15 lines in 2 files changed: 14 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/4697.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4697/head:pull/4697 PR: https://git.openjdk.java.net/jdk/pull/4697 From shade at openjdk.java.net Fri Jul 9 17:23:54 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 9 Jul 2021 17:23:54 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible [v3] In-Reply-To: References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: On Fri, 9 Jul 2021 15:31:23 GMT, Roman Kennke wrote: >> We've observed test failures in jcstress, see: >> https://bugs.openjdk.java.net/browse/JDK-8269897 >> >> We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. >> >> A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. >> This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. >> >> Testing: >> - [x] the provided jcstress test >> - [x] hotspot_gc_shenandoah >> - [x] tier1 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Add asserts for various decorators to runtime barriers src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.hpp line 128: > 126: class AccessBarrier: public BarrierSet::AccessBarrier { > 127: typedef BarrierSet::AccessBarrier Raw; > 128: This change looks unnecessary. src/hotspot/share/gc/shenandoah/shenandoahBarrierSet.inline.hpp line 320: > 318: template > 319: inline oop ShenandoahBarrierSet::AccessBarrier::oop_atomic_xchg_in_heap_at(oop base, ptrdiff_t offset, oop new_value) { > 320: assert((decorators & ON_STRONG_OOP_REF) != 0, "must be present"); Shouldn't this be `| ON_UNKNOWN_OOP_REF`? We are doing `resolve_possibly_unknown_oop_ref_strength` later. In fact, I am not so sure about asserting `ON_STRONG_OOP_REF` here and elsewhere as well. The future `ON_WEAK_OOP_REF` code should work fine here too? I think only asserting that `ON_Uresolve_possibly_unknown_oop_ref_strengthNKNOWN_OOP_REF` is not passed on bad paths is enough? ------------- PR: https://git.openjdk.java.net/jdk/pull/4697 From zgu at openjdk.java.net Fri Jul 9 17:37:57 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 9 Jul 2021 17:37:57 GMT Subject: Integrated: 8270171: Shenandoah: Cleanup TestStringDedup and TestStringDedupStress tests In-Reply-To: References: Message-ID: On Fri, 9 Jul 2021 13:23:56 GMT, Zhengyu Gu wrote: > Please review this trivial patch to cleanup TestStringDedup and TestStringDedupStress tests, removes unused Unsafe usages. > > Test: > - [x] Both tests still pass This pull request has now been integrated. Changeset: e4c24f01 Author: Zhengyu Gu URL: https://git.openjdk.java.net/jdk/commit/e4c24f01ef97de1c9a5a5faca5dd8ea2ad8c62d9 Stats: 20 lines in 2 files changed: 0 ins; 19 del; 1 mod 8270171: Shenandoah: Cleanup TestStringDedup and TestStringDedupStress tests Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/4742 From rkennke at openjdk.java.net Mon Jul 12 10:34:07 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Mon, 12 Jul 2021 10:34:07 GMT Subject: RFR: 8270110: Shenandoah: Add test for JDK-8269661 In-Reply-To: References: Message-ID: On Fri, 9 Jul 2021 13:07:40 GMT, Zhengyu Gu wrote: > I would like to add a test for JDK-8269661. > > Currently, JDK-8269661 only affects Shenandoah GC, as it is the only GC supports object pinning and string deduplication. > > Test: > - [x] new test passes with JDK-8269661, fails after backout JDK-8269661 fix. Marked as reviewed by rkennke (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/4740 From rkennke at openjdk.java.net Mon Jul 12 11:21:27 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Mon, 12 Jul 2021 11:21:27 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible [v4] In-Reply-To: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Aleksey's comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/4697/files - new: https://git.openjdk.java.net/jdk/pull/4697/files/307ce30b..3dcac72f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=4697&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=4697&range=02-03 Stats: 11 lines in 2 files changed: 1 ins; 6 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/4697.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4697/head:pull/4697 PR: https://git.openjdk.java.net/jdk/pull/4697 From zgu at openjdk.java.net Mon Jul 12 12:26:00 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Mon, 12 Jul 2021 12:26:00 GMT Subject: Integrated: 8270110: Shenandoah: Add test for JDK-8269661 In-Reply-To: References: Message-ID: On Fri, 9 Jul 2021 13:07:40 GMT, Zhengyu Gu wrote: > I would like to add a test for JDK-8269661. > > Currently, JDK-8269661 only affects Shenandoah GC, as it is the only GC supports object pinning and string deduplication. > > Test: > - [x] new test passes with JDK-8269661, fails after backout JDK-8269661 fix. This pull request has now been integrated. Changeset: 548bb311 Author: Zhengyu Gu URL: https://git.openjdk.java.net/jdk/commit/548bb3119cf792bb1e65e15e990a4addc9155312 Stats: 208 lines in 3 files changed: 207 ins; 0 del; 1 mod 8270110: Shenandoah: Add test for JDK-8269661 Reviewed-by: rkennke ------------- PR: https://git.openjdk.java.net/jdk/pull/4740 From hseigel at openjdk.java.net Mon Jul 12 14:24:56 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 12 Jul 2021 14:24:56 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Tue, 6 Jul 2021 07:01:22 GMT, Kim Barrett wrote: >> src/hotspot/share/classfile/stackMapTableFormat.hpp line 47: >> >>> 45: >>> 46: protected: >>> 47: // No constructors - should be 'private', but GCC issues a warning if it is >> >> Have we checked this is still the case? Might be a very old gcc issue. > > I did a little bit of experimenting and couldn't get gcc to warn, though other compilers might do so. I think such a warning is not unreasonable. Based on Kim's comment, I decided to leave the change as it currently is. ------------- PR: https://git.openjdk.java.net/jdk/pull/4652 From hseigel at openjdk.java.net Mon Jul 12 14:24:55 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 12 Jul 2021 14:24:55 GMT Subject: RFR: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Thu, 1 Jul 2021 13:06:55 GMT, Harold Seigel wrote: > Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold Thanks David, Kim, and Aleksey for reviewing this change! ------------- PR: https://git.openjdk.java.net/jdk/pull/4652 From hseigel at openjdk.java.net Mon Jul 12 14:24:56 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Mon, 12 Jul 2021 14:24:56 GMT Subject: Integrated: 8244162: Additional opportunities to use NONCOPYABLE In-Reply-To: References: Message-ID: On Thu, 1 Jul 2021 13:06:55 GMT, Harold Seigel wrote: > Please review this small change to use NONCOPYABLE macro where applicable. The change was tested by running Mach5 tiers 1-2 on Linux, Mac OS, and Windows, and Mach5 tiers 3-5 on Linux x64. > > Thanks, Harold This pull request has now been integrated. Changeset: 92ae6a51 Author: Harold Seigel URL: https://git.openjdk.java.net/jdk/commit/92ae6a512340485f75a12479dc1c1b8d3261bc76 Stats: 21 lines in 6 files changed: 6 ins; 9 del; 6 mod 8244162: Additional opportunities to use NONCOPYABLE Reviewed-by: dholmes, kbarrett, shade ------------- PR: https://git.openjdk.java.net/jdk/pull/4652 From shade at openjdk.java.net Mon Jul 12 16:00:53 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 12 Jul 2021 16:00:53 GMT Subject: RFR: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible [v4] In-Reply-To: References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: On Mon, 12 Jul 2021 11:21:27 GMT, Roman Kennke wrote: >> We've observed test failures in jcstress, see: >> https://bugs.openjdk.java.net/browse/JDK-8269897 >> >> We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. >> >> A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. >> This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. >> >> Testing: >> - [x] the provided jcstress test >> - [x] hotspot_gc_shenandoah >> - [x] tier1 > > Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: > > Aleksey's comments This still passes testing and looks fine to me. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4697 From ihse at openjdk.java.net Mon Jul 12 18:35:11 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 12 Jul 2021 18:35:11 GMT Subject: RFR: 8270320: JDK-8270110 committed invalid copyright headers Message-ID: The fix for JDK-8270110 added two new files with copyright headers that do not follow the prescribed format. ------------- Commit messages: - 8270320: JDK-8270110 committed invalid copyright headers Changes: https://git.openjdk.java.net/jdk/pull/4757/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4757&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8270320 Stats: 2 lines in 2 files changed: 2 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/4757.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4757/head:pull/4757 PR: https://git.openjdk.java.net/jdk/pull/4757 From kvn at openjdk.java.net Mon Jul 12 18:54:01 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 12 Jul 2021 18:54:01 GMT Subject: RFR: 8270320: JDK-8270110 committed invalid copyright headers In-Reply-To: References: Message-ID: On Mon, 12 Jul 2021 18:26:53 GMT, Magnus Ihse Bursie wrote: > The fix for JDK-8270110 added two new files with copyright headers that do not follow the prescribed format. Trivial. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4757 From ihse at openjdk.java.net Mon Jul 12 18:59:57 2021 From: ihse at openjdk.java.net (Magnus Ihse Bursie) Date: Mon, 12 Jul 2021 18:59:57 GMT Subject: Integrated: 8270320: JDK-8270110 committed invalid copyright headers In-Reply-To: References: Message-ID: On Mon, 12 Jul 2021 18:26:53 GMT, Magnus Ihse Bursie wrote: > The fix for JDK-8270110 added two new files with copyright headers that do not follow the prescribed format. This pull request has now been integrated. Changeset: 353e9c86 Author: Magnus Ihse Bursie URL: https://git.openjdk.java.net/jdk/commit/353e9c8607e3a433aac786bd7b818d77ff75dd11 Stats: 2 lines in 2 files changed: 2 ins; 0 del; 0 mod 8270320: JDK-8270110 committed invalid copyright headers Reviewed-by: kvn ------------- PR: https://git.openjdk.java.net/jdk/pull/4757 From rkennke at openjdk.java.net Tue Jul 13 13:33:59 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Tue, 13 Jul 2021 13:33:59 GMT Subject: Integrated: 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible In-Reply-To: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> References: <5zmnbK6-6Co7oRsmFUdv5PAWxXWxnH_1EcxzWyD98KQ=.3e58011d-8fff-4998-9678-f65be97dee54@github.com> Message-ID: On Tue, 6 Jul 2021 19:00:59 GMT, Roman Kennke wrote: > We've observed test failures in jcstress, see: > https://bugs.openjdk.java.net/browse/JDK-8269897 > > We used to treat UNKNOWN reference accesses like weak accesses. UNKNOWN is used for Unsafe, reflection and JNI accesses, where it cannot be determined at compilation-time if we are accessing a regular field or a Reflection.referent field. The rationale for treating UNKNOWN as weak was that if the reference is a regular reference, then the value would be strongly reachable anyway, and only if it is a referent field would reachability matter. However, it turns out that this assumption is wrong: the test shows that a reference that is only weakly reachable can be legitimately written into a field, thus resurrecting the reference, and when that weakly reachable reference is loaded, it would be (wrongly) filtered as NULL. > > A fix is to treat UNKNOWN accesses as strong. Accessing Reference.referent via reflection, JNI or Unsafe is Bad Idea anyway. > This test shows the problem with CAS, but I believe it affects all accesses via reflection, JNI, etc. > > Testing: > - [x] the provided jcstress test > - [x] hotspot_gc_shenandoah > - [x] tier1 This pull request has now been integrated. Changeset: 7ac08161 Author: Roman Kennke URL: https://git.openjdk.java.net/jdk/commit/7ac08161a8b2080ed054bc00a30d6f694ece4df3 Stats: 116 lines in 3 files changed: 49 ins; 33 del; 34 mod 8269897: Shenandoah: Resolve UNKNOWN access strength, where possible Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/4697 From rkennke at openjdk.java.net Thu Jul 15 14:44:21 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 15 Jul 2021 14:44:21 GMT Subject: RFR: 8270554: Shenandoah: Optimize heap scan loop Message-ID: This is a fall-out from Lilliput. I noticed that in the heap scan loop, we load the size of objects in the size-based object scanner, even though all of the object closures already load the size, or at least in some cases, the Klass* which is necessary to determine the size. We can optimize that by making the scan loop co-operate with the closures. In other words, this changes the loop to avoid double-loading the Klass* in most cases in the size-based part of the scan loop. Note: the motivation in Lilliput is not performance, but correctness, because there loading the Klass* means loading the header, and this needs to be done carefully because of concurrent evacuation and concurrent locking code both messing with the header, and thus depends a lot on the actual closures to do it correctly. Implementation notes: - SH::evacuate_object() has been changed so that it can return both the forwardee and the size. I opted to return the size as return-value because otherwise I'd have to null check an incoming pointer in the cases when we're not interested in the size. The way it is done, it can simply be ignored (and optimized-out) by the compiler. - I added a do_object_size() variant to all affected iterators. I tried to do it with templates, but could not figure out how to please the compiler. - While I was at it, I marked all do_object() methods as 'inline'. - I ran some benchmarks. I think I see consistent but small improvements in evac and update-refs times, but it's not large enough to say that it is a definite improvement. Testing: - [x] hotspot_gc_shenandoah - [ ] tier1 (+UseShenandoahGC) - [x] specjvm testing ------------- Commit messages: - 8270554: Shenandoah: Optimize heap scan loop Changes: https://git.openjdk.java.net/jdk/pull/4797/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=4797&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8270554 Stats: 92 lines in 10 files changed: 59 ins; 3 del; 30 mod Patch: https://git.openjdk.java.net/jdk/pull/4797.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/4797/head:pull/4797 PR: https://git.openjdk.java.net/jdk/pull/4797 From rkennke at openjdk.java.net Thu Jul 15 14:52:45 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 15 Jul 2021 14:52:45 GMT Subject: RFR: Do old satb during interrupting young In-Reply-To: References: Message-ID: On Wed, 7 Jul 2021 16:49:58 GMT, Kelvin Nilsen wrote: > This patch includes bug fixes discovered during integration and testing of concurrent remembered set scanning. The bug is that processing of SATB barrier by young-gen collections that preempt a concurrent old-gen concurrent marking effort was ignoring pointers to old-gen. It needs to pass these pointers along to the old-gen concurrent mark effort. > > Since the bug manifests even outside concurrent remembered set scanning, we propose to upstream it now. > > The patch has been tested with jtreg:tier1_gc_shenandoah with no regressions. Looks good, thank you! ------------- Marked as reviewed by rkennke (Lead). PR: https://git.openjdk.java.net/shenandoah/pull/51 From rkennke at openjdk.java.net Thu Jul 15 14:54:41 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 15 Jul 2021 14:54:41 GMT Subject: RFR: Young refproc test In-Reply-To: References: Message-ID: On Thu, 20 May 2021 20:31:38 GMT, wadehenn wrote: > Young refproc test Ping? What happened to this? ------------- PR: https://git.openjdk.java.net/shenandoah/pull/42 From rkennke at openjdk.java.net Thu Jul 15 14:59:09 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 15 Jul 2021 14:59:09 GMT Subject: RFR: Introduce second mark bitmap to support remembered set scan In-Reply-To: References: Message-ID: <4cpvbKKkc35lCeC2WrqDiw8G7EpVBl7s9n_RBbgerzE=.719478e9-2771-43c8-8caa-c2c5b35839ae@github.com> On Mon, 21 Jun 2021 21:48:32 GMT, William Kemper wrote: > ## Motivation > After final mark, Shenandoah chooses the collection set (regions that will be reclaimed). If there is a sufficient amount of reclaimable memory in regions that are 100% garbage, it recycles them immediately and skips the evacuation and update reference phases (there are no pointers into these 100% garbage regions). Note that in this case there are still dead objects in regions outside of the collection set. During subsequent remembered set scans, Shenandoah must safely iterate objects in regions of memory indicated by marked cards. Specifically, it must not attempt to iterate over oops in the dead objects outside the collection set. Prior to the changes in this review, Generational Shenandoah disarmed these dead objects by coalescing them and then overwriting them with filler objects during the update references phase. There are some drawbacks to this approach: > > * Originally, Generational Shenandoah used the oopDesc::size API to compute the size of filler objects. This API uses the klass for the oop to provide the size. Since we were using the size method on dead oops, if there were no instances of the klass still alive it could be unloaded during weak root processing and the size method would fail. To prevent this issue, we disabled ClassUnloading. > * We now use the mark bitmap to compute the size of dead objects, but there is still a risk here: when young collections coalesce and fill dead objects during concurrent old generation marking, the mark bitmap is incomplete. > * It forced us to go through evacuation and update reference phases for every cycle. Our tests (Dacapo) show that approximately 33% of collections (with the default settings) find enough pure garbage regions to complete without evacuations. > * It forces the collector to perform work proportional to the amount of garbage outside the collection set (in the old generation). For performance reasons, the collector should avoid any work which is proportional to the amount of garbage. > > ## Overview of Changes > Rather than filling in dead objects to provide safety to the remembered set scan, the scan now uses the marking bit map to skip over dead objects. This leverages the work already completed during the marking phase and no longer requires any additional maintenance work during the update reference phase. There are, however, a few wrinkles to consider: > > * The evacuation phase needs to update the mark bitmap. This essentially mimics the work currently done to register objects with the remembered set. > * During concurrent old marking, the remembered set scan cannot rely on the mark bitmap for the old generation. For this reason, a second bitmap has been added. At the beginning of a concurrent old mark, the bitmaps are switched. During concurrent old marking, the remembered set scan will use the previous (i.e., stable) bitmap for determining object liveness. > * Though not directly related to these changes, a fix for reference processing has been implemented in terms of these changes (i.e., the fix uses the secondary bitmap). Please note, the "fix" for reference processing is incorrect. It does not preserve the atomicity required when null'ing out references. Work on reference processing is proceeding on a separate branch, unrelated to these changes. > > ## Remaining Work > > * Object registration should no longer be necessary. We can use the mark bitmap to tell us if a card has any objects. > * Shenandoah has an optimization to avoid clearing bitmap regions that are known to contain no marked bits. This optimization is currently disabled. It needs to be debugged (I think it is confused by extra marks made during evacuation). > * Region promotions that happen during a concurrent old mark must copy the bitmap of the promoted region - this is currently done in a most inefficient way. Looks good overall, only some cosmetic requests. Looks good to me. Thank you! src/hotspot/share/gc/shenandoah/shenandoahBitmapRegion.cpp line 2: > 1: /* > 2: * Copyright (c) 2013, 2021, Red Hat, Inc. All rights reserved. I believe a new file should have only 2021 here. src/hotspot/share/gc/shenandoah/shenandoahBitmapRegion.hpp line 2: > 1: /* > 2: * Copyright (c) 2013, 2021, Red Hat, Inc. All rights reserved. Should only have 2021 here. src/hotspot/share/gc/shenandoah/shenandoahBitmapRegion.hpp line 32: > 30: class ShenandoahHeapRegion; > 31: > 32: class ShenandoahBitmapRegion { It would be nice to have some comments here. What is this class? What's its purpose? Etc. ------------- Changes requested by rkennke (Lead). PR: https://git.openjdk.java.net/shenandoah/pull/50Marked as reviewed by rkennke (Lead). From wkemper at openjdk.java.net Thu Jul 15 14:59:09 2021 From: wkemper at openjdk.java.net (William Kemper) Date: Thu, 15 Jul 2021 14:59:09 GMT Subject: RFR: Introduce second mark bitmap to support remembered set scan In-Reply-To: <4cpvbKKkc35lCeC2WrqDiw8G7EpVBl7s9n_RBbgerzE=.719478e9-2771-43c8-8caa-c2c5b35839ae@github.com> References: <4cpvbKKkc35lCeC2WrqDiw8G7EpVBl7s9n_RBbgerzE=.719478e9-2771-43c8-8caa-c2c5b35839ae@github.com> Message-ID: On Tue, 29 Jun 2021 09:30:06 GMT, Roman Kennke wrote: >> ## Motivation >> After final mark, Shenandoah chooses the collection set (regions that will be reclaimed). If there is a sufficient amount of reclaimable memory in regions that are 100% garbage, it recycles them immediately and skips the evacuation and update reference phases (there are no pointers into these 100% garbage regions). Note that in this case there are still dead objects in regions outside of the collection set. During subsequent remembered set scans, Shenandoah must safely iterate objects in regions of memory indicated by marked cards. Specifically, it must not attempt to iterate over oops in the dead objects outside the collection set. Prior to the changes in this review, Generational Shenandoah disarmed these dead objects by coalescing them and then overwriting them with filler objects during the update references phase. There are some drawbacks to this approach: >> >> * Originally, Generational Shenandoah used the oopDesc::size API to compute the size of filler objects. This API uses the klass for the oop to provide the size. Since we were using the size method on dead oops, if there were no instances of the klass still alive it could be unloaded during weak root processing and the size method would fail. To prevent this issue, we disabled ClassUnloading. >> * We now use the mark bitmap to compute the size of dead objects, but there is still a risk here: when young collections coalesce and fill dead objects during concurrent old generation marking, the mark bitmap is incomplete. >> * It forced us to go through evacuation and update reference phases for every cycle. Our tests (Dacapo) show that approximately 33% of collections (with the default settings) find enough pure garbage regions to complete without evacuations. >> * It forces the collector to perform work proportional to the amount of garbage outside the collection set (in the old generation). For performance reasons, the collector should avoid any work which is proportional to the amount of garbage. >> >> ## Overview of Changes >> Rather than filling in dead objects to provide safety to the remembered set scan, the scan now uses the marking bit map to skip over dead objects. This leverages the work already completed during the marking phase and no longer requires any additional maintenance work during the update reference phase. There are, however, a few wrinkles to consider: >> >> * The evacuation phase needs to update the mark bitmap. This essentially mimics the work currently done to register objects with the remembered set. >> * During concurrent old marking, the remembered set scan cannot rely on the mark bitmap for the old generation. For this reason, a second bitmap has been added. At the beginning of a concurrent old mark, the bitmaps are switched. During concurrent old marking, the remembered set scan will use the previous (i.e., stable) bitmap for determining object liveness. >> * Though not directly related to these changes, a fix for reference processing has been implemented in terms of these changes (i.e., the fix uses the secondary bitmap). Please note, the "fix" for reference processing is incorrect. It does not preserve the atomicity required when null'ing out references. Work on reference processing is proceeding on a separate branch, unrelated to these changes. >> >> ## Remaining Work >> >> * Object registration should no longer be necessary. We can use the mark bitmap to tell us if a card has any objects. >> * Shenandoah has an optimization to avoid clearing bitmap regions that are known to contain no marked bits. This optimization is currently disabled. It needs to be debugged (I think it is confused by extra marks made during evacuation). >> * Region promotions that happen during a concurrent old mark must copy the bitmap of the promoted region - this is currently done in a most inefficient way. > > Looks good overall, only some cosmetic requests. Thanks for taking a look at this @rkennke . I apologize for the late reply (was on vacation and missed your comments). Are you concerned about the additional memory used for the second bitmap (at least two additional bits for each live object)? I think it would be possible to _only_ have the secondary bitmap slices committed during old generation marking. ------------- PR: https://git.openjdk.java.net/shenandoah/pull/50 From gnu.andrew at redhat.com Fri Jul 16 04:26:03 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Fri, 16 Jul 2021 05:26:03 +0100 Subject: [RFR] [8u] 8u302-b06 Upstream Sync In-Reply-To: <528074e9-ca65-249f-301a-c370a8f4aaff@redhat.com> References: <20210709025239.GA3047528@rincewind> <528074e9-ca65-249f-301a-c370a8f4aaff@redhat.com> Message-ID: <20210716042603.GB429558@rincewind> On 10:11 Fri 09 Jul , Aleksey Shipilev wrote: > On 7/9/21 4:52 AM, Andrew Hughes wrote: > > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/ > > > > Merge changesets: > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/corba/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jaxp/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jaxws/merge.changeset > > Look trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/jdk/merge.changeset > > Looks good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/hotspot/merge.changeset > > Looks good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/langtools/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/nashorn/merge.changeset > > Look trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b06/root/merge.changeset > > Looks good. > > > Ok to push? > > Yes. > > > -- > Thanks, > -Aleksey > Thanks. Pushed. -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From gnu.andrew at redhat.com Tue Jul 20 17:22:28 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Tue, 20 Jul 2021 18:22:28 +0100 Subject: [RFR] [8u] 8u302-b07 Upstream Sync Message-ID: <20210720172228.GB705815@rincewind> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/ Merge changesets: http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/corba/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jaxp/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jaxws/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jdk/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/hotspot/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/langtools/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/nashorn/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/root/merge.changeset Changes in aarch64-shenandoah-jdk8u302-b07: - JDK-8269388: Default build of OpenJDK 8 fails on newer GCCs with warnings as errors on format-overflow - JDK-8269468: JDK-8269388 breaks the build on older GCCs Main issues of note: None, clean merge (only HotSpot changes are to a Makefile). diffstat for root b/.hgtags | 1 b/common/autoconf/flags.m4 | 7 +- b/common/autoconf/generated-configure.sh | 101 ++++++++++++++++++++++++++++++- b/common/autoconf/hotspot-spec.gmk.in | 1 4 files changed, 107 insertions(+), 3 deletions(-) diffstat for corba b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxp b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxws b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for langtools b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for nashorn b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jdk b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for hotspot b/.hgtags | 1 + b/make/linux/makefiles/gcc.make | 12 ++++++++++++ 2 files changed, 13 insertions(+) Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero), ppc64, ppc64le, aarch32 (Zero) & aarch64. Ok to push? Thanks, -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From shade at redhat.com Tue Jul 20 18:50:50 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 20 Jul 2021 21:50:50 +0300 Subject: [RFR] [8u] 8u302-b07 Upstream Sync In-Reply-To: <20210720172228.GB705815@rincewind> References: <20210720172228.GB705815@rincewind> Message-ID: <119d448d-37c8-3d0a-a04a-9a008e6f76ad@redhat.com> On 7/20/21 8:22 PM, Andrew Hughes wrote: > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/ > > Merge changesets: > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/corba/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jaxp/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jaxws/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jdk/merge.changeset Look trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/hotspot/merge.changeset Looks fine. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/langtools/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/nashorn/merge.changeset Look trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/root/merge.changeset Looks fine. > Ok to push? Yes. -- Thanks, -Aleksey From gnu.andrew at redhat.com Wed Jul 21 17:48:32 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 21 Jul 2021 18:48:32 +0100 Subject: [RFR] [8u] 8u302-b07 Upstream Sync In-Reply-To: <119d448d-37c8-3d0a-a04a-9a008e6f76ad@redhat.com> References: <20210720172228.GB705815@rincewind> <119d448d-37c8-3d0a-a04a-9a008e6f76ad@redhat.com> Message-ID: <20210721174832.GE754562@rincewind> On 21:50 Tue 20 Jul , Aleksey Shipilev wrote: > On 7/20/21 8:22 PM, Andrew Hughes wrote: > > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/ > > > > Merge changesets: > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/corba/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jaxp/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jaxws/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/jdk/merge.changeset > > Look trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/hotspot/merge.changeset > > Looks fine. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/langtools/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/nashorn/merge.changeset > > Look trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b07/root/merge.changeset > > Looks fine. > > > Ok to push? > > Yes. > > -- > Thanks, > -Aleksey > Thanks, pushed. -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From rkennke at openjdk.java.net Thu Jul 22 14:19:26 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 22 Jul 2021 14:19:26 GMT Subject: RFR: Concurrent remembered set scanning [v3] In-Reply-To: References: Message-ID: On Wed, 21 Jul 2021 17:15:32 GMT, Kelvin Nilsen wrote: >> This pull request includes code to perform remembered set scanning concurrently with ongoing mutator execution. As currently implemented, the remembered set is copied to a read-only remembered set during init-mark safepoint. After copying, all cards in the remembered set are restored to a clean value. Following the init-mark safepoint, mutator thread write barriers update the active remembered set copy every time a reference field is overwritten by dirtying the corresponding cards. >> >> Concurrent remembered set scanning for the purpose of marking works with the read-only copy of the remembered set. When concurrent remembered set scanning encounters an interesting pointer, the corresponding card of the active remembered set is overwritten dirty. >> >> The remembered set is scanned a second time concurrently during the update-refs phase of gc. The second scan uses the same read/write active copy of the remembered set that is being overwritten by the mutator write barrier. >> >> The SATB barrier and load reference barriers are sufficient to resolve any races between setting of dirty flags and scanning of the remembered set. >> >> A possible future enhancement involves replacing the remembered set copy (during init-mark safepoint) with a simple pointer swap operation to exchange the roles of a read-byte-map and a write-byte-map. Making this change is deferred because it requires far-reaching changes to the implementations of mutator write barriers. As currently implemented, these barriers hard-code the address of the card-table byte map. >> >> This patch passes tier1 and hotspot_gc_shenandoah regression tests. > > Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision: > > Remove extraneous space from comment Looks good now! Thank you! ------------- Marked as reviewed by rkennke (Lead). PR: https://git.openjdk.java.net/shenandoah/pull/37 From kdnilsen at openjdk.java.net Thu Jul 22 14:32:29 2021 From: kdnilsen at openjdk.java.net (Kelvin Nilsen) Date: Thu, 22 Jul 2021 14:32:29 GMT Subject: Integrated: Concurrent remembered set scanning In-Reply-To: References: Message-ID: On Tue, 11 May 2021 14:40:15 GMT, Kelvin Nilsen wrote: > This pull request includes code to perform remembered set scanning concurrently with ongoing mutator execution. As currently implemented, the remembered set is copied to a read-only remembered set during init-mark safepoint. After copying, all cards in the remembered set are restored to a clean value. Following the init-mark safepoint, mutator thread write barriers update the active remembered set copy every time a reference field is overwritten by dirtying the corresponding cards. > > Concurrent remembered set scanning for the purpose of marking works with the read-only copy of the remembered set. When concurrent remembered set scanning encounters an interesting pointer, the corresponding card of the active remembered set is overwritten dirty. > > The remembered set is scanned a second time concurrently during the update-refs phase of gc. The second scan uses the same read/write active copy of the remembered set that is being overwritten by the mutator write barrier. > > The SATB barrier and load reference barriers are sufficient to resolve any races between setting of dirty flags and scanning of the remembered set. > > A possible future enhancement involves replacing the remembered set copy (during init-mark safepoint) with a simple pointer swap operation to exchange the roles of a read-byte-map and a write-byte-map. Making this change is deferred because it requires far-reaching changes to the implementations of mutator write barriers. As currently implemented, these barriers hard-code the address of the card-table byte map. > > This patch passes tier1 and hotspot_gc_shenandoah regression tests. This pull request has now been integrated. Changeset: d775516e Author: Kelvin Nilsen Committer: Roman Kennke URL: https://git.openjdk.java.net/shenandoah/commit/d775516e9b2216735fa059f8e986c7a8d280c9f0 Stats: 1658 lines in 46 files changed: 1287 ins; 123 del; 248 mod Concurrent remembered set scanning Reviewed-by: rkennke ------------- PR: https://git.openjdk.java.net/shenandoah/pull/37 From rkennke at redhat.com Thu Jul 22 15:48:37 2021 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 22 Jul 2021 17:48:37 +0200 Subject: CFV: New Shenandoah Committer: Kelvin Nilsen Message-ID: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> I hereby nominate Kelvin Nilsen to Shenandoah Committer. Kelvin is an engineer at Amazon and has been a principal author in the generational Shenandoah effort. His contributions so far, some of them very large: https://github.com/openjdk/shenandoah/commits?author=kdnilsen Votes are due by August 2nd 2021 6:00pm CEST. Only current Shenandoah Committers [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Roman Kennke [1] https://openjdk.java.net/census [2] https://openjdk.java.net/projects/#committer-vote From wkemper at openjdk.java.net Thu Jul 22 15:55:35 2021 From: wkemper at openjdk.java.net (William Kemper) Date: Thu, 22 Jul 2021 15:55:35 GMT Subject: RFR: Check that pointer is in heap before checking the affiliation of owning region Message-ID: This is a small bug fix that was meant to go in with the concurrent remembered set scanning changes ------------- Commit messages: - Check that pointer is in heap before checking the affiliation of owning region Changes: https://git.openjdk.java.net/shenandoah/pull/52/files Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=52&range=00 Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/shenandoah/pull/52.diff Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/52/head:pull/52 PR: https://git.openjdk.java.net/shenandoah/pull/52 From rkennke at redhat.com Thu Jul 22 16:06:53 2021 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 22 Jul 2021 18:06:53 +0200 Subject: CFV: New Shenandoah Committer: Kelvin Nilsen In-Reply-To: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> References: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> Message-ID: Vote: yes Thanks, Roman > I hereby nominate Kelvin Nilsen to Shenandoah Committer. > > Kelvin is an engineer at Amazon and has been a principal author in the > generational Shenandoah effort. > > His contributions so far, some of them very large: > https://github.com/openjdk/shenandoah/commits?author=kdnilsen > > Votes are due by August 2nd 2021 6:00pm CEST. > > Only current Shenandoah Committers [1] are eligible to vote > on this nomination.? Votes must be cast in the open by replying > to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Roman Kennke > > [1] https://openjdk.java.net/census > [2] https://openjdk.java.net/projects/#committer-vote From rkennke at openjdk.java.net Thu Jul 22 16:08:25 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 22 Jul 2021 16:08:25 GMT Subject: RFR: Check that pointer is in heap before checking the affiliation of owning region In-Reply-To: References: Message-ID: <34u4CguWQqke68KyaiRNhVVgxVKJ7hijQqr0PPqs8Ew=.b7e1f4d4-b126-45d0-aefb-eb80e6679a7e@github.com> On Thu, 22 Jul 2021 15:48:08 GMT, William Kemper wrote: > This is a small bug fix that was meant to go in with the concurrent remembered set scanning changes Changes requested by rkennke (Lead). src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 756: > 754: > 755: bool ShenandoahHeap::is_in_young(const void* p) const { > 756: return is_in(p) && heap_region_containing(p)->affiliation() == ShenandoahRegionAffiliation::YOUNG_GENERATION; Is there a valid case where we would check off-heap pointers? If not, then better do this as assert? src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 399: > 397: // When this reference was discovered, it would not have been marked. If it ends up surviving > 398: // the cycle, we need to dirty the card if the reference is old and the referent is young. Note > 399: // that if the reference is not dropped, then its pointer to the referent will be cleared before Those changes seem unrelated. ------------- PR: https://git.openjdk.java.net/shenandoah/pull/52 From wkemper at openjdk.java.net Thu Jul 22 17:33:41 2021 From: wkemper at openjdk.java.net (William Kemper) Date: Thu, 22 Jul 2021 17:33:41 GMT Subject: RFR: Check that pointer is in heap before checking the affiliation of owning region In-Reply-To: <34u4CguWQqke68KyaiRNhVVgxVKJ7hijQqr0PPqs8Ew=.b7e1f4d4-b126-45d0-aefb-eb80e6679a7e@github.com> References: <34u4CguWQqke68KyaiRNhVVgxVKJ7hijQqr0PPqs8Ew=.b7e1f4d4-b126-45d0-aefb-eb80e6679a7e@github.com> Message-ID: <922bAQZHID5z66NIN4307-Y8xAU8yFIzy_B7lGB39To=.630b050c-1fb1-4aef-99bd-fa5a447200b4@github.com> On Thu, 22 Jul 2021 16:05:07 GMT, Roman Kennke wrote: >> This is a small bug fix that was meant to go in with the concurrent remembered set scanning changes > > src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 756: > >> 754: >> 755: bool ShenandoahHeap::is_in_young(const void* p) const { >> 756: return is_in(p) && heap_region_containing(p)->affiliation() == ShenandoahRegionAffiliation::YOUNG_GENERATION; > > Is there a valid case where we would check off-heap pointers? If not, then better do this as assert? This also handles the case of `NULL` pointers and there are times when this API could be called for off-heap roots (during scanning). > src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 399: > >> 397: // When this reference was discovered, it would not have been marked. If it ends up surviving >> 398: // the cycle, we need to dirty the card if the reference is old and the referent is young. Note >> 399: // that if the reference is not dropped, then its pointer to the referent will be cleared before > > Those changes seem unrelated. Will revert this change. ------------- PR: https://git.openjdk.java.net/shenandoah/pull/52 From kemperw at amazon.com Thu Jul 22 17:36:34 2021 From: kemperw at amazon.com (Kemper, William) Date: Thu, 22 Jul 2021 17:36:34 +0000 Subject: CFV: New Shenandoah Committer: Kelvin Nilsen In-Reply-To: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> References: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> Message-ID: <1626975394559.48377@amazon.com> Vote: yes ________________________________________ From: shenandoah-dev on behalf of Roman Kennke Sent: Thursday, July 22, 2021 8:48 AM To: shenandoah-dev at openjdk.java.net Subject: CFV: New Shenandoah Committer: Kelvin Nilsen I hereby nominate Kelvin Nilsen to Shenandoah Committer. Kelvin is an engineer at Amazon and has been a principal author in the generational Shenandoah effort. His contributions so far, some of them very large: https://github.com/openjdk/shenandoah/commits?author=kdnilsen Votes are due by August 2nd 2021 6:00pm CEST. Only current Shenandoah Committers [1] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Lazy Consensus voting instructions, see [2]. Roman Kennke [1] https://openjdk.java.net/census [2] https://openjdk.java.net/projects/#committer-vote From wkemper at openjdk.java.net Thu Jul 22 17:46:51 2021 From: wkemper at openjdk.java.net (William Kemper) Date: Thu, 22 Jul 2021 17:46:51 GMT Subject: RFR: Check that pointer is in heap before checking the affiliation of owning region [v2] In-Reply-To: References: Message-ID: > This is a small bug fix that was meant to go in with the concurrent remembered set scanning changes William Kemper has updated the pull request incrementally with one additional commit since the last revision: Revert unrelated change to comment ------------- Changes: - all: https://git.openjdk.java.net/shenandoah/pull/52/files - new: https://git.openjdk.java.net/shenandoah/pull/52/files/e2e979bb..1236af75 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=52&range=01 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=52&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/shenandoah/pull/52.diff Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/52/head:pull/52 PR: https://git.openjdk.java.net/shenandoah/pull/52 From wkemper at openjdk.java.net Thu Jul 22 22:59:50 2021 From: wkemper at openjdk.java.net (William Kemper) Date: Thu, 22 Jul 2021 22:59:50 GMT Subject: RFR: Generational support for weak roots and references Message-ID: ### Summary The LRB for non-strong references is modified to permit resurrection of objects outside the generation being collected. In other words, resurrection is only blocked for unmarked objects in the generation being collected. Each `ShenandoahGeneration` has its own reference processor instance. In some cases, a reference from the old generation may end up on the young generation discovered list if the reference points to a young referent (this would happen if the old reference is in the remembered set). However, young references that point to referents in the old generation are _not_ discovered. This has the effect of strongly marking the old generation referent. This also avoids the case of having young references on the old generation discovered list being evacuated/relocated while they wait for old generation reference processing (although we believe this case would be handled correctly by the existing update references code). ------------- Commit messages: - Configure soft reference policy for old generation - Merge branch 'shenandoah' into generational-weak-refs - Perform coalesce and fill after class unloading - Formatting fixes, remove TODO - Do not discover referents outside the active generation - Move reference processor into generation abstraction - Fix indentation, use override keyword rather than redundant virtual - Rename method for consistency with existing APIs - Improve type safety for affiliation_name API - Coalesce and fill after processing old weak roots to preserve existing assertions - ... and 6 more: https://git.openjdk.java.net/shenandoah/compare/d775516e...80821d08 Changes: https://git.openjdk.java.net/shenandoah/pull/53/files Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=53&range=00 Stats: 143 lines in 21 files changed: 77 ins; 22 del; 44 mod Patch: https://git.openjdk.java.net/shenandoah/pull/53.diff Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/53/head:pull/53 PR: https://git.openjdk.java.net/shenandoah/pull/53 From gnu.andrew at redhat.com Fri Jul 23 05:42:19 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Fri, 23 Jul 2021 06:42:19 +0100 Subject: [RFR] [8u] 8u302-b08 Upstream Sync Message-ID: <20210723054219.GA799068@rincewind> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/ Merge changesets: http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/corba/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jaxp/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jaxws/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jdk/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/hotspot/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/langtools/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/nashorn/merge.changeset http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/root/merge.changeset Changes in aarch64-shenandoah-jdk8u302-b08: - JDK-8256157: Improve bytecode assembly - JDK-8256491: Better HTTP transport - JDK-8258432: Improve file transfers - JDK-8260453: Improve Font Bounding - JDK-8260960: Signs of jarsigner signing - JDK-8260967: Better jar file validation - JDK-8262380: Enhance XML processing passes - JDK-8262403: Enhanced data transfers - JDK-8262410: Enhanced rules for zones - JDK-8262477: Enhance String Conclusions - JDK-8262967: Improve Zip file support - JDK-8264066: Enhance compiler validation - JDK-8264079: Improve abstractions - JDK-8264460: Improve NTLM support - JDK-8270533: AArch64: size_fits_all_mem_uses should return false if its output is a CAS Main issues of note: None, clean merge. diffstat for root b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for corba b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jaxp b/.hgtags | 1 b/src/com/sun/org/apache/xerces/internal/impl/XML11EntityScanner.java | 6 ++- b/src/com/sun/org/apache/xerces/internal/impl/XMLEntityScanner.java | 11 +++++ b/src/com/sun/org/apache/xerces/internal/impl/msg/XMLMessages.properties | 2 - b/src/com/sun/org/apache/xml/internal/serializer/ToStream.java | 19 +++++----- b/src/jdk/xml/internal/JdkXmlUtils.java | 18 ++++++++- 6 files changed, 43 insertions(+), 14 deletions(-) diffstat for jaxws b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for langtools b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for nashorn b/.hgtags | 1 + 1 file changed, 1 insertion(+) diffstat for jdk b/.hgtags | 1 b/make/mapfiles/libzip/mapfile-vers | 1 b/make/mapfiles/libzip/reorder-sparc | 1 b/make/mapfiles/libzip/reorder-sparcv9 | 1 b/make/mapfiles/libzip/reorder-x86 | 1 b/src/share/classes/com/sun/security/ntlm/NTLM.java | 4 b/src/share/classes/java/awt/datatransfer/MimeType.java | 10 b/src/share/classes/java/time/zone/ZoneRules.java | 14 + b/src/share/classes/java/util/jar/JarFile.java | 13 - b/src/share/classes/java/util/jar/JarInputStream.java | 4 b/src/share/classes/java/util/jar/JarVerifier.java | 21 + b/src/share/classes/java/util/zip/ZipFile.java | 22 + b/src/share/classes/jdk/internal/org/objectweb/asm/ClassReader.java | 6 b/src/share/classes/sun/font/TrueTypeFont.java | 24 +- b/src/share/classes/sun/misc/JavaUtilZipFileAccess.java | 4 b/src/share/classes/sun/net/ftp/impl/FtpClient.java | 116 +++++++++- b/src/share/classes/sun/net/httpserver/ServerImpl.java | 59 ++++- b/src/share/classes/sun/security/tools/jarsigner/Main.java | 25 +- b/src/share/classes/sun/security/tools/jarsigner/Resources.java | 7 b/src/share/classes/sun/security/util/SignatureFileVerifier.java | 10 b/src/share/native/java/util/zip/ZipFile.c | 8 b/src/share/native/java/util/zip/zip_util.c | 29 ++ b/src/share/native/java/util/zip/zip_util.h | 1 b/test/sun/security/tools/jarsigner/concise_jarsigner.sh | 20 + 24 files changed, 343 insertions(+), 59 deletions(-) diffstat for hotspot b/.hgtags | 1 b/src/cpu/aarch64/vm/aarch64.ad | 7 + b/src/share/vm/c1/c1_RangeCheckElimination.cpp | 4 b/src/share/vm/code/dependencies.cpp | 106 ++++++++++++++++++++----- b/src/share/vm/code/dependencies.hpp | 3 5 files changed, 101 insertions(+), 20 deletions(-) Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero), ppc64, ppc64le, aarch32 (Zero) & aarch64. Ok to push? Thanks, -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From shade at redhat.com Fri Jul 23 05:50:18 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 23 Jul 2021 08:50:18 +0300 Subject: [RFR] [8u] 8u302-b08 Upstream Sync In-Reply-To: <20210723054219.GA799068@rincewind> References: <20210723054219.GA799068@rincewind> Message-ID: <81b105ae-d3a3-a7a6-5a0e-10f5f2ed3c4a@redhat.com> On 7/23/21 8:42 AM, Andrew Hughes wrote: > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/corba/merge.changeset Looks trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jaxp/merge.changeset Looks good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jaxws/merge.changeset Looks trivially good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jdk/merge.changeset Looks good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/hotspot/merge.changeset Looks good. > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/langtools/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/nashorn/merge.changeset > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/root/merge.changeset Looks trivially good. > Ok to push? Yes. -- Thanks, -Aleksey From gnu.andrew at redhat.com Fri Jul 23 06:41:34 2021 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Fri, 23 Jul 2021 07:41:34 +0100 Subject: [RFR] [8u] 8u302-b08 Upstream Sync In-Reply-To: <81b105ae-d3a3-a7a6-5a0e-10f5f2ed3c4a@redhat.com> References: <20210723054219.GA799068@rincewind> <81b105ae-d3a3-a7a6-5a0e-10f5f2ed3c4a@redhat.com> Message-ID: <20210723064134.GB799068@rincewind> On 08:50 Fri 23 Jul , Aleksey Shipilev wrote: > On 7/23/21 8:42 AM, Andrew Hughes wrote: > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/corba/merge.changeset > > Looks trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jaxp/merge.changeset > > Looks good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jaxws/merge.changeset > > Looks trivially good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/jdk/merge.changeset > > Looks good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/hotspot/merge.changeset > > Looks good. > > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/langtools/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/nashorn/merge.changeset > > http://cr.openjdk.java.net/~andrew/shenandoah-8/u302-b08/root/merge.changeset > > Looks trivially good. > > > Ok to push? > > Yes. > > -- > Thanks, > -Aleksey > Thanks, pushed. -- Andrew :) Pronouns: he / him or they / them Senior Free Java Software Engineer OpenJDK Package Owner Red Hat, Inc. (http://www.redhat.com) PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From shade at redhat.com Fri Jul 23 10:02:02 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 23 Jul 2021 13:02:02 +0300 Subject: CFV: New Shenandoah Committer: Kelvin Nilsen In-Reply-To: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> References: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> Message-ID: Vote: yes On 7/22/21 6:48 PM, Roman Kennke wrote: > I hereby nominate Kelvin Nilsen to Shenandoah Committer. -- Thanks, -Aleksey From rkennke at openjdk.java.net Fri Jul 23 10:40:35 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Fri, 23 Jul 2021 10:40:35 GMT Subject: RFR: Generational support for weak roots and references In-Reply-To: References: Message-ID: On Thu, 22 Jul 2021 22:53:56 GMT, William Kemper wrote: > ### Summary > The LRB for non-strong references is modified to permit resurrection of objects outside the generation being collected. In other words, resurrection is only blocked for unmarked objects in the generation being collected. > > Each `ShenandoahGeneration` has its own reference processor instance. In some cases, a reference from the old generation may end up on the young generation discovered list if the reference points to a young referent (this would happen if the old reference is in the remembered set). However, young references that point to referents in the old generation are _not_ discovered. This has the effect of strongly marking the old generation referent. This also avoids the case of having young references on the old generation discovered list being evacuated/relocated while they wait for old generation reference processing (although we believe this case would be handled correctly by the existing update references code). Before I review this, can you give me more details of your thinking? In particular: - why do we need one reference processor per generation? Why is a single global reference processor not enough? - why would we want to allow resurrection of objects in the not-collected generation? ------------- PR: https://git.openjdk.java.net/shenandoah/pull/53 From wkemper at openjdk.java.net Fri Jul 23 18:52:30 2021 From: wkemper at openjdk.java.net (William Kemper) Date: Fri, 23 Jul 2021 18:52:30 GMT Subject: RFR: Generational support for weak roots and references In-Reply-To: References: Message-ID: On Thu, 22 Jul 2021 22:53:56 GMT, William Kemper wrote: > ### Summary > The LRB for non-strong references is modified to permit resurrection of objects outside the generation being collected. In other words, resurrection is only blocked for unmarked objects in the generation being collected. > > Each `ShenandoahGeneration` has its own reference processor instance. In some cases, a reference from the old generation may end up on the young generation discovered list if the reference points to a young referent (this would happen if the old reference is in the remembered set). However, young references that point to referents in the old generation are _not_ discovered. This has the effect of strongly marking the old generation referent. This also avoids the case of having young references on the old generation discovered list being evacuated/relocated while they wait for old generation reference processing (although we believe this case would be handled correctly by the existing update references code). Yes of course, there are many scenarios to consider here. We did consider a solution that would have the collector process references only during a _young_ and _global_ collection, but our ultimate goal is to eliminate the _global_ collections from generational mode. The need to have a reference processor per generation comes from the requirement for all references to a referent to be cleared _atomically_ by the collector. If we have only one reference processor, it would necessarily be shared by the young and old collection cycles, however this would break the _atomically_ cleared requirement. Young collection cycles are allowed to interrupt old collections. A young cycle which interrupts old marking would not be allowed to process references because marking of old (discovering of all references) would not yet be complete[1]. So we have two separate discovered lists, separated by the generation of their _referents_. Since a young collection will only collect young _referents_, we put old and young _references_ alike on the same _young_ discovered list. This also motivates the changes in the LRB - I probably did not explain that well in the summary. The LRB code is such that it blocks resurrection only when `concurrent_weak_root_in_progress`, with the intention that weak references/roots pointing to _unmarked_ objects are destined to be cleared and so can no longer be accessed (resurrected). With two generations, it now matters which one is being collected. The weak roots are off heap[2] and so themselves are not distinguishable by generation, however, they do hold references to objects in the heap. At the beginning of old marking, the mark bitmaps are cleared. If a young collection occurs _before_ the old marking is complete it will block resurrection while `concurrent_weak_root_in_progress` is true - however, the mark bitmap for the old generation is incomplete so it should _not_ block resurrection for old objects. In other words, this change in the LRB effectively gives us `old_concurrent_weak_root_in_progress` and `young_concurr ent_weak_root_in_progress` without having to introduce the complexity of additional states to the collector's state machine. [1] As I think on this, we could consider a design that would have young cycles which interrupt old marking _defer_ reference processing to the old generation cycle, but we would need to be careful of dangling referents (that is, referents which were _not_ strongly reachable at the end of the young cycle would be collected, but the reference to them would not yet be cleared). It doesn't seem like this would be safe. [2] This is certainly true for _some_ weak roots, but I'm not sure it's true for _all_ weak roots. ------------- PR: https://git.openjdk.java.net/shenandoah/pull/53 From zgu at redhat.com Sun Jul 25 23:17:45 2021 From: zgu at redhat.com (Zhengyu Gu) Date: Sun, 25 Jul 2021 19:17:45 -0400 Subject: CFV: New Shenandoah Committer: Kelvin Nilsen In-Reply-To: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> References: <0c60e9c8-0dd2-df39-0217-7b11afaf022d@redhat.com> Message-ID: Vote: yes. -Zhengyu On 7/22/21 11:48 AM, Roman Kennke wrote: > I hereby nominate Kelvin Nilsen to Shenandoah Committer. > > Kelvin is an engineer at Amazon and has been a principal author in the > generational Shenandoah effort. > > His contributions so far, some of them very large: > https://github.com/openjdk/shenandoah/commits?author=kdnilsen > > Votes are due by August 2nd 2021 6:00pm CEST. > > Only current Shenandoah Committers [1] are eligible to vote > on this nomination.? Votes must be cast in the open by replying > to this mailing list. > > For Lazy Consensus voting instructions, see [2]. > > Roman Kennke > > [1] https://openjdk.java.net/census > [2] https://openjdk.java.net/projects/#committer-vote > From shade at openjdk.java.net Mon Jul 26 07:48:08 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 26 Jul 2021 07:48:08 GMT Subject: RFR: 8269924: Shenandoah: Introduce weak/strong marking asserts In-Reply-To: References: Message-ID: On Tue, 6 Jul 2021 15:13:07 GMT, Aleksey Shipilev wrote: > For debugging, it is sometimes important to have not only `shenandoah_assert_marked`, but also `shenandoah_assert_marked_(weak|strong)` flavors. No takers? :) ------------- PR: https://git.openjdk.java.net/jdk/pull/4691 From rkennke at openjdk.java.net Mon Jul 26 11:03:10 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Mon, 26 Jul 2021 11:03:10 GMT Subject: RFR: 8269924: Shenandoah: Introduce weak/strong marking asserts In-Reply-To: References: Message-ID: On Tue, 6 Jul 2021 15:13:07 GMT, Aleksey Shipilev wrote: > For debugging, it is sometimes important to have not only `shenandoah_assert_marked`, but also `shenandoah_assert_marked_(weak|strong)` flavors. Looks good. ------------- Marked as reviewed by rkennke (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/4691 From shade at openjdk.java.net Mon Jul 26 14:45:13 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 26 Jul 2021 14:45:13 GMT Subject: RFR: 8269924: Shenandoah: Introduce weak/strong marking asserts In-Reply-To: References: Message-ID: On Tue, 6 Jul 2021 15:13:07 GMT, Aleksey Shipilev wrote: > For debugging, it is sometimes important to have not only `shenandoah_assert_marked`, but also `shenandoah_assert_marked_(weak|strong)` flavors. Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/4691 From shade at openjdk.java.net Mon Jul 26 14:45:14 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 26 Jul 2021 14:45:14 GMT Subject: Integrated: 8269924: Shenandoah: Introduce weak/strong marking asserts In-Reply-To: References: Message-ID: On Tue, 6 Jul 2021 15:13:07 GMT, Aleksey Shipilev wrote: > For debugging, it is sometimes important to have not only `shenandoah_assert_marked`, but also `shenandoah_assert_marked_(weak|strong)` flavors. This pull request has now been integrated. Changeset: 515113d8 Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/515113d8589875e76980ebf7473cb81f778bb86f Stats: 46 lines in 2 files changed: 46 ins; 0 del; 0 mod 8269924: Shenandoah: Introduce weak/strong marking asserts Reviewed-by: rkennke ------------- PR: https://git.openjdk.java.net/jdk/pull/4691 From zgu at redhat.com Mon Jul 26 18:48:47 2021 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 26 Jul 2021 14:48:47 -0400 Subject: [sh/jdk8u] RFR 8269661: JNI_GetStringCritical does not lock char array Message-ID: This is the bug that only impacts Shenandoah GC, as it is the only GC supports object pinning and string deduplication. Original bug: https://bugs.openjdk.java.net/browse/JDK-8269661 Original patch: https://github.com/openjdk/jdk17/commit/0f4e07b7d9190dd44b2fd65eff58fb6ec983a467 The sh/jdk8u backport this based on jdk11u backport patch: https://github.com/openjdk/jdk11u-dev/commit/2a3d9080e19282f8451cdb93e3e36191f622ea3c sh/8u webrev: http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8269661-8u/webrev.00/ The patch diverges from 11u due to that 8u does not support CompactStrings. Test: hotspot_gc_shenandoah Thanks, -Zhengyu From shade at redhat.com Tue Jul 27 13:24:38 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 27 Jul 2021 16:24:38 +0300 Subject: [sh/jdk8u] RFR 8269661: JNI_GetStringCritical does not lock char array In-Reply-To: References: Message-ID: <78f6fe5f-9ec1-c30f-855a-78110a82c303@redhat.com> On 7/26/21 9:48 PM, Zhengyu Gu wrote: > This is the bug that only impacts Shenandoah GC, as it is the only GC > supports object pinning and string deduplication. > > Original bug: https://bugs.openjdk.java.net/browse/JDK-8269661 > Original patch: > https://github.com/openjdk/jdk17/commit/0f4e07b7d9190dd44b2fd65eff58fb6ec983a467 > > The sh/jdk8u backport this based on jdk11u backport patch: > https://github.com/openjdk/jdk11u-dev/commit/2a3d9080e19282f8451cdb93e3e36191f622ea3c > > sh/8u webrev: > http://cr.openjdk.java.net/~zgu/shenandoah/JDK-8269661-8u/webrev.00/ This looks fine. -- Thanks, -Aleksey From zgu at redhat.com Tue Jul 27 15:13:18 2021 From: zgu at redhat.com (zgu at redhat.com) Date: Tue, 27 Jul 2021 15:13:18 +0000 Subject: hg: shenandoah/jdk8/hotspot: [backport] 8269661: JNI_GetStringCritical does not lock char array Message-ID: <202107271513.16RFDIKE021301@aojmv0008.oracle.com> Changeset: 53d3b8842b81 Author: zgu Date: 2021-07-26 18:09 +0000 URL: https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/53d3b8842b81 [backport] 8269661: JNI_GetStringCritical does not lock char array [backport] 8269650: Optimize gc-locker in [Get|Release]StringCritical for latin string Reviewed-by: shade ! src/share/vm/prims/jni.cpp From iignatyev at openjdk.java.net Thu Jul 29 00:53:12 2021 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Thu, 29 Jul 2021 00:53:12 GMT Subject: [jdk17] RFR: 8067223: [TESTBUG] Rename Whitebox API package Message-ID: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> Hi all, could you please review this big tedious and trivial(-ish) patch which moves `sun.hotspot.WhiteBox` and related classes to `jdk.test.whitebox` package? the majority of the patch is the following substitutions: - `s~sun/hotspot/WhiteBox~jdk/test/whitebox/WhiteBox~g` - `s/sun.hotspot.parser/jdk.test.whitebox.parser/g` - `s/sun.hotspot.cpuinfo/jdk.test.whitebox.cpuinfo/g` - `s/sun.hotspot.code/jdk.test.whitebox.code/g` - `s/sun.hotspot.gc/jdk.test.whitebox.gc/g` - `s/sun.hotspot.WhiteBox/jdk.test.whitebox.WhiteBox/g` testing: tier1-4 Thanks, -- Igor ------------- Commit messages: - copyright year - update TEST.ROOT - jdk: s/sun\.hotspot\.gc/jdk.test.whitebox.gc/g - jdk: s/sun\.hotspot\.code/jdk.test.whitebox.code/g - jdk: s/sun\.hotspot\.WhiteBox/jdk.test.whitebox.WhiteBox/g - hotspot: 's~sun/hotspot/WhiteBox~jdk/test/whitebox/WhiteBox~g' - hotspot: s/sun\.hotspot\.parser/jdk.test.whitebox.parser/g - hotspot: s/sun\.hotspot\.cpuinfo/jdk.test.whitebox.cpuinfo/g - hotspot: s/sun\.hotspot\.code/jdk.test.whitebox.code/g - hotspot: 's/sun\.hotspot\.gc/jdk.test.whitebox.gc/g' - ... and 9 more: https://git.openjdk.java.net/jdk17/compare/c8ae7e5b...8f12f2cf Changes: https://git.openjdk.java.net/jdk17/pull/290/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk17&pr=290&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8067223 Stats: 2310 lines in 949 files changed: 0 ins; 0 del; 2310 mod Patch: https://git.openjdk.java.net/jdk17/pull/290.diff Fetch: git fetch https://git.openjdk.java.net/jdk17 pull/290/head:pull/290 PR: https://git.openjdk.java.net/jdk17/pull/290 From kvn at openjdk.java.net Thu Jul 29 01:33:29 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Thu, 29 Jul 2021 01:33:29 GMT Subject: [jdk17] RFR: 8067223: [TESTBUG] Rename Whitebox API package In-Reply-To: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> References: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> Message-ID: <_3viZFs2iZ00hMdeP3Nq9gTmwfIHeTzZ7NVT8thQ1BM=.10005882-f135-4592-a663-b75747da3f6c@github.com> On Wed, 28 Jul 2021 17:13:49 GMT, Igor Ignatyev wrote: > Hi all, > > could you please review this big tedious and trivial(-ish) patch which moves `sun.hotspot.WhiteBox` and related classes to `jdk.test.whitebox` package? > > the majority of the patch is the following substitutions: > - `s~sun/hotspot/WhiteBox~jdk/test/whitebox/WhiteBox~g` > - `s/sun.hotspot.parser/jdk.test.whitebox.parser/g` > - `s/sun.hotspot.cpuinfo/jdk.test.whitebox.cpuinfo/g` > - `s/sun.hotspot.code/jdk.test.whitebox.code/g` > - `s/sun.hotspot.gc/jdk.test.whitebox.gc/g` > - `s/sun.hotspot.WhiteBox/jdk.test.whitebox.WhiteBox/g` > > testing: tier1-4 > > Thanks, > -- Igor I know that tests fixes could be pushed during RDP2 without approval. But these one is very big and it is not a fix - it is enhancement. What is the reason you want to push it into JDK 17 just few days before first Release Candidate? Instead of pushing it into JDK 18. And I can't even review it because GutHub UI hangs on these big changes. ------------- PR: https://git.openjdk.java.net/jdk17/pull/290 From dholmes at openjdk.java.net Thu Jul 29 01:59:34 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 29 Jul 2021 01:59:34 GMT Subject: [jdk17] RFR: 8067223: [TESTBUG] Rename Whitebox API package In-Reply-To: <_3viZFs2iZ00hMdeP3Nq9gTmwfIHeTzZ7NVT8thQ1BM=.10005882-f135-4592-a663-b75747da3f6c@github.com> References: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> <_3viZFs2iZ00hMdeP3Nq9gTmwfIHeTzZ7NVT8thQ1BM=.10005882-f135-4592-a663-b75747da3f6c@github.com> Message-ID: On Thu, 29 Jul 2021 01:30:37 GMT, Vladimir Kozlov wrote: >> Hi all, >> >> could you please review this big tedious and trivial(-ish) patch which moves `sun.hotspot.WhiteBox` and related classes to `jdk.test.whitebox` package? >> >> the majority of the patch is the following substitutions: >> - `s~sun/hotspot/WhiteBox~jdk/test/whitebox/WhiteBox~g` >> - `s/sun.hotspot.parser/jdk.test.whitebox.parser/g` >> - `s/sun.hotspot.cpuinfo/jdk.test.whitebox.cpuinfo/g` >> - `s/sun.hotspot.code/jdk.test.whitebox.code/g` >> - `s/sun.hotspot.gc/jdk.test.whitebox.gc/g` >> - `s/sun.hotspot.WhiteBox/jdk.test.whitebox.WhiteBox/g` >> >> testing: tier1-4 >> >> Thanks, >> -- Igor > > I know that tests fixes could be pushed during RDP2 without approval. > But these one is very big and it is not a fix - it is enhancement. > > What is the reason you want to push it into JDK 17 just few days before first Release Candidate? Instead of pushing it into JDK 18. > > And I can't even review it because GutHub UI hangs on these big changes. I agree with @vnkozlov this too big and disruptive for 17 at this stage of the release. I also think a better approach to this cleanup would have been to copy the WhiteBox class to the new package structure, then update the tests in chunks, then remove the old WhiteBox classes when that is complete. Thanks, David ------------- PR: https://git.openjdk.java.net/jdk17/pull/290 From zgu at redhat.com Thu Jul 29 13:07:05 2021 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 29 Jul 2021 09:07:05 -0400 Subject: [sh/jdk8u] MetadataOnStackMark::record() is not MT-safe Message-ID: <7cd3b64f-0312-8586-7385-311a41a54b0d@redhat.com> This is not a Shenandoah specific bug, I believe G1 suffers the same problem. The reason to fix Shenandoah first, in hope that we can have a build for our customer to test this fix, as we don't have a reproducer. I can move the fix to jdk8u if you think otherwise. Apparently, MetadataOnStackMark::record() was never designed for MT, but G1 and Shenandoah's parallel cleaning code can call nmethod::do_unloading_parallel() from different worker threads, and nmethod::do_unloading_parallel() in turn calls nmethod::clean_ic_if_metadata_is_dead() which may call Metadata::mark_on_stack() to record metadata to MetadataOnStackMark(). Calling MetadataOnStackMark::record() without synchronization, may corrupt internal buffer, which can result fatal error in MetadataOnStackMark's destructor, when it walks the buffers. This problem is hard to reproduce, cause it is only possible when there are redefined classes. Webrev: http://cr.openjdk.java.net/~zgu/shenandoah/metadata_on_stack/webrev.00/ Thanks, -Zhengyu From rkennke at redhat.com Thu Jul 29 14:22:40 2021 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Jul 2021 16:22:40 +0200 Subject: [sh/jdk8u] MetadataOnStackMark::record() is not MT-safe In-Reply-To: <7cd3b64f-0312-8586-7385-311a41a54b0d@redhat.com> References: <7cd3b64f-0312-8586-7385-311a41a54b0d@redhat.com> Message-ID: Ok. How about later JDKs? Could they be affected, too? Thanks, Roman > This is not a Shenandoah specific bug, I believe G1 suffers the same > problem. The reason to fix Shenandoah first, in hope that we can have a > build for our customer to test this fix, as we don't have a reproducer. > I can move the fix to jdk8u if you think otherwise. > > Apparently, MetadataOnStackMark::record() was never designed for MT, but > G1 and Shenandoah's parallel cleaning code can call > nmethod::do_unloading_parallel() from different worker threads, and > nmethod::do_unloading_parallel() in turn calls > nmethod::clean_ic_if_metadata_is_dead() which may call > Metadata::mark_on_stack() to record metadata to MetadataOnStackMark(). > > Calling MetadataOnStackMark::record() without synchronization, may > corrupt internal buffer, which can result fatal error in > MetadataOnStackMark's destructor, when it walks the buffers. > > > This problem is hard to reproduce, cause it is only possible when there > are redefined classes. > > > Webrev: > http://cr.openjdk.java.net/~zgu/shenandoah/metadata_on_stack/webrev.00/ > > Thanks, > > -Zhengyu > From zgu at redhat.com Thu Jul 29 14:23:43 2021 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 29 Jul 2021 10:23:43 -0400 Subject: [sh/jdk8u] MetadataOnStackMark::record() is not MT-safe In-Reply-To: References: <7cd3b64f-0312-8586-7385-311a41a54b0d@redhat.com> Message-ID: <812561d8-daa5-9909-3316-aa7da5e79dc1@redhat.com> Tested 11u, does not seem affected by this bug. -Zhengyu On 7/29/21 10:22 AM, Roman Kennke wrote: > Ok. > > How about later JDKs? Could they be affected, too? > > Thanks, > Roman > >> This is not a Shenandoah specific bug, I believe G1 suffers the same >> problem. The reason to fix Shenandoah first, in hope that we can have >> a build for our customer to test this fix, as we don't have a >> reproducer. I can move the fix to jdk8u if you think otherwise. >> >> Apparently, MetadataOnStackMark::record() was never designed for MT, >> but G1 and Shenandoah's parallel cleaning code can call >> nmethod::do_unloading_parallel() from different worker threads, and >> nmethod::do_unloading_parallel() in turn calls >> nmethod::clean_ic_if_metadata_is_dead() which may call >> Metadata::mark_on_stack() to record metadata to MetadataOnStackMark(). >> >> Calling MetadataOnStackMark::record() without synchronization, may >> corrupt internal buffer, which can result fatal error in >> MetadataOnStackMark's destructor, when it walks the buffers. >> >> >> This problem is hard to reproduce, cause it is only possible when >> there are redefined classes. >> >> >> Webrev: >> http://cr.openjdk.java.net/~zgu/shenandoah/metadata_on_stack/webrev.00/ >> >> Thanks, >> >> -Zhengyu >> > From zgu at openjdk.java.net Thu Jul 29 14:25:03 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Thu, 29 Jul 2021 14:25:03 GMT Subject: RFR: Generational support for weak roots and references In-Reply-To: References: Message-ID: On Thu, 22 Jul 2021 22:53:56 GMT, William Kemper wrote: > ### Summary > The LRB for non-strong references is modified to permit resurrection of objects outside the generation being collected. In other words, resurrection is only blocked for unmarked objects in the generation being collected. > > Each `ShenandoahGeneration` has its own reference processor instance. In some cases, a reference from the old generation may end up on the young generation discovered list if the reference points to a young referent (this would happen if the old reference is in the remembered set). However, young references that point to referents in the old generation are _not_ discovered. This has the effect of strongly marking the old generation referent. This also avoids the case of having young references on the old generation discovered list being evacuated/relocated while they wait for old generation reference processing (although we believe this case would be handled correctly by the existing update references code). Would it be simpler to encapsulate generational info in MarkingContext for is_marked_xxx tests? e.g. for young marking context, everything in old gen returns is_marked_strong = true? ------------- PR: https://git.openjdk.java.net/shenandoah/pull/53 From rkennke at redhat.com Thu Jul 29 14:38:23 2021 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Jul 2021 16:38:23 +0200 Subject: [sh/jdk8u] MetadataOnStackMark::record() is not MT-safe In-Reply-To: <812561d8-daa5-9909-3316-aa7da5e79dc1@redhat.com> References: <7cd3b64f-0312-8586-7385-311a41a54b0d@redhat.com> <812561d8-daa5-9909-3316-aa7da5e79dc1@redhat.com> Message-ID: <8de26ad2-1838-5407-4c43-de98121cf0c4@redhat.com> > Tested 11u, does not seem affected by this bug. Do you have an idea why? Maybe there is a change in 11 that fixes or prevents it (and could be backported)? Roman > -Zhengyu > > On 7/29/21 10:22 AM, Roman Kennke wrote: >> Ok. >> >> How about later JDKs? Could they be affected, too? >> >> Thanks, >> Roman >> >>> This is not a Shenandoah specific bug, I believe G1 suffers the same >>> problem. The reason to fix Shenandoah first, in hope that we can have >>> a build for our customer to test this fix, as we don't have a >>> reproducer. I can move the fix to jdk8u if you think otherwise. >>> >>> Apparently, MetadataOnStackMark::record() was never designed for MT, >>> but G1 and Shenandoah's parallel cleaning code can call >>> nmethod::do_unloading_parallel() from different worker threads, and >>> nmethod::do_unloading_parallel() in turn calls >>> nmethod::clean_ic_if_metadata_is_dead() which may call >>> Metadata::mark_on_stack() to record metadata to MetadataOnStackMark(). >>> >>> Calling MetadataOnStackMark::record() without synchronization, may >>> corrupt internal buffer, which can result fatal error in >>> MetadataOnStackMark's destructor, when it walks the buffers. >>> >>> >>> This problem is hard to reproduce, cause it is only possible when >>> there are redefined classes. >>> >>> >>> Webrev: >>> http://cr.openjdk.java.net/~zgu/shenandoah/metadata_on_stack/webrev.00/ >>> >>> Thanks, >>> >>> -Zhengyu >>> >> > From zgu at redhat.com Thu Jul 29 14:44:51 2021 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 29 Jul 2021 10:44:51 -0400 Subject: [sh/jdk8u] MetadataOnStackMark::record() is not MT-safe In-Reply-To: <8de26ad2-1838-5407-4c43-de98121cf0c4@redhat.com> References: <7cd3b64f-0312-8586-7385-311a41a54b0d@redhat.com> <812561d8-daa5-9909-3316-aa7da5e79dc1@redhat.com> <8de26ad2-1838-5407-4c43-de98121cf0c4@redhat.com> Message-ID: <0ce8d801-2328-c24e-b1c5-3d8fe42bf9dd@redhat.com> 11u already encapsulated metadata in nmethod (see nmethod::do_unloading_oops(), which is quite different from 8u), so on stack metadata (when GC scans thread roots) keeps nmethod alive. -Zhengyu On 7/29/21 10:38 AM, Roman Kennke wrote: >> Tested 11u, does not seem affected by this bug. > > Do you have an idea why? Maybe there is a change in 11 that fixes or > prevents it (and could be backported)? > > Roman > >> -Zhengyu >> >> On 7/29/21 10:22 AM, Roman Kennke wrote: >>> Ok. >>> >>> How about later JDKs? Could they be affected, too? >>> >>> Thanks, >>> Roman >>> >>>> This is not a Shenandoah specific bug, I believe G1 suffers the same >>>> problem. The reason to fix Shenandoah first, in hope that we can >>>> have a build for our customer to test this fix, as we don't have a >>>> reproducer. I can move the fix to jdk8u if you think otherwise. >>>> >>>> Apparently, MetadataOnStackMark::record() was never designed for MT, >>>> but G1 and Shenandoah's parallel cleaning code can call >>>> nmethod::do_unloading_parallel() from different worker threads, and >>>> nmethod::do_unloading_parallel() in turn calls >>>> nmethod::clean_ic_if_metadata_is_dead() which may call >>>> Metadata::mark_on_stack() to record metadata to MetadataOnStackMark(). >>>> >>>> Calling MetadataOnStackMark::record() without synchronization, may >>>> corrupt internal buffer, which can result fatal error in >>>> MetadataOnStackMark's destructor, when it walks the buffers. >>>> >>>> >>>> This problem is hard to reproduce, cause it is only possible when >>>> there are redefined classes. >>>> >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~zgu/shenandoah/metadata_on_stack/webrev.00/ >>>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>> >> > From rkennke at redhat.com Thu Jul 29 16:10:23 2021 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Jul 2021 18:10:23 +0200 Subject: [sh/jdk8u] MetadataOnStackMark::record() is not MT-safe In-Reply-To: <0ce8d801-2328-c24e-b1c5-3d8fe42bf9dd@redhat.com> References: <7cd3b64f-0312-8586-7385-311a41a54b0d@redhat.com> <812561d8-daa5-9909-3316-aa7da5e79dc1@redhat.com> <8de26ad2-1838-5407-4c43-de98121cf0c4@redhat.com> <0ce8d801-2328-c24e-b1c5-3d8fe42bf9dd@redhat.com> Message-ID: <3761e05c-0472-d585-759d-31223bc2fd8d@redhat.com> Ok then. Go! Thanks, Roman > 11u already encapsulated metadata in nmethod (see > nmethod::do_unloading_oops(), which is quite different from 8u), so on > stack metadata (when GC scans thread roots) keeps nmethod alive. > > -Zhengyu > > > > On 7/29/21 10:38 AM, Roman Kennke wrote: >>> Tested 11u, does not seem affected by this bug. >> >> Do you have an idea why? Maybe there is a change in 11 that fixes or >> prevents it (and could be backported)? >> >> Roman >> >>> -Zhengyu >>> >>> On 7/29/21 10:22 AM, Roman Kennke wrote: >>>> Ok. >>>> >>>> How about later JDKs? Could they be affected, too? >>>> >>>> Thanks, >>>> Roman >>>> >>>>> This is not a Shenandoah specific bug, I believe G1 suffers the >>>>> same problem. The reason to fix Shenandoah first, in hope that we >>>>> can have a build for our customer to test this fix, as we don't >>>>> have a reproducer. I can move the fix to jdk8u if you think otherwise. >>>>> >>>>> Apparently, MetadataOnStackMark::record() was never designed for >>>>> MT, but G1 and Shenandoah's parallel cleaning code can call >>>>> nmethod::do_unloading_parallel() from different worker threads, and >>>>> nmethod::do_unloading_parallel() in turn calls >>>>> nmethod::clean_ic_if_metadata_is_dead() which may call >>>>> Metadata::mark_on_stack() to record metadata to MetadataOnStackMark(). >>>>> >>>>> Calling MetadataOnStackMark::record() without synchronization, may >>>>> corrupt internal buffer, which can result fatal error in >>>>> MetadataOnStackMark's destructor, when it walks the buffers. >>>>> >>>>> >>>>> This problem is hard to reproduce, cause it is only possible when >>>>> there are redefined classes. >>>>> >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~zgu/shenandoah/metadata_on_stack/webrev.00/ >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> -Zhengyu >>>>> >>>> >>> >> > From zgu at redhat.com Thu Jul 29 16:40:57 2021 From: zgu at redhat.com (zgu at redhat.com) Date: Thu, 29 Jul 2021 16:40:57 +0000 Subject: hg: shenandoah/jdk8/hotspot: MetadataOnStackMark::record() is not MT-safe Message-ID: <202107291640.16TGewak019786@aojmv0008.oracle.com> Changeset: b261d2db2ea2 Author: zgu Date: 2021-07-29 12:40 -0400 URL: https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/b261d2db2ea2 MetadataOnStackMark::record() is not MT-safe Reviewed-by: rkennke ! src/share/vm/classfile/metadataOnStackMark.cpp ! src/share/vm/runtime/mutexLocker.cpp ! src/share/vm/runtime/mutexLocker.hpp From wkemper at openjdk.java.net Thu Jul 29 18:50:13 2021 From: wkemper at openjdk.java.net (William Kemper) Date: Thu, 29 Jul 2021 18:50:13 GMT Subject: RFR: Generational support for weak roots and references In-Reply-To: References: Message-ID: On Thu, 22 Jul 2021 22:53:56 GMT, William Kemper wrote: > ### Summary > The LRB for non-strong references is modified to permit resurrection of objects outside the generation being collected. In other words, resurrection is only blocked for unmarked objects in the generation being collected. > > Each `ShenandoahGeneration` has its own reference processor instance. In some cases, a reference from the old generation may end up on the young generation discovered list if the reference points to a young referent (this would happen if the old reference is in the remembered set). However, young references that point to referents in the old generation are _not_ discovered. This has the effect of strongly marking the old generation referent. This also avoids the case of having young references on the old generation discovered list being evacuated/relocated while they wait for old generation reference processing (although we believe this case would be handled correctly by the existing update references code). There are `is_marked_or_old` methods in `MarkingContext` for use with the verifier (because it visits all objects, even when it's run during a young collect). I think so long as we want to process weak references in young collections _and_ old collections we need to maintain two discovered lists. I can see a way through this where we process references in young and _global_ collections (i.e., entire heap) using one discovered list (since young and global collections cannot overlap). ------------- PR: https://git.openjdk.java.net/shenandoah/pull/53 From iignatyev at openjdk.java.net Fri Jul 30 15:36:35 2021 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Fri, 30 Jul 2021 15:36:35 GMT Subject: [jdk17] RFR: 8067223: [TESTBUG] Rename Whitebox API package In-Reply-To: <_3viZFs2iZ00hMdeP3Nq9gTmwfIHeTzZ7NVT8thQ1BM=.10005882-f135-4592-a663-b75747da3f6c@github.com> References: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> <_3viZFs2iZ00hMdeP3Nq9gTmwfIHeTzZ7NVT8thQ1BM=.10005882-f135-4592-a663-b75747da3f6c@github.com> Message-ID: On Thu, 29 Jul 2021 01:30:37 GMT, Vladimir Kozlov wrote: >> Hi all, >> >> could you please review this big tedious and trivial(-ish) patch which moves `sun.hotspot.WhiteBox` and related classes to `jdk.test.whitebox` package? >> >> the majority of the patch is the following substitutions: >> - `s~sun/hotspot/WhiteBox~jdk/test/whitebox/WhiteBox~g` >> - `s/sun.hotspot.parser/jdk.test.whitebox.parser/g` >> - `s/sun.hotspot.cpuinfo/jdk.test.whitebox.cpuinfo/g` >> - `s/sun.hotspot.code/jdk.test.whitebox.code/g` >> - `s/sun.hotspot.gc/jdk.test.whitebox.gc/g` >> - `s/sun.hotspot.WhiteBox/jdk.test.whitebox.WhiteBox/g` >> >> testing: tier1-4 >> >> Thanks, >> -- Igor > > I know that tests fixes could be pushed during RDP2 without approval. > But these one is very big and it is not a fix - it is enhancement. > > What is the reason you want to push it into JDK 17 just few days before first Release Candidate? Instead of pushing it into JDK 18. > > And I can't even review it because GutHub UI hangs on these big changes. @vnkozlov, @dholmes-ora, Thank you for looking at this! I want this to be integrated into JDK 17 b/c some "external" libraries use (used to use) WhiteBox API, e.g. jcstress[[2]] used WhiteBox API to deoptimize compiled methods[[3]], and it would be easier for maintainers of such libraries to condition package name based on JDK major version. Also, given JDK 17 is an LTS, it would be beneficial for everyone not to have big differences in test bases b/w it and the mainline. according to JEP 3, test RFEs are allowed until the very end and don't require late enhancement approval: "Enhancements to tests and documentation during RDP 1 and RDP 2 do not require approval, as long as the relevant issues are identified with a noreg-self or noreg-doc label, as appropriate"[[1]]. So, process-wise, I don't see any issues w/ integrating this RFE, yet, I fully agree that due to its size, this patch can be disruptive and can cause massive failures, which is something we obviously don't want at the current stage of JDK 17. I like David's idea about phasing this clean-up, and, due to the reasons described above, I would like to integrate the first part, copying WhiteBox classes to the new package structure and associated changes w/o updating all the tests, into JDK 17 and update the tests on the mainline (w/ backporting into jdk17u). WDYT? Cheers, -- Igor [1]: https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process [2]: https://github.com/openjdk/jcstress [3]: https://github.com/openjdk/jcstress/blob/df83b446f187ae0b0fa31fa54decb59db9e955da/jcstress-core/src/main/java/org/openjdk/jcstress/vm/WhiteBoxSupport.java ------------- PR: https://git.openjdk.java.net/jdk17/pull/290 From iignatyev at openjdk.java.net Sat Jul 31 20:42:10 2021 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Sat, 31 Jul 2021 20:42:10 GMT Subject: [jdk17] RFR: 8067223: [TESTBUG] Rename Whitebox API package [v2] In-Reply-To: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> References: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> Message-ID: > Hi all, > > could you please review this big tedious and trivial(-ish) patch which moves `sun.hotspot.WhiteBox` and related classes to `jdk.test.whitebox` package? > > the majority of the patch is the following substitutions: > - `s~sun/hotspot/WhiteBox~jdk/test/whitebox/WhiteBox~g` > - `s/sun.hotspot.parser/jdk.test.whitebox.parser/g` > - `s/sun.hotspot.cpuinfo/jdk.test.whitebox.cpuinfo/g` > - `s/sun.hotspot.code/jdk.test.whitebox.code/g` > - `s/sun.hotspot.gc/jdk.test.whitebox.gc/g` > - `s/sun.hotspot.WhiteBox/jdk.test.whitebox.WhiteBox/g` > > testing: tier1-4 > > Thanks, > -- Igor Igor Ignatyev has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains 12 new commits since the last revision: - fixed ctw build - updated runtime/cds/appcds/JarBuilder to copy j.t.w.WhiteBox's inner class - updated requires.VMProps - updated TEST.ROOT - adjusted hotspot source - added test - moved and adjusted WhiteBox tests (test/lib-test/sun/hotspot/whitebox) - updated ClassFileInstaller to copy j.t.w.WhiteBox's inner class - removed sun/hotspot/parser/DiagnosticCommand - deprecated sun/hotspot classes disallowed s.h.WhiteBox w/ security manager - ... and 2 more: https://git.openjdk.java.net/jdk17/compare/8f12f2cf...237e8860 ------------- Changes: - all: https://git.openjdk.java.net/jdk17/pull/290/files - new: https://git.openjdk.java.net/jdk17/pull/290/files/8f12f2cf..237e8860 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk17&pr=290&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk17&pr=290&range=00-01 Stats: 3248 lines in 939 files changed: 969 ins; 0 del; 2279 mod Patch: https://git.openjdk.java.net/jdk17/pull/290.diff Fetch: git fetch https://git.openjdk.java.net/jdk17 pull/290/head:pull/290 PR: https://git.openjdk.java.net/jdk17/pull/290 From iignatyev at openjdk.java.net Sat Jul 31 20:51:46 2021 From: iignatyev at openjdk.java.net (Igor Ignatyev) Date: Sat, 31 Jul 2021 20:51:46 GMT Subject: [jdk17] RFR: 8067223: [TESTBUG] Rename Whitebox API package [v2] In-Reply-To: References: <39PNb3wa-43yB5IFR3XroqFf4vMNZPitzF85lO1Gw58=.f2587741-d2c4-467d-b722-b17269587e7a@github.com> Message-ID: On Sat, 31 Jul 2021 20:42:10 GMT, Igor Ignatyev wrote: >> Hi all, >> >> could you please review this big tedious and trivial(-ish) patch which moves `sun.hotspot.WhiteBox` and related classes to `jdk.test.whitebox` package? >> >> the majority of the patch is the following substitutions: >> - `s~sun/hotspot/WhiteBox~jdk/test/whitebox/WhiteBox~g` >> - `s/sun.hotspot.parser/jdk.test.whitebox.parser/g` >> - `s/sun.hotspot.cpuinfo/jdk.test.whitebox.cpuinfo/g` >> - `s/sun.hotspot.code/jdk.test.whitebox.code/g` >> - `s/sun.hotspot.gc/jdk.test.whitebox.gc/g` >> - `s/sun.hotspot.WhiteBox/jdk.test.whitebox.WhiteBox/g` >> >> testing: tier1-4 >> >> Thanks, >> -- Igor > > Igor Ignatyev has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. Vladimir, David, I've (forced) pushed a smaller version of the renaming. instead of removing `sun.hotspot` classes, it copies them to `jdk.test.whitebox` (w/ `s.h.parser.DiagnosticCommand` being removed as it's used in WhiteBox signature and it was easier to update a few tests that use it), updates hotspot code to register native methods for both `sun.hotspot.WhiteBox` and `jdk.test.whitebox.WhiteBox` classes. To make it easier and not to introduce extra dependency, I've made it impossible to use `s.h.WB` w/ a security manager enabled, otherwise there would be a dependency b/w `s.h.WB` and `j.t.w.WB$WhiteBoxPermission` or there would be 2 permissions. There are no open JDK tests that are impacted by this limitation. With minor tweaks in closed source, the patch successfully passes Oracle's tier1-4. -- Igor ------------- PR: https://git.openjdk.java.net/jdk17/pull/290