From dean.long at oracle.com Thu Nov 1 00:13:09 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 31 Oct 2018 17:13:09 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <6523ec93-af32-da8f-704a-0322e56de00f@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <6523ec93-af32-da8f-704a-0322e56de00f@oracle.com> Message-ID: <9a5a08bf-331d-8def-9f2d-98dd47fcf7ef@oracle.com> On 10/31/18 4:06 PM, David Holmes wrote: > Hi Dean, > > Looking only at the hotspot changes. The removal of the DoPrivileged > and related privileged_stack code seems okay. I have a few related > comments: > > src/hotspot/share/classfile/systemDictionary.hpp > > You added the java_security_AccessController class after > java_security_AccessControlContext. Did you actually verify where in > the load/initialization order the AccessController class appears > today, and where it appears after your change? (Note the comments at > the start of WK_KLASSES_DO). Changes to the initialization order would > be my biggest concern with this patch. > No, I did not notice that comment and assumed alphabetical order was OK here.? However, these classes appear to be only resolved, not initialized (and AccessController does not have a static initializer), so could you explain how the order in this list can affect initialization order? I only need this in JVM_GetStackAccessControlContext, which is a static JNI method, so I could get the klass* from the incoming jclass instead of using a well-known class entry. > --- > > I'm unclear about the change to the test: > > test/hotspot/jtreg/runtime/JVMDoPrivileged/DoPrivRunAbstract.jasm > > as it still refers to the now non-existent JVM_DoPrivileged in the > summary. Does this test still make sense? Should it be moved to the > Java side now it doesn't actually test anything in the VM? > I think these tests still make sense, unless we have similar coverage somewhere else.? How about if I fix the reference to JVM_DoPrivileged for now and file a separate bug about moving or removing these tests? > --- > > There may be further dead code to remove now: > > vmSymbols.hpp: codesource_permissioncollection_signature is now > unreferenced and can be removed. > > javaClasses.*: > ? - java_lang_System::has_security_manager > ? - java_security_AccessControlContext::is_authorized > > ./share/memory/universe.hpp:? static Method* > protection_domain_implies_method() > Good catches!? I have uploaded a new webrev: http://cr.openjdk.java.net/~dlong/8212605/webrev.2/ http://cr.openjdk.java.net/~dlong/8212605/webrev.2.incr/ (incremental diff) dl > --- > > Thanks, > David > > > On 1/11/2018 8:23 AM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8212605 >> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >> >> This change implements AccessController.doPrivileged in Java. This >> gives a performance improvement while also being useful to Project >> Loom by removing the Java --> native --> Java transition.? One reason >> doPrivileged has historically been in native is because of the need >> to guarantee the cleanup of the privileged context when doPrivileged >> returns.? To do that in Java, the information that >> AccessController.getContext needs is pushed onto the Java stack as >> arguments to a method that getContext will recognize during its stack >> walk.? This allows us to remove the native privileged stack while >> guaranteeing that the privileged context goes away when the method >> returns. >> >> Tested with tier1-tier3 hotspot and jdk tests and JCK >> api/java_security tests.? For the first few rounds of testing, I kept >> the old native privileged stack and compared the results of the old >> and new implementations for each getContext call, which did catch >> some early bugs.? The changes were also examined by internal security >> experts and run through additional internal security tests. >> >> The improvement on this [1] doPrivileged microbenchmark is >> approximate 50x. >> >> There is no attempt to optimize getContext() or security permission >> checks in this change, however, this is intended to be a first step >> towards other possible improvements, for example those proposed here >> [2]. >> >> dl >> >> [1] >> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >> >> [2] >> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >> >> From igor.ignatyev at oracle.com Thu Nov 1 00:25:00 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 31 Oct 2018 17:25:00 -0700 Subject: RFR(XS) : 8177711 : Convert TestVirtualSpaceNode_test to GTest In-Reply-To: <3396B7BC-C085-497B-BF1D-A042750837CB@oracle.com> References: <3396B7BC-C085-497B-BF1D-A042750837CB@oracle.com> Message-ID: <3C0FC249-73D1-4E8D-860A-9C79772A6B07@oracle.com> a friendly reminder, could someone Review this? -- Igor > On Oct 24, 2018, at 11:50 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8177711/webrev.00/index.html >> 405 lines changed: 244 ins; 159 del; 2 mod; > > > Hi all, > > could you please review this small and straightforward patch which converts TestVirtualSpaceNode_test into GTest? TestVirtualSpaceNode_test have been removed from executed by 8198423[1] and some of its asserts aren't true any more. I have them commented them out, and filed 8212932[2] to clean them up. > > webrev: http://cr.openjdk.java.net/~iignatyev//8177711/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8177711 > > Thanks, > -- Igor > > [1] https://bugs.openjdk.java.net/browse/JDK-8198423 > [2] https://bugs.openjdk.java.net/browse/JDK-8212932 > From igor.ignatyev at oracle.com Thu Nov 1 00:25:40 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 31 Oct 2018 17:25:40 -0700 Subject: RFR(XS) : 8177708 : Convert TestReserveMemorySpecial_test to Gtest In-Reply-To: References: Message-ID: <2CED36D8-FBBE-4319-85B4-CFFA87216025@oracle.com> a friendly reminder, could someone Review this? -- Igor > On Oct 22, 2018, at 11:36 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >> 331 lines changed: 330 ins; 1 del; 0 mod; > Hi all, > > could you please review this small and trivial patch which converts TestReserveMemorySpecial_test tests to Gtest? the old tests[1,2] haven't been removed as they are used by WhiteBox::runMemoryUnitTests they will be removed when the rest tests are converted. > > webrev: http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8177708 > [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/linux/os_linux.cpp#l5969 > [2] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/windows/os_windows.cpp#l5564 > > Thanks, > -- Igor From david.holmes at oracle.com Thu Nov 1 00:54:58 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 1 Nov 2018 10:54:58 +1000 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <9a5a08bf-331d-8def-9f2d-98dd47fcf7ef@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <6523ec93-af32-da8f-704a-0322e56de00f@oracle.com> <9a5a08bf-331d-8def-9f2d-98dd47fcf7ef@oracle.com> Message-ID: <4fe219c8-7e29-1bfb-839d-1e1caab34efa@oracle.com> Hi Dean, On 1/11/2018 10:13 AM, dean.long at oracle.com wrote: > On 10/31/18 4:06 PM, David Holmes wrote: >> Hi Dean, >> >> Looking only at the hotspot changes. The removal of the DoPrivileged >> and related privileged_stack code seems okay. I have a few related >> comments: >> >> src/hotspot/share/classfile/systemDictionary.hpp >> >> You added the java_security_AccessController class after >> java_security_AccessControlContext. Did you actually verify where in >> the load/initialization order the AccessController class appears >> today, and where it appears after your change? (Note the comments at >> the start of WK_KLASSES_DO). Changes to the initialization order would >> be my biggest concern with this patch. >> > > No, I did not notice that comment and assumed alphabetical order was OK > here.? However, these classes appear to be only resolved, not > initialized (and AccessController does not have a static initializer), > so could you explain how the order in this list can affect > initialization order? You're right it doesn't. There are a couple of comments that refer to "initialization" but it's not static initialization of these classes. I'm unclear how the resolution order in that list may interact with other parts of the startup sequence. > I only need this in JVM_GetStackAccessControlContext, which is a static > JNI method, so I could get the klass* from the incoming jclass instead > of using a well-known class entry. I think it's okay given we have AccessControlContext there anyway. >> --- >> >> I'm unclear about the change to the test: >> >> test/hotspot/jtreg/runtime/JVMDoPrivileged/DoPrivRunAbstract.jasm >> >> as it still refers to the now non-existent JVM_DoPrivileged in the >> summary. Does this test still make sense? Should it be moved to the >> Java side now it doesn't actually test anything in the VM? >> > > I think these tests still make sense, unless we have similar coverage > somewhere else.? How about if I fix the reference to JVM_DoPrivileged > for now and file a separate bug about moving or removing these tests? Yep that's fine. >> --- >> >> There may be further dead code to remove now: >> >> vmSymbols.hpp: codesource_permissioncollection_signature is now >> unreferenced and can be removed. >> >> javaClasses.*: >> ? - java_lang_System::has_security_manager >> ? - java_security_AccessControlContext::is_authorized >> >> ./share/memory/universe.hpp:? static Method* >> protection_domain_implies_method() >> > > Good catches!? I have uploaded a new webrev: > > http://cr.openjdk.java.net/~dlong/8212605/webrev.2/ > http://cr.openjdk.java.net/~dlong/8212605/webrev.2.incr/ (incremental diff) Updates all look fine. Thanks, David ----- > dl > >> --- >> >> Thanks, >> David >> >> >> On 1/11/2018 8:23 AM, dean.long at oracle.com wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8212605 >>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >>> >>> This change implements AccessController.doPrivileged in Java. This >>> gives a performance improvement while also being useful to Project >>> Loom by removing the Java --> native --> Java transition.? One reason >>> doPrivileged has historically been in native is because of the need >>> to guarantee the cleanup of the privileged context when doPrivileged >>> returns.? To do that in Java, the information that >>> AccessController.getContext needs is pushed onto the Java stack as >>> arguments to a method that getContext will recognize during its stack >>> walk.? This allows us to remove the native privileged stack while >>> guaranteeing that the privileged context goes away when the method >>> returns. >>> >>> Tested with tier1-tier3 hotspot and jdk tests and JCK >>> api/java_security tests.? For the first few rounds of testing, I kept >>> the old native privileged stack and compared the results of the old >>> and new implementations for each getContext call, which did catch >>> some early bugs.? The changes were also examined by internal security >>> experts and run through additional internal security tests. >>> >>> The improvement on this [1] doPrivileged microbenchmark is >>> approximate 50x. >>> >>> There is no attempt to optimize getContext() or security permission >>> checks in this change, however, this is intended to be a first step >>> towards other possible improvements, for example those proposed here >>> [2]. >>> >>> dl >>> >>> [1] >>> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >>> >>> [2] >>> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >>> >>> > From ioi.lam at oracle.com Thu Nov 1 01:01:35 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 31 Oct 2018 18:01:35 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <9a5a08bf-331d-8def-9f2d-98dd47fcf7ef@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <6523ec93-af32-da8f-704a-0322e56de00f@oracle.com> <9a5a08bf-331d-8def-9f2d-98dd47fcf7ef@oracle.com> Message-ID: <920b1841-53cf-23c1-1f35-365518935ee0@oracle.com> On 10/31/18 5:13 PM, dean.long at oracle.com wrote: > On 10/31/18 4:06 PM, David Holmes wrote: >> Hi Dean, >> >> Looking only at the hotspot changes. The removal of the DoPrivileged >> and related privileged_stack code seems okay. I have a few related >> comments: >> >> src/hotspot/share/classfile/systemDictionary.hpp >> >> You added the java_security_AccessController class after >> java_security_AccessControlContext. Did you actually verify where in >> the load/initialization order the AccessController class appears >> today, and where it appears after your change? (Note the comments at >> the start of WK_KLASSES_DO). Changes to the initialization order >> would be my biggest concern with this patch. >> > > No, I did not notice that comment and assumed alphabetical order was > OK here.? However, these classes appear to be only resolved, not > initialized (and AccessController does not have a static initializer), > so could you explain how the order in this list can affect > initialization order? > > I only need this in JVM_GetStackAccessControlContext, which is a > static JNI method, so I could get the klass* from the incoming jclass > instead of using a well-known class entry. Hi Dean, You're correct that those classes are only resolved, and not initialized. So the order shouldn't matter too much. However, the order of the first few classes is important, as the creation of Object, Serializable, Cloneable, String, etc, has to be done in a certain order. I'm not sure whether the order of the reference classes, 292 classes, and box classes are important. I'll experiment of getting rid of the separate phases after the call to Universe::fixup_mirrors(). This might be relics from old days where the classes were once indeed initialized in SystemDictionary::initialize_well_known_classes, which was the old name for SystemDictionary::resolve_well_known_classes. (-XX:+TraceBytecodes shows no Java code is executed before resolve_well_known_classes returns). I filed https://bugs.openjdk.java.net/browse/JDK-8213230 For the time being, I think as long as you put the new AccessController class near the existing class AccessControlContext, you should be OK. Thanks - Ioi > >> --- >> >> I'm unclear about the change to the test: >> >> test/hotspot/jtreg/runtime/JVMDoPrivileged/DoPrivRunAbstract.jasm >> >> as it still refers to the now non-existent JVM_DoPrivileged in the >> summary. Does this test still make sense? Should it be moved to the >> Java side now it doesn't actually test anything in the VM? >> > > I think these tests still make sense, unless we have similar coverage > somewhere else.? How about if I fix the reference to JVM_DoPrivileged > for now and file a separate bug about moving or removing these tests? > >> --- >> >> There may be further dead code to remove now: >> >> vmSymbols.hpp: codesource_permissioncollection_signature is now >> unreferenced and can be removed. >> >> javaClasses.*: >> ? - java_lang_System::has_security_manager >> ? - java_security_AccessControlContext::is_authorized >> >> ./share/memory/universe.hpp:? static Method* >> protection_domain_implies_method() >> > > Good catches!? I have uploaded a new webrev: > > http://cr.openjdk.java.net/~dlong/8212605/webrev.2/ > http://cr.openjdk.java.net/~dlong/8212605/webrev.2.incr/ (incremental > diff) > > dl > >> --- >> >> Thanks, >> David >> >> >> On 1/11/2018 8:23 AM, dean.long at oracle.com wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8212605 >>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >>> >>> This change implements AccessController.doPrivileged in Java. This >>> gives a performance improvement while also being useful to Project >>> Loom by removing the Java --> native --> Java transition.? One >>> reason doPrivileged has historically been in native is because of >>> the need to guarantee the cleanup of the privileged context when >>> doPrivileged returns.? To do that in Java, the information that >>> AccessController.getContext needs is pushed onto the Java stack as >>> arguments to a method that getContext will recognize during its >>> stack walk.? This allows us to remove the native privileged stack >>> while guaranteeing that the privileged context goes away when the >>> method returns. >>> >>> Tested with tier1-tier3 hotspot and jdk tests and JCK >>> api/java_security tests.? For the first few rounds of testing, I >>> kept the old native privileged stack and compared the results of the >>> old and new implementations for each getContext call, which did >>> catch some early bugs.? The changes were also examined by internal >>> security experts and run through additional internal security tests. >>> >>> The improvement on this [1] doPrivileged microbenchmark is >>> approximate 50x. >>> >>> There is no attempt to optimize getContext() or security permission >>> checks in this change, however, this is intended to be a first step >>> towards other possible improvements, for example those proposed here >>> [2]. >>> >>> dl >>> >>> [1] >>> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >>> >>> [2] >>> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >>> >>> > From vladimir.x.ivanov at oracle.com Thu Nov 1 01:11:11 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 31 Oct 2018 18:11:11 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> Message-ID: <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> Dean, src/java.base/share/classes/java/security/AccessController.java: + /** + * Internal marker for hidden implementation frames. + */ + /*non-public*/ + @Target(ElementType.METHOD) + @Retention(RetentionPolicy.RUNTIME) + @interface Hidden { + } You declare @Hidden, but then map it to _method_Hidden along with @Hidden from java.lang.invoke.LambdaForm. What do you think about moving LambdaForm.Hidden to jdk.internal.vm.annotation instead and share among all usages? Best regards, Vladimir Ivanov On 31/10/2018 15:23, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8212605 > http://cr.openjdk.java.net/~dlong/8212605/webrev.1 > > This change implements AccessController.doPrivileged in Java.? This > gives a performance improvement while also being useful to Project Loom > by removing the Java --> native --> Java transition.? One reason > doPrivileged has historically been in native is because of the need to > guarantee the cleanup of the privileged context when doPrivileged > returns.? To do that in Java, the information that > AccessController.getContext needs is pushed onto the Java stack as > arguments to a method that getContext will recognize during its stack > walk.? This allows us to remove the native privileged stack while > guaranteeing that the privileged context goes away when the method returns. > > Tested with tier1-tier3 hotspot and jdk tests and JCK api/java_security > tests.? For the first few rounds of testing, I kept the old native > privileged stack and compared the results of the old and new > implementations for each getContext call, which did catch some early > bugs.? The changes were also examined by internal security experts and > run through additional internal security tests. > > The improvement on this [1] doPrivileged microbenchmark is approximate 50x. > > There is no attempt to optimize getContext() or security permission > checks in this change, however, this is intended to be a first step > towards other possible improvements, for example those proposed here [2]. > > dl > > [1] > http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java > > [2] > http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html > > From jesper.wilhelmsson at oracle.com Thu Nov 1 01:24:51 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Thu, 1 Nov 2018 02:24:51 +0100 Subject: RFR(XS) : 8177711 : Convert TestVirtualSpaceNode_test to GTest In-Reply-To: <3C0FC249-73D1-4E8D-860A-9C79772A6B07@oracle.com> References: <3396B7BC-C085-497B-BF1D-A042750837CB@oracle.com> <3C0FC249-73D1-4E8D-860A-9C79772A6B07@oracle.com> Message-ID: <1DF2292B-C044-4D28-B5AD-703BE94E6DF5@oracle.com> Looks good! /Jesper > On 1 Nov 2018, at 01:25, Igor Ignatyev wrote: > > a friendly reminder, could someone Review this? > -- Igor > >> On Oct 24, 2018, at 11:50 AM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8177711/webrev.00/index.html >>> 405 lines changed: 244 ins; 159 del; 2 mod; >> >> >> Hi all, >> >> could you please review this small and straightforward patch which converts TestVirtualSpaceNode_test into GTest? TestVirtualSpaceNode_test have been removed from executed by 8198423[1] and some of its asserts aren't true any more. I have them commented them out, and filed 8212932[2] to clean them up. >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8177711/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8177711 >> >> Thanks, >> -- Igor >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8198423 >> [2] https://bugs.openjdk.java.net/browse/JDK-8212932 >> > From dean.long at oracle.com Thu Nov 1 03:57:54 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 31 Oct 2018 20:57:54 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <4fe219c8-7e29-1bfb-839d-1e1caab34efa@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <6523ec93-af32-da8f-704a-0322e56de00f@oracle.com> <9a5a08bf-331d-8def-9f2d-98dd47fcf7ef@oracle.com> <4fe219c8-7e29-1bfb-839d-1e1caab34efa@oracle.com> Message-ID: Thanks David. dl On 10/31/18 5:54 PM, David Holmes wrote: > Hi Dean, > > On 1/11/2018 10:13 AM, dean.long at oracle.com wrote: >> On 10/31/18 4:06 PM, David Holmes wrote: >>> Hi Dean, >>> >>> Looking only at the hotspot changes. The removal of the DoPrivileged >>> and related privileged_stack code seems okay. I have a few related >>> comments: >>> >>> src/hotspot/share/classfile/systemDictionary.hpp >>> >>> You added the java_security_AccessController class after >>> java_security_AccessControlContext. Did you actually verify where in >>> the load/initialization order the AccessController class appears >>> today, and where it appears after your change? (Note the comments at >>> the start of WK_KLASSES_DO). Changes to the initialization order >>> would be my biggest concern with this patch. >>> >> >> No, I did not notice that comment and assumed alphabetical order was >> OK here.? However, these classes appear to be only resolved, not >> initialized (and AccessController does not have a static >> initializer), so could you explain how the order in this list can >> affect initialization order? > > You're right it doesn't. There are a couple of comments that refer to > "initialization" but it's not static initialization of these classes. > > I'm unclear how the resolution order in that list may interact with > other parts of the startup sequence. > >> I only need this in JVM_GetStackAccessControlContext, which is a >> static JNI method, so I could get the klass* from the incoming jclass >> instead of using a well-known class entry. > > I think it's okay given we have AccessControlContext there anyway. > >>> --- >>> >>> I'm unclear about the change to the test: >>> >>> test/hotspot/jtreg/runtime/JVMDoPrivileged/DoPrivRunAbstract.jasm >>> >>> as it still refers to the now non-existent JVM_DoPrivileged in the >>> summary. Does this test still make sense? Should it be moved to the >>> Java side now it doesn't actually test anything in the VM? >>> >> >> I think these tests still make sense, unless we have similar coverage >> somewhere else.? How about if I fix the reference to JVM_DoPrivileged >> for now and file a separate bug about moving or removing these tests? > > Yep that's fine. > >>> --- >>> >>> There may be further dead code to remove now: >>> >>> vmSymbols.hpp: codesource_permissioncollection_signature is now >>> unreferenced and can be removed. >>> >>> javaClasses.*: >>> ? - java_lang_System::has_security_manager >>> ? - java_security_AccessControlContext::is_authorized >>> >>> ./share/memory/universe.hpp:? static Method* >>> protection_domain_implies_method() >>> >> >> Good catches!? I have uploaded a new webrev: >> >> http://cr.openjdk.java.net/~dlong/8212605/webrev.2/ >> http://cr.openjdk.java.net/~dlong/8212605/webrev.2.incr/ (incremental >> diff) > > Updates all look fine. > > Thanks, > David > ----- > >> dl >> >>> --- >>> >>> Thanks, >>> David >>> >>> >>> On 1/11/2018 8:23 AM, dean.long at oracle.com wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8212605 >>>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >>>> >>>> This change implements AccessController.doPrivileged in Java. This >>>> gives a performance improvement while also being useful to Project >>>> Loom by removing the Java --> native --> Java transition.? One >>>> reason doPrivileged has historically been in native is because of >>>> the need to guarantee the cleanup of the privileged context when >>>> doPrivileged returns.? To do that in Java, the information that >>>> AccessController.getContext needs is pushed onto the Java stack as >>>> arguments to a method that getContext will recognize during its >>>> stack walk.? This allows us to remove the native privileged stack >>>> while guaranteeing that the privileged context goes away when the >>>> method returns. >>>> >>>> Tested with tier1-tier3 hotspot and jdk tests and JCK >>>> api/java_security tests.? For the first few rounds of testing, I >>>> kept the old native privileged stack and compared the results of >>>> the old and new implementations for each getContext call, which did >>>> catch some early bugs.? The changes were also examined by internal >>>> security experts and run through additional internal security tests. >>>> >>>> The improvement on this [1] doPrivileged microbenchmark is >>>> approximate 50x. >>>> >>>> There is no attempt to optimize getContext() or security permission >>>> checks in this change, however, this is intended to be a first step >>>> towards other possible improvements, for example those proposed >>>> here [2]. >>>> >>>> dl >>>> >>>> [1] >>>> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >>>> >>>> [2] >>>> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >>>> >>>> >> From dean.long at oracle.com Thu Nov 1 03:59:12 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 31 Oct 2018 20:59:12 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <920b1841-53cf-23c1-1f35-365518935ee0@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <6523ec93-af32-da8f-704a-0322e56de00f@oracle.com> <9a5a08bf-331d-8def-9f2d-98dd47fcf7ef@oracle.com> <920b1841-53cf-23c1-1f35-365518935ee0@oracle.com> Message-ID: Thanks Ioi. dl On 10/31/18 6:01 PM, Ioi Lam wrote: > > > On 10/31/18 5:13 PM, dean.long at oracle.com wrote: >> On 10/31/18 4:06 PM, David Holmes wrote: >>> Hi Dean, >>> >>> Looking only at the hotspot changes. The removal of the DoPrivileged >>> and related privileged_stack code seems okay. I have a few related >>> comments: >>> >>> src/hotspot/share/classfile/systemDictionary.hpp >>> >>> You added the java_security_AccessController class after >>> java_security_AccessControlContext. Did you actually verify where in >>> the load/initialization order the AccessController class appears >>> today, and where it appears after your change? (Note the comments at >>> the start of WK_KLASSES_DO). Changes to the initialization order >>> would be my biggest concern with this patch. >>> >> >> No, I did not notice that comment and assumed alphabetical order was >> OK here.? However, these classes appear to be only resolved, not >> initialized (and AccessController does not have a static >> initializer), so could you explain how the order in this list can >> affect initialization order? >> >> I only need this in JVM_GetStackAccessControlContext, which is a >> static JNI method, so I could get the klass* from the incoming jclass >> instead of using a well-known class entry. > > Hi Dean, > > You're correct that those classes are only resolved, and not > initialized. So the order shouldn't matter too much. However, the > order of the first few classes is important, as the creation of > Object, Serializable, Cloneable, String, etc, has to be done in a > certain order. > > I'm not sure whether the order of the reference classes, 292 classes, > and box classes are important. I'll experiment of getting rid of the > separate phases after the call to Universe::fixup_mirrors(). This > might be relics from old days where the classes were once indeed > initialized in SystemDictionary::initialize_well_known_classes, which > was the old name for SystemDictionary::resolve_well_known_classes. > > (-XX:+TraceBytecodes shows no Java code is executed before > resolve_well_known_classes returns). > > I filed https://bugs.openjdk.java.net/browse/JDK-8213230 > > For the time being, I think as long as you put the new > AccessController class near the existing class AccessControlContext, > you should be OK. > > Thanks > - Ioi >> >>> --- >>> >>> I'm unclear about the change to the test: >>> >>> test/hotspot/jtreg/runtime/JVMDoPrivileged/DoPrivRunAbstract.jasm >>> >>> as it still refers to the now non-existent JVM_DoPrivileged in the >>> summary. Does this test still make sense? Should it be moved to the >>> Java side now it doesn't actually test anything in the VM? >>> >> >> I think these tests still make sense, unless we have similar coverage >> somewhere else.? How about if I fix the reference to JVM_DoPrivileged >> for now and file a separate bug about moving or removing these tests? >> >>> --- >>> >>> There may be further dead code to remove now: >>> >>> vmSymbols.hpp: codesource_permissioncollection_signature is now >>> unreferenced and can be removed. >>> >>> javaClasses.*: >>> ? - java_lang_System::has_security_manager >>> ? - java_security_AccessControlContext::is_authorized >>> >>> ./share/memory/universe.hpp:? static Method* >>> protection_domain_implies_method() >>> >> >> Good catches!? I have uploaded a new webrev: >> >> http://cr.openjdk.java.net/~dlong/8212605/webrev.2/ >> http://cr.openjdk.java.net/~dlong/8212605/webrev.2.incr/ (incremental >> diff) >> >> dl >> >>> --- >>> >>> Thanks, >>> David >>> >>> >>> On 1/11/2018 8:23 AM, dean.long at oracle.com wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8212605 >>>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >>>> >>>> This change implements AccessController.doPrivileged in Java. This >>>> gives a performance improvement while also being useful to Project >>>> Loom by removing the Java --> native --> Java transition.? One >>>> reason doPrivileged has historically been in native is because of >>>> the need to guarantee the cleanup of the privileged context when >>>> doPrivileged returns.? To do that in Java, the information that >>>> AccessController.getContext needs is pushed onto the Java stack as >>>> arguments to a method that getContext will recognize during its >>>> stack walk.? This allows us to remove the native privileged stack >>>> while guaranteeing that the privileged context goes away when the >>>> method returns. >>>> >>>> Tested with tier1-tier3 hotspot and jdk tests and JCK >>>> api/java_security tests.? For the first few rounds of testing, I >>>> kept the old native privileged stack and compared the results of >>>> the old and new implementations for each getContext call, which did >>>> catch some early bugs.? The changes were also examined by internal >>>> security experts and run through additional internal security tests. >>>> >>>> The improvement on this [1] doPrivileged microbenchmark is >>>> approximate 50x. >>>> >>>> There is no attempt to optimize getContext() or security permission >>>> checks in this change, however, this is intended to be a first step >>>> towards other possible improvements, for example those proposed >>>> here [2]. >>>> >>>> dl >>>> >>>> [1] >>>> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >>>> >>>> [2] >>>> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >>>> >>>> >> > From dean.long at oracle.com Thu Nov 1 04:06:22 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 31 Oct 2018 21:06:22 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> Message-ID: <363cd5f9-e143-79cc-9698-6c8cb213ef1e@oracle.com> I think it's a good idea, but I believe it would require a CSR request.? Do you mind if I file a separate issue for jdk.internal.vm.annotation.Hidden? dl On 10/31/18 6:11 PM, Vladimir Ivanov wrote: > Dean, > > src/java.base/share/classes/java/security/AccessController.java: > +??? /** > +???? * Internal marker for hidden implementation frames. > +???? */ > +??? /*non-public*/ > +??? @Target(ElementType.METHOD) > +??? @Retention(RetentionPolicy.RUNTIME) > +??? @interface Hidden { > +??? } > > You declare @Hidden, but then map it to _method_Hidden along with > @Hidden from java.lang.invoke.LambdaForm. > > What do you think about moving LambdaForm.Hidden to > jdk.internal.vm.annotation instead and share among all usages? > > Best regards, > Vladimir Ivanov > > On 31/10/2018 15:23, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8212605 >> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >> >> This change implements AccessController.doPrivileged in Java. This >> gives a performance improvement while also being useful to Project >> Loom by removing the Java --> native --> Java transition.? One reason >> doPrivileged has historically been in native is because of the need >> to guarantee the cleanup of the privileged context when doPrivileged >> returns.? To do that in Java, the information that >> AccessController.getContext needs is pushed onto the Java stack as >> arguments to a method that getContext will recognize during its stack >> walk.? This allows us to remove the native privileged stack while >> guaranteeing that the privileged context goes away when the method >> returns. >> >> Tested with tier1-tier3 hotspot and jdk tests and JCK >> api/java_security tests.? For the first few rounds of testing, I kept >> the old native privileged stack and compared the results of the old >> and new implementations for each getContext call, which did catch >> some early bugs.? The changes were also examined by internal security >> experts and run through additional internal security tests. >> >> The improvement on this [1] doPrivileged microbenchmark is >> approximate 50x. >> >> There is no attempt to optimize getContext() or security permission >> checks in this change, however, this is intended to be a first step >> towards other possible improvements, for example those proposed here >> [2]. >> >> dl >> >> [1] >> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >> >> [2] >> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >> >> From dean.long at oracle.com Thu Nov 1 05:29:24 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 31 Oct 2018 22:29:24 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> Message-ID: <61f7523d-3fce-023b-f806-aa49a02f6d9c@oracle.com> Hi Bernd, On 10/31/18 9:39 PM, Bernd Eckenfels wrote: > http://cr.openjdk.java.net/~dlong/8212605/webrev.1/src/java.base/share/classes/java/security/AccessController.java.udiff.html > > In checkContext should the security manager be null checked first instead of last to optimize for the typical case? (If the side effects in that expression are desired it should be documented) I was following the example of createWrapper.? The side-effects of getInnocuousAcc() will only happen once, so the order shouldn't matter here, except for performance reasons.? I don't have a strong opinion about the order, but it looks like the typical case for createWrapper would also be the typical case for checkContext, so maybe they both should be changed, if indeed a null security manager is the more typical case. > I find the tail call optimization comment in wrapException adds only confusion to an otherwise clear helper. But maybe it?s just me who does not understand it. OK, I'm happy to remove it if it's confusing or not helpful.? The reason for the other @ForceInline and @ReservedStackAccess annotations is to reduce that chance of getting a StackOverflowError, and tail call elimination would also help with that.? Let's see if anyone else finds the comment helpful or confusing. dl > Gruss > Bernd > -- > http://bernd.eckenfels.net > > ________________________________ > Von: core-libs-dev im Auftrag von Vladimir Ivanov > Gesendet: Donnerstag, November 1, 2018 5:11 AM > An: dean.long at oracle.com; security-dev at openjdk.java.net; core-libs-dev Libs; hotspot-dev developers > Betreff: Re: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged > > Dean, > > src/java.base/share/classes/java/security/AccessController.java: > + /** > + * Internal marker for hidden implementation frames. > + */ > + /*non-public*/ > + @Target(ElementType.METHOD) > + @Retention(RetentionPolicy.RUNTIME) > + @interface Hidden { > + } > > You declare @Hidden, but then map it to _method_Hidden along with > @Hidden from java.lang.invoke.LambdaForm. > > What do you think about moving LambdaForm.Hidden to > jdk.internal.vm.annotation instead and share among all usages? > > Best regards, > Vladimir Ivanov > > On 31/10/2018 15:23, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8212605 >> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >> >> This change implements AccessController.doPrivileged in Java. This >> gives a performance improvement while also being useful to Project Loom >> by removing the Java --> native --> Java transition. One reason >> doPrivileged has historically been in native is because of the need to >> guarantee the cleanup of the privileged context when doPrivileged >> returns. To do that in Java, the information that >> AccessController.getContext needs is pushed onto the Java stack as >> arguments to a method that getContext will recognize during its stack >> walk. This allows us to remove the native privileged stack while >> guaranteeing that the privileged context goes away when the method returns. >> >> Tested with tier1-tier3 hotspot and jdk tests and JCK api/java_security >> tests. For the first few rounds of testing, I kept the old native >> privileged stack and compared the results of the old and new >> implementations for each getContext call, which did catch some early >> bugs. The changes were also examined by internal security experts and >> run through additional internal security tests. >> >> The improvement on this [1] doPrivileged microbenchmark is approximate 50x. >> >> There is no attempt to optimize getContext() or security permission >> checks in this change, however, this is intended to be a first step >> towards other possible improvements, for example those proposed here [2]. >> >> dl >> >> [1] >> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >> >> [2] >> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >> >> From vladimir.x.ivanov at oracle.com Thu Nov 1 08:18:37 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 1 Nov 2018 01:18:37 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <363cd5f9-e143-79cc-9698-6c8cb213ef1e@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <363cd5f9-e143-79cc-9698-6c8cb213ef1e@oracle.com> Message-ID: <43be67e3-c6a5-3ad2-d835-e9c32f90374f@oracle.com> > I think it's a good idea, but I believe it would require a CSR request. > Do you mind if I file a separate issue for > jdk.internal.vm.annotation.Hidden? Sure. Most of the annotations in jdk.internal.vm.annotation were originally located in java.lang.invoke. Not sure CSR will be required in this particular case. Best regards, Vladimir Ivanov > On 10/31/18 6:11 PM, Vladimir Ivanov wrote: >> Dean, >> >> src/java.base/share/classes/java/security/AccessController.java: >> +??? /** >> +???? * Internal marker for hidden implementation frames. >> +???? */ >> +??? /*non-public*/ >> +??? @Target(ElementType.METHOD) >> +??? @Retention(RetentionPolicy.RUNTIME) >> +??? @interface Hidden { >> +??? } >> >> You declare @Hidden, but then map it to _method_Hidden along with >> @Hidden from java.lang.invoke.LambdaForm. >> >> What do you think about moving LambdaForm.Hidden to >> jdk.internal.vm.annotation instead and share among all usages? >> >> Best regards, >> Vladimir Ivanov >> >> On 31/10/2018 15:23, dean.long at oracle.com wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8212605 >>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >>> >>> This change implements AccessController.doPrivileged in Java. This >>> gives a performance improvement while also being useful to Project >>> Loom by removing the Java --> native --> Java transition.? One reason >>> doPrivileged has historically been in native is because of the need >>> to guarantee the cleanup of the privileged context when doPrivileged >>> returns.? To do that in Java, the information that >>> AccessController.getContext needs is pushed onto the Java stack as >>> arguments to a method that getContext will recognize during its stack >>> walk.? This allows us to remove the native privileged stack while >>> guaranteeing that the privileged context goes away when the method >>> returns. >>> >>> Tested with tier1-tier3 hotspot and jdk tests and JCK >>> api/java_security tests.? For the first few rounds of testing, I kept >>> the old native privileged stack and compared the results of the old >>> and new implementations for each getContext call, which did catch >>> some early bugs.? The changes were also examined by internal security >>> experts and run through additional internal security tests. >>> >>> The improvement on this [1] doPrivileged microbenchmark is >>> approximate 50x. >>> >>> There is no attempt to optimize getContext() or security permission >>> checks in this change, however, this is intended to be a first step >>> towards other possible improvements, for example those proposed here >>> [2]. >>> >>> dl >>> >>> [1] >>> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >>> >>> [2] >>> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >>> >>> > From magnus.ihse.bursie at oracle.com Thu Nov 1 11:13:37 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 1 Nov 2018 12:13:37 +0100 Subject: What to include in the PCH list -- statistics on hotspot header files Message-ID: Prompted by a discussion on build-dev if the current set of header files included in the PCH are optimal (which it most definitely is not!), I collected some statistics. Here's a list of the current set of header files in hotspot with the number of times it is included, directly or indirectly, in a C++ file (as determined by the gcc dependency tracking). It is sorted in descending order, so for example allocation.hpp is included 586 times, while globalDefinitions.hpp is included 579 times. For comparison, there are 922 compilation units (C++ files). All numbers is from my linux build, so they might differ slightly between platforms or build configurations, but the overall picture is likely to remain the same. One way to tackle this information is to conclude that almost single header file is ever included in more than 50% of the compilations, so PCHs are not worth it at all. Another way is to conclude that there are 34 header files that are each included more than 150 times, and that means that it is reasonable to include it in the PCH. Or or some other cutoff, like 13 files over 300 times; this looks like a Pareto distribution, so the fewer files you include, they will be exponentially more useful. /Magnus -------------- next part -------------- hotspot/share/memory/allocation.hpp: 586 hotspot/share/utilities/globalDefinitions.hpp: 579 hotspot/share/runtime/os.hpp: 444 hotspot/share/utilities/align.hpp: 434 hotspot/share/utilities/macros.hpp: 407 hotspot/share/runtime/atomic.hpp: 390 hotspot/share/oops/oop.inline.hpp: 374 hotspot/share/services/memTracker.hpp: 361 hotspot/share/memory/resourceArea.hpp: 354 hotspot/share/memory/allocation.inline.hpp: 339 hotspot/share/logging/log.hpp: 311 hotspot/share/utilities/debug.hpp: 303 hotspot/share/oops/oop.hpp: 303 hotspot/share/utilities/growableArray.hpp: 285 hotspot/share/classfile/systemDictionary.hpp: 244 hotspot/share/oops/oopsHierarchy.hpp: 237 hotspot/share/utilities/ostream.hpp: 234 hotspot/share/runtime/handles.hpp: 227 hotspot/share/memory/universe.hpp: 226 hotspot/share/runtime/thread.hpp: 215 hotspot/share/oops/access.inline.hpp: 212 hotspot/share/gc/shared/collectedHeap.hpp: 212 hotspot/share/oops/instanceKlass.hpp: 204 hotspot/share/classfile/javaClasses.hpp: 197 hotspot/share/runtime/orderAccess.hpp: 196 hotspot/share/runtime/timer.hpp: 192 hotspot/share/runtime/handles.inline.hpp: 192 hotspot/share/oops/objArrayOop.hpp: 179 hotspot/share/runtime/globals.hpp: 175 hotspot/share/memory/iterator.hpp: 168 hotspot/share/utilities/copy.hpp: 163 hotspot/share/utilities/ticks.hpp: 156 hotspot/share/oops/method.hpp: 156 hotspot/share/oops/klass.hpp: 153 hotspot/share/runtime/mutex.hpp: 150 hotspot/share/memory/memRegion.hpp: 149 hotspot/share/classfile/classLoaderData.hpp: 149 hotspot/share/gc/shared/gcCause.hpp: 147 hotspot/share/utilities/bitMap.hpp: 140 hotspot/share/utilities/exceptions.hpp: 137 hotspot/share/oops/objArrayKlass.hpp: 131 hotspot/share/runtime/thread.inline.hpp: 128 hotspot/share/utilities/bitMap.inline.hpp: 122 hotspot/share/oops/arrayOop.inline.hpp: 122 hotspot/share/gc/shared/gcId.hpp: 122 hotspot/share/runtime/mutexLocker.hpp: 121 hotspot/share/oops/objArrayOop.inline.hpp: 118 hotspot/share/classfile/javaClasses.inline.hpp: 116 hotspot/share/memory/referenceType.hpp: 115 java.base/jni.h: 114 hotspot/share/oops/weakHandle.hpp: 114 hotspot/share/oops/compressedOops.inline.hpp: 114 hotspot/share/gc/shared/barrierSet.hpp: 113 hotspot/share/utilities/stack.hpp: 112 hotspot/share/gc/g1/g1YCTypes.hpp: 111 hotspot/share/memory/padded.hpp: 110 hotspot/share/logging/logHandle.hpp: 110 hotspot/share/runtime/deoptimization.hpp: 109 hotspot/share/gc/shared/workgroup.hpp: 109 hotspot/share/interpreter/interpreter.hpp: 108 hotspot/share/gc/shared/collectorCounters.hpp: 108 hotspot/share/gc/shared/taskqueue.hpp: 107 hotspot/share/services/memoryUsage.hpp: 106 hotspot/share/services/memoryService.hpp: 104 hotspot/share/runtime/vm_operations.hpp: 104 hotspot/share/runtime/arguments.hpp: 102 hotspot/share/oops/typeArrayKlass.hpp: 102 hotspot/share/memory/metaspace.hpp: 102 hotspot/share/classfile/vmSymbols.hpp: 101 hotspot/share/oops/typeArrayOop.hpp: 100 hotspot/share/oops/instanceMirrorKlass.hpp: 100 hotspot/share/services/memoryManager.hpp: 99 hotspot/share/oops/oopHandle.hpp: 99 hotspot/share/gc/shared/oopStorageParState.hpp: 98 hotspot/share/runtime/safepoint.hpp: 97 hotspot/share/gc/shared/gcUtil.hpp: 97 hotspot/share/oops/arrayKlass.hpp: 95 hotspot/share/utilities/stack.inline.hpp: 94 hotspot/share/runtime/sharedRuntime.hpp: 94 hotspot/share/runtime/jniHandles.hpp: 92 hotspot/share/gc/shared/referenceProcessor.hpp: 92 hotspot/share/logging/logLevel.hpp: 91 hotspot/share/runtime/signature.hpp: 90 hotspot/share/logging/logStream.hpp: 90 hotspot/share/oops/symbol.hpp: 89 hotspot/share/oops/instanceRefKlass.hpp: 89 hotspot/share/gc/shared/preservedMarks.hpp: 88 hotspot/share/runtime/prefetch.inline.hpp: 87 hotspot/share/include/jvm.h: 86 hotspot/share/runtime/vmThread.hpp: 84 hotspot/share/gc/shared/space.hpp: 84 hotspot/share/utilities/numberSeq.hpp: 83 hotspot/share/gc/shared/softRefPolicy.hpp: 83 hotspot/share/gc/g1/g1BarrierSet.hpp: 83 hotspot/share/oops/instanceClassLoaderKlass.hpp: 82 hotspot/share/oops/instanceKlass.inline.hpp: 81 hotspot/share/gc/shared/generationCounters.hpp: 81 hotspot/share/oops/typeArrayKlass.inline.hpp: 80 hotspot/share/oops/objArrayKlass.inline.hpp: 80 hotspot/share/oops/instanceRefKlass.inline.hpp: 80 hotspot/share/oops/instanceMirrorKlass.inline.hpp: 79 hotspot/share/oops/instanceClassLoaderKlass.inline.hpp: 79 hotspot/share/memory/iterator.inline.hpp: 79 hotspot/share/gc/shared/plab.hpp: 79 hotspot/share/gc/shared/gcHeapSummary.hpp: 79 hotspot/share/gc/shared/cardTableBarrierSet.hpp: 79 hotspot/share/gc/g1/heapRegion.hpp: 79 hotspot/share/code/codeCache.hpp: 79 hotspot/share/interpreter/bytecode.hpp: 78 hotspot/share/classfile/symbolTable.hpp: 78 hotspot/share/runtime/interfaceSupport.inline.hpp: 77 hotspot/share/memory/virtualspace.hpp: 76 hotspot/share/gc/g1/g1CardTable.hpp: 76 hotspot/share/runtime/java.hpp: 75 hotspot/share/gc/g1/g1RegionToSpaceMapper.hpp: 73 hotspot/share/gc/g1/dirtyCardQueue.hpp: 73 hotspot/share/oops/markOop.hpp: 71 hotspot/share/prims/jvmtiExport.hpp: 70 hotspot/share/oops/oopHandle.inline.hpp: 70 hotspot/share/gc/g1/g1BiasedArray.hpp: 70 hotspot/share/oops/array.hpp: 69 hotspot/share/oops/accessDecorators.hpp: 69 hotspot/share/gc/shared/referenceProcessorStats.hpp: 69 hotspot/share/gc/shared/gcWhen.hpp: 69 hotspot/share/gc/g1/g1OopClosures.hpp: 69 hotspot/share/utilities/pair.hpp: 68 hotspot/share/interpreter/linkResolver.hpp: 68 hotspot/share/gc/g1/g1InCSetState.hpp: 68 hotspot/share/gc/g1/evacuationInfo.hpp: 67 hotspot/share/utilities/hashtable.hpp: 66 hotspot/share/runtime/frame.hpp: 66 hotspot/share/gc/shared/workerDataArray.hpp: 66 hotspot/share/gc/shared/oopStorage.hpp: 66 hotspot/share/gc/shared/markBitMap.hpp: 66 hotspot/share/gc/g1/g1SurvivorRegions.hpp: 66 hotspot/share/gc/g1/g1RegionMarkStatsCache.hpp: 66 hotspot/share/gc/g1/g1EvacStats.hpp: 66 hotspot/share/compiler/compileBroker.hpp: 66 hotspot/share/utilities/sizes.hpp: 65 hotspot/share/runtime/javaCalls.hpp: 65 hotspot/share/memory/oopFactory.hpp: 65 hotspot/share/gc/shared/taskqueue.inline.hpp: 65 hotspot/share/gc/shared/copyFailedInfo.hpp: 65 hotspot/share/gc/g1/heapRegionSet.hpp: 65 hotspot/share/gc/g1/heapRegionManager.hpp: 65 hotspot/share/gc/g1/g1MonitoringSupport.hpp: 65 hotspot/share/gc/g1/g1HRPrinter.hpp: 65 hotspot/share/gc/g1/g1HeapVerifier.hpp: 65 hotspot/share/gc/g1/g1HeapTransition.hpp: 65 hotspot/share/gc/g1/g1EvacFailure.hpp: 65 hotspot/share/gc/g1/g1EdenRegions.hpp: 65 hotspot/share/gc/g1/g1ConcurrentMarkObjArrayProcessor.hpp: 65 hotspot/share/gc/g1/g1ConcurrentMark.hpp: 65 hotspot/share/gc/g1/g1ConcurrentMarkBitMap.hpp: 65 hotspot/share/gc/g1/g1CollectorState.hpp: 65 hotspot/share/gc/g1/g1CollectionSet.hpp: 65 hotspot/share/gc/g1/g1CollectedHeap.hpp: 65 hotspot/share/gc/g1/collectionSetChooser.hpp: 65 hotspot/share/ci/ciMethodData.hpp: 65 hotspot/share/gc/shared/referenceProcessorPhaseTimes.hpp: 64 hotspot/share/ci/ciMethod.hpp: 64 hotspot/share/runtime/frame.inline.hpp: 63 hotspot/share/gc/shared/gcName.hpp: 63 hotspot/share/utilities/resourceHash.hpp: 62 hotspot/share/gc/shared/gcTrace.hpp: 62 hotspot/share/gc/shared/collectedHeap.inline.hpp: 62 hotspot/share/jfr/utilities/jfrAllocation.hpp: 61 hotspot/share/gc/shared/ageTable.hpp: 61 hotspot/share/utilities/concurrentHashTable.hpp: 60 hotspot/share/utilities/bytes.hpp: 60 hotspot/share/classfile/classLoader.hpp: 60 hotspot/share/gc/shared/gcTimer.hpp: 59 hotspot/share/gc/g1/heapRegionType.hpp: 59 hotspot/share/code/stubs.hpp: 59 hotspot/share/gc/shared/weakProcessorPhases.hpp: 58 hotspot/share/gc/shared/suspendibleThreadSet.hpp: 58 hotspot/share/oops/weakHandle.inline.hpp: 57 hotspot/share/gc/shared/weakProcessorPhaseTimes.hpp: 57 hotspot/share/gc/shared/memset_with_concurrent_readers.hpp: 57 hotspot/share/gc/g1/g1YoungGenSizer.hpp: 57 hotspot/share/gc/g1/g1FromCardCache.hpp: 57 hotspot/share/gc/g1/g1CodeCacheRemSet.hpp: 57 hotspot/share/gc/g1/g1BlockOffsetTable.hpp: 57 hotspot/share/runtime/threadSMR.hpp: 56 hotspot/share/opto/loopnode.hpp: 56 hotspot/share/gc/g1/sparsePRT.hpp: 56 hotspot/share/gc/g1/heapRegionRemSet.hpp: 56 hotspot/share/gc/g1/g1Predictions.hpp: 56 hotspot/share/gc/g1/g1MMUTracker.hpp: 56 hotspot/share/code/nmethod.hpp: 56 hotspot/share/classfile/stringTable.hpp: 56 gensrc/jvmtifiles/jvmti.h: 56 hotspot/share/runtime/stubRoutines.hpp: 55 hotspot/share/opto/addnode.hpp: 55 hotspot/share/gc/shared/markBitMap.inline.hpp: 55 hotspot/share/gc/g1/g1RemSetTrackingPolicy.hpp: 55 hotspot/share/gc/g1/g1RemSetSummary.hpp: 55 hotspot/share/gc/g1/g1RemSet.hpp: 55 hotspot/share/gc/g1/g1RegionMarkStatsCache.inline.hpp: 55 hotspot/share/gc/g1/g1Policy.hpp: 55 hotspot/share/gc/g1/g1InitialMarkToMixedTimeTracker.hpp: 55 hotspot/share/gc/g1/g1GCPhaseTimes.hpp: 55 hotspot/share/runtime/globals_extension.hpp: 54 hotspot/share/opto/phaseX.hpp: 54 hotspot/share/gc/g1/heapRegionSet.inline.hpp: 54 hotspot/share/gc/g1/heapRegionManager.inline.hpp: 54 hotspot/share/gc/g1/heapRegion.inline.hpp: 54 hotspot/share/gc/g1/g1ParScanThreadState.inline.hpp: 54 hotspot/share/gc/g1/g1ParScanThreadState.hpp: 54 hotspot/share/gc/g1/g1OopClosures.inline.hpp: 54 hotspot/share/gc/g1/g1ConcurrentMarkObjArrayProcessor.inline.hpp: 54 hotspot/share/gc/g1/g1ConcurrentMark.inline.hpp: 54 hotspot/share/gc/g1/g1ConcurrentMarkBitMap.inline.hpp: 54 hotspot/share/gc/g1/g1CollectedHeap.inline.hpp: 54 hotspot/share/gc/g1/g1BlockOffsetTable.inline.hpp: 54 hotspot/share/opto/cfgnode.hpp: 53 hotspot/share/asm/macroAssembler.hpp: 53 hotspot/share/opto/node.hpp: 52 hotspot/share/opto/callnode.hpp: 52 hotspot/share/classfile/classLoaderData.inline.hpp: 52 hotspot/share/runtime/synchronizer.hpp: 51 hotspot/share/opto/compile.hpp: 51 hotspot/share/oops/annotations.hpp: 51 hotspot/share/gc/z/zGlobals.hpp: 51 hotspot/share/runtime/perfData.hpp: 50 hotspot/share/opto/opcodes.hpp: 50 hotspot/share/opto/subnode.hpp: 49 hotspot/share/gc/shared/genCollectedHeap.hpp: 49 hotspot/share/opto/memnode.hpp: 48 hotspot/share/jfr/utilities/jfrTypes.hpp: 48 hotspot/share/code/nativeInst.hpp: 47 hotspot/share/jfr/jfrEvents.hpp: 46 hotspot/share/runtime/init.hpp: 45 hotspot/share/memory/metaspaceShared.hpp: 45 hotspot/share/classfile/compactHashtable.hpp: 45 hotspot/share/ci/ciField.hpp: 45 hotspot/share/runtime/fieldDescriptor.hpp: 44 hotspot/share/opto/type.hpp: 44 hotspot/share/oops/methodData.hpp: 44 hotspot/share/jfr/utilities/jfrTime.hpp: 44 hotspot/share/c1/c1_FrameMap.hpp: 44 hotspot/share/runtime/safepointVerifiers.hpp: 43 hotspot/share/interpreter/bytecodeStream.hpp: 43 hotspot/share/gc/shared/concurrentGCThread.hpp: 43 hotspot/share/opto/connode.hpp: 42 hotspot/share/metaprogramming/primitiveConversions.hpp: 42 hotspot/share/runtime/vframe.hpp: 41 hotspot/share/opto/mulnode.hpp: 41 hotspot/share/interpreter/bytecodes.hpp: 41 hotspot/share/gc/shared/spaceDecorator.hpp: 41 hotspot/share/utilities/events.hpp: 40 hotspot/share/services/memoryPool.hpp: 40 hotspot/share/prims/jvmtiEventController.hpp: 40 hotspot/share/c1/c1_LIR.hpp: 40 hotspot/share/runtime/objectMonitor.hpp: 39 hotspot/share/opto/matcher.hpp: 39 hotspot/share/memory/heapInspection.hpp: 39 hotspot/share/code/pcDesc.hpp: 39 hotspot/share/ci/ciEnv.hpp: 39 hotspot/share/services/management.hpp: 38 hotspot/share/runtime/jniHandles.inline.hpp: 38 hotspot/share/opto/runtime.hpp: 38 hotspot/share/include/jmm.h: 38 hotspot/share/gc/shared/c2/barrierSetC2.hpp: 38 hotspot/share/compiler/compileLog.hpp: 38 hotspot/share/c1/c1_ValueType.hpp: 38 hotspot/share/c1/c1_Compilation.hpp: 38 hotspot/share/utilities/xmlstream.hpp: 37 hotspot/share/runtime/os.inline.hpp: 37 hotspot/share/runtime/flags/jvmFlag.hpp: 37 hotspot/share/opto/multnode.hpp: 37 hotspot/share/libadt/vectset.hpp: 37 hotspot/share/code/vmreg.hpp: 37 hotspot/share/ci/ciStreams.hpp: 37 hotspot/share/prims/jvmtiThreadState.hpp: 36 hotspot/share/jfr/writers/jfrStorageAdapter.hpp: 36 hotspot/share/jfr/writers/jfrMemoryWriterHost.inline.hpp: 36 hotspot/share/gc/z/zBitField.hpp: 36 hotspot/share/gc/shared/genOopClosures.hpp: 36 hotspot/share/gc/shared/collectorPolicy.hpp: 36 hotspot/share/classfile/moduleEntry.hpp: 36 hotspot/share/c1/c1_Runtime1.hpp: 36 hotspot/share/utilities/dtrace.hpp: 35 hotspot/share/runtime/vm_version.hpp: 35 hotspot/share/oops/constantPool.hpp: 35 gensrc/dtracefiles/hs_private.h: 35 gensrc/dtracefiles/hotspot_jni.h: 35 gensrc/dtracefiles/hotspot.h: 35 hotspot/share/oops/generateOopMap.hpp: 34 hotspot/share/ci/ciUtilities.inline.hpp: 34 hotspot/share/oops/typeArrayOop.inline.hpp: 33 hotspot/share/oops/markOop.inline.hpp: 33 hotspot/share/oops/arrayOop.hpp: 33 hotspot/share/gc/z/zOop.inline.hpp: 33 hotspot/share/gc/z/zAddress.inline.hpp: 33 hotspot/share/ci/ciExceptionHandler.hpp: 33 hotspot/share/c1/c1_Instruction.hpp: 33 hotspot/share/runtime/stackValueCollection.hpp: 32 hotspot/share/opto/machnode.hpp: 32 hotspot/share/opto/divnode.hpp: 32 hotspot/share/opto/castnode.hpp: 32 hotspot/share/jfr/recorder/storage/jfrBuffer.hpp: 32 hotspot/share/gc/z/zMarkStack.hpp: 32 hotspot/share/gc/z/zMarkStackEntry.hpp: 32 hotspot/share/gc/serial/markSweep.hpp: 32 hotspot/share/code/debugInfo.hpp: 32 hotspot/share/jfr/support/jfrIntrinsics.hpp: 31 hotspot/share/gc/z/zNUMA.hpp: 31 hotspot/share/gc/z/zMemory.hpp: 31 hotspot/share/gc/z/zList.hpp: 31 hotspot/share/compiler/disassembler.hpp: 31 hotspot/share/opto/rootnode.hpp: 30 hotspot/share/logging/logTag.hpp: 30 hotspot/share/gc/z/zUtils.hpp: 30 hotspot/share/gc/shared/gcLocker.hpp: 30 hotspot/share/utilities/vmError.hpp: 29 hotspot/share/prims/jvmtiUtil.hpp: 29 hotspot/share/prims/jvmtiEnvThreadState.hpp: 29 hotspot/share/metaprogramming/integralConstant.hpp: 29 hotspot/share/gc/shared/generation.hpp: 29 hotspot/share/gc/shared/cardTable.hpp: 29 hotspot/share/ci/ciKlass.hpp: 29 hotspot/share/asm/codeBuffer.hpp: 29 hotspot/os_cpu/linux_x86/gc/z/zBackingFile_linux_x86.hpp: 29 hotspot/share/runtime/fieldDescriptor.inline.hpp: 28 hotspot/share/prims/jvmtiTrace.hpp: 28 hotspot/share/prims/jvmtiImpl.hpp: 28 hotspot/share/jfr/recorder/checkpoint/jfrCheckpointBlob.hpp: 28 hotspot/share/gc/z/zVirtualMemory.hpp: 28 hotspot/share/gc/z/zPhysicalMemory.hpp: 28 hotspot/share/gc/z/zMarkStackAllocator.hpp: 28 hotspot/share/gc/z/zLock.hpp: 28 hotspot/os_cpu/linux_x86/gc/z/zPhysicalMemoryBacking_linux_x86.hpp: 28 hotspot/share/runtime/semaphore.hpp: 27 hotspot/share/runtime/monitorChunk.hpp: 27 hotspot/share/oops/cpCache.hpp: 27 hotspot/share/oops/access.hpp: 27 hotspot/share/gc/z/zForwardingTable.hpp: 27 hotspot/share/gc/z/zForwardingTableEntry.hpp: 27 hotspot/share/gc/shared/vmGCOperations.hpp: 27 hotspot/share/gc/shared/gcTraceTime.hpp: 27 hotspot/share/gc/serial/defNewGeneration.hpp: 27 hotspot/share/c1/c1_IR.hpp: 27 hotspot/share/runtime/biasedLocking.hpp: 26 hotspot/share/prims/jvmtiEnvBase.hpp: 26 hotspot/share/opto/regmask.hpp: 26 hotspot/share/opto/graphKit.hpp: 26 hotspot/share/opto/convertnode.hpp: 26 hotspot/share/oops/compiledICHolder.hpp: 26 hotspot/share/gc/z/zThread.hpp: 26 hotspot/share/gc/z/zPage.hpp: 26 hotspot/share/gc/z/zMarkTerminate.hpp: 26 hotspot/share/gc/z/zMark.hpp: 26 hotspot/share/gc/z/zLiveMap.hpp: 26 hotspot/share/gc/z/zCPU.hpp: 26 hotspot/share/gc/z/zBitMap.hpp: 26 hotspot/share/gc/shared/gcTraceTime.inline.hpp: 26 hotspot/share/gc/shared/blockOffsetTable.hpp: 26 hotspot/share/compiler/abstractCompiler.hpp: 26 hotspot/share/jfr/writers/jfrWriterHost.inline.hpp: 25 hotspot/share/gc/z/zUtils.inline.hpp: 25 hotspot/share/gc/z/zPhysicalMemory.inline.hpp: 25 hotspot/share/gc/z/zMemory.inline.hpp: 25 hotspot/share/gc/z/zHash.inline.hpp: 25 hotspot/share/gc/z/zHash.hpp: 25 hotspot/share/gc/serial/markSweep.inline.hpp: 25 hotspot/share/gc/parallel/parallelScavengeHeap.hpp: 25 hotspot/share/gc/parallel/mutableSpace.hpp: 25 hotspot/cpu/x86/vm_version_x86.hpp: 25 gensrc/jvmtifiles/jvmtiEnv.hpp: 25 hotspot/share/utilities/formatBuffer.hpp: 24 hotspot/share/runtime/timerTrace.hpp: 24 hotspot/share/runtime/flags/flagSetting.hpp: 24 hotspot/share/opto/phase.hpp: 24 hotspot/share/oops/method.inline.hpp: 24 hotspot/share/logging/logTagSet.hpp: 24 hotspot/share/gc/z/zVirtualMemory.inline.hpp: 24 hotspot/share/gc/z/zValue.hpp: 24 hotspot/share/gc/z/zForwardingTable.inline.hpp: 24 hotspot/share/gc/z/zArray.hpp: 24 hotspot/share/gc/shared/referenceDiscoverer.hpp: 24 hotspot/share/gc/shared/cardTableRS.hpp: 24 hotspot/share/gc/g1/g1_globals.hpp: 24 hotspot/share/utilities/defaultStream.hpp: 23 hotspot/share/services/diagnosticArgument.hpp: 23 hotspot/share/runtime/threadCritical.hpp: 23 hotspot/share/prims/methodHandles.hpp: 23 hotspot/share/opto/regalloc.hpp: 23 hotspot/share/metaprogramming/conditional.hpp: 23 hotspot/share/gc/z/zRelocationSet.hpp: 23 hotspot/share/gc/z/zPage.inline.hpp: 23 hotspot/share/gc/z/zLiveMap.inline.hpp: 23 hotspot/share/gc/z/zBitMap.inline.hpp: 23 hotspot/share/gc/z/zAllocationFlags.hpp: 23 hotspot/share/gc/z/zAddressRangeMap.hpp: 23 hotspot/share/gc/shared/space.inline.hpp: 23 hotspot/share/gc/shared/blockOffsetTable.inline.hpp: 23 hotspot/share/compiler/methodLiveness.hpp: 23 hotspot/share/asm/assembler.hpp: 23 hotspot/share/utilities/preserveException.hpp: 22 hotspot/share/services/diagnosticFramework.hpp: 22 hotspot/share/runtime/osThread.hpp: 22 hotspot/share/runtime/compilationPolicy.hpp: 22 hotspot/share/memory/metadataFactory.hpp: 22 hotspot/share/memory/filemap.hpp: 22 hotspot/share/logging/logDecorators.hpp: 22 hotspot/share/include/cds.h: 22 hotspot/share/gc/z/zWorkers.hpp: 22 hotspot/share/gc/z/zRootsIterator.hpp: 22 hotspot/share/gc/z/zRelocationSetSelector.hpp: 22 hotspot/share/gc/z/zPreMappedMemory.hpp: 22 hotspot/share/gc/z/zPageTable.hpp: 22 hotspot/share/gc/z/zPageTableEntry.hpp: 22 hotspot/share/gc/z/zPageCache.hpp: 22 hotspot/share/gc/shared/strongRootsScope.hpp: 22 hotspot/share/gc/shared/stringdedup/stringDedupStat.hpp: 22 hotspot/share/gc/shared/satbMarkQueue.hpp: 22 hotspot/share/gc/parallel/objectStartArray.hpp: 22 hotspot/share/code/scopeDesc.hpp: 22 hotspot/share/classfile/packageEntry.hpp: 22 hotspot/share/ci/ciMethodBlocks.hpp: 22 hotspot/share/ci/ciInstance.hpp: 22 hotspot/share/oops/fieldInfo.hpp: 21 hotspot/share/memory/metaspaceClosure.hpp: 21 hotspot/share/jvmci/jvmciRuntime.hpp: 21 hotspot/share/jfr/writers/jfrStreamWriterHost.inline.hpp: 21 hotspot/share/jfr/writers/jfrStreamWriterHost.hpp: 21 hotspot/share/jfr/recorder/repository/jfrChunkWriter.hpp: 21 hotspot/share/gc/z/zWeakRootsProcessor.hpp: 21 hotspot/share/gc/z/zThreadLocalData.hpp: 21 hotspot/share/gc/z/zServiceability.hpp: 21 hotspot/share/gc/z/zRelocate.hpp: 21 hotspot/share/gc/z/zReferenceProcessor.hpp: 21 hotspot/share/gc/z/zPageAllocator.hpp: 21 hotspot/share/gc/z/zObjectAllocator.hpp: 21 hotspot/share/gc/z/zMetronome.hpp: 21 hotspot/share/gc/z/zHeap.hpp: 21 hotspot/share/gc/z/zBarrierSet.hpp: 21 hotspot/share/gc/epsilon/epsilon_globals.hpp: 21 hotspot/share/compiler/compilerDirectives.hpp: 21 hotspot/share/code/compiledIC.hpp: 21 hotspot/share/classfile/protectionDomainCache.hpp: 21 hotspot/share/asm/macroAssembler.inline.hpp: 21 hotspot/share/aot/aotLoader.hpp: 21 hotspot/cpu/x86/macroAssembler_x86.inline.hpp: 21 hotspot/share/runtime/stubCodeGenerator.hpp: 20 hotspot/share/runtime/reflection.hpp: 20 hotspot/share/opto/block.hpp: 20 hotspot/share/gc/z/zStat.hpp: 20 hotspot/share/gc/shared/stringdedup/stringDedupTable.hpp: 20 hotspot/share/gc/shared/stringdedup/stringDedupQueue.hpp: 20 hotspot/share/gc/shared/stringdedup/stringDedup.hpp: 20 hotspot/share/gc/shared/barrierSetAssembler.hpp: 20 hotspot/share/gc/parallel/psVirtualspace.hpp: 20 hotspot/share/compiler/oopMap.hpp: 20 hotspot/share/services/threadService.hpp: 19 hotspot/share/oops/metadata.hpp: 19 hotspot/share/oops/klass.inline.hpp: 19 hotspot/share/oops/instanceOop.hpp: 19 hotspot/share/jfr/writers/jfrEventWriterHost.inline.hpp: 19 hotspot/share/gc/z/zMarkStack.inline.hpp: 19 hotspot/share/gc/z/zBarrier.inline.hpp: 19 hotspot/share/gc/z/zAddressRangeMap.inline.hpp: 19 hotspot/share/code/codeBlob.hpp: 19 hotspot/share/ci/ciTypeFlow.hpp: 19 hotspot/share/c1/c1_LIRGenerator.hpp: 19 hotspot/share/c1/c1_Decorators.hpp: 19 hotspot/cpu/x86/vmreg_x86.inline.hpp: 19 hotspot/share/services/diagnosticCommand.hpp: 18 hotspot/share/runtime/safepointMechanism.hpp: 18 hotspot/share/oops/cpCache.inline.hpp: 18 hotspot/share/metaprogramming/isRegisteredEnum.hpp: 18 hotspot/share/jfr/support/jfrThreadLocal.hpp: 18 hotspot/share/jfr/recorder/jfrRecorder.hpp: 18 hotspot/share/jfr/jni/jfrJavaSupport.hpp: 18 hotspot/share/jfr/jni/jfrJavaCall.hpp: 18 hotspot/share/interpreter/interpreterRuntime.hpp: 18 hotspot/share/gc/z/zPageTable.inline.hpp: 18 hotspot/share/gc/shared/plab.inline.hpp: 18 hotspot/share/gc/parallel/psOldGen.hpp: 18 hotspot/share/compiler/compilerOracle.hpp: 18 hotspot/share/code/relocInfo.hpp: 18 hotspot/share/code/debugInfoRec.hpp: 18 hotspot/share/classfile/dictionary.hpp: 18 hotspot/share/c1/c1_ValueStack.hpp: 18 hotspot/cpu/x86/interpreterRT_x86.hpp: 18 gensrc/jfrfiles/jfrTypes.hpp: 18 hotspot/share/utilities/compilerWarnings.hpp: 17 hotspot/share/runtime/fieldType.hpp: 17 hotspot/share/opto/movenode.hpp: 17 hotspot/share/memory/heap.hpp: 17 hotspot/share/jfr/utilities/jfrDoublyLinkedList.hpp: 17 hotspot/share/jfr/recorder/service/jfrOptionSet.hpp: 17 hotspot/share/interpreter/templateTable.hpp: 17 hotspot/share/gc/z/zMark.inline.hpp: 17 hotspot/share/gc/z/zHeap.inline.hpp: 17 hotspot/share/gc/shared/adaptiveSizePolicy.hpp: 17 hotspot/share/gc/g1/g1AllocRegion.inline.hpp: 17 hotspot/share/gc/g1/g1AllocRegion.hpp: 17 hotspot/share/code/icBuffer.hpp: 17 hotspot/share/classfile/classFileStream.hpp: 17 hotspot/share/c1/c1_globals.hpp: 17 hotspot/cpu/x86/macroAssembler_x86.hpp: 17 java.base/linux/jni_md.h: 16 hotspot/share/utilities/histogram.hpp: 16 hotspot/share/utilities/hashtable.inline.hpp: 16 hotspot/share/utilities/globalDefinitions_gcc.hpp: 16 hotspot/share/utilities/breakpoint.hpp: 16 hotspot/share/services/mallocTracker.hpp: 16 hotspot/share/runtime/vframe.inline.hpp: 16 hotspot/share/runtime/task.hpp: 16 hotspot/share/runtime/icache.hpp: 16 hotspot/share/runtime/globals_ext.hpp: 16 hotspot/share/opto/opaquenode.hpp: 16 hotspot/share/memory/metaspaceChunkFreeListSummary.hpp: 16 hotspot/share/jfr/utilities/jfrIterator.hpp: 16 hotspot/share/gc/z/z_globals.hpp: 16 hotspot/share/gc/shared/threadLocalAllocBuffer.hpp: 16 hotspot/share/gc/shared/isGCActiveMark.hpp: 16 hotspot/share/gc/shared/gc_globals.hpp: 16 hotspot/share/gc/serial/serial_globals.hpp: 16 hotspot/share/gc/parallel/parMarkBitMap.hpp: 16 hotspot/share/gc/parallel/parallel_globals.hpp: 16 hotspot/share/gc/g1/g1Allocator.inline.hpp: 16 hotspot/share/gc/g1/g1Allocator.hpp: 16 hotspot/share/gc/cms/parOopClosures.hpp: 16 hotspot/share/gc/cms/cms_globals.hpp: 16 hotspot/share/ci/ciUtilities.hpp: 16 hotspot/os/linux/globals_linux.hpp: 16 hotspot/os/linux/c2_globals_linux.hpp: 16 hotspot/os/linux/c1_globals_linux.hpp: 16 hotspot/os_cpu/linux_x86/os_linux_x86.hpp: 16 hotspot/os_cpu/linux_x86/globals_linux_x86.hpp: 16 hotspot/cpu/x86/globals_x86.hpp: 16 hotspot/cpu/x86/globalDefinitions_x86.hpp: 16 hotspot/cpu/x86/c2_globals_x86.hpp: 16 hotspot/cpu/x86/c1_globals_x86.hpp: 16 java.base/classfile_constants.h: 15 hotspot/share/utilities/chunkedList.hpp: 15 hotspot/share/runtime/vframeArray.hpp: 15 hotspot/share/runtime/perfMemory.hpp: 15 hotspot/share/runtime/os_ext.hpp: 15 hotspot/share/runtime/extendedPC.hpp: 15 hotspot/share/runtime/basicLock.hpp: 15 hotspot/share/opto/parse.hpp: 15 hotspot/share/opto/c2_globals.hpp: 15 hotspot/share/oops/klassVtable.hpp: 15 hotspot/share/metaprogramming/removeCV.hpp: 15 hotspot/share/metaprogramming/isSigned.hpp: 15 hotspot/share/metaprogramming/isIntegral.hpp: 15 hotspot/share/metaprogramming/isFloatingPoint.hpp: 15 hotspot/share/metaprogramming/enableIf.hpp: 15 hotspot/share/logging/logPrefix.hpp: 15 hotspot/share/jvmci/jvmciJavaClasses.hpp: 15 hotspot/share/jfr/support/jfrThreadId.hpp: 15 hotspot/share/jfr/recorder/storage/jfrMemorySpaceRetrieval.hpp: 15 hotspot/share/jfr/recorder/storage/jfrMemorySpace.hpp: 15 hotspot/share/jfr/recorder/checkpoint/jfrCheckpointWriter.hpp: 15 hotspot/share/interpreter/oopMapCache.hpp: 15 hotspot/share/interpreter/bytecodeHistogram.hpp: 15 hotspot/share/gc/g1/g1StringDedup.hpp: 15 hotspot/share/gc/cms/concurrentMarkSweepGeneration.hpp: 15 hotspot/share/code/dependencies.hpp: 15 hotspot/share/c1/c1_CodeStubs.hpp: 15 hotspot/share/asm/register.hpp: 15 hotspot/share/asm/assembler.inline.hpp: 15 hotspot/os/posix/os_posix.hpp: 15 hotspot/os/posix/include/jvm_md.h: 15 hotspot/os/linux/os_linux.hpp: 15 hotspot/share/utilities/accessFlags.hpp: 14 hotspot/share/runtime/safepointMechanism.inline.hpp: 14 hotspot/share/runtime/reflectionUtils.hpp: 14 hotspot/share/opto/narrowptrnode.hpp: 14 hotspot/share/opto/indexSet.hpp: 14 hotspot/share/logging/logConfiguration.hpp: 14 hotspot/share/jvmci/jvmciCompiler.hpp: 14 hotspot/share/jfr/periodic/sampling/jfrThreadSampler.hpp: 14 hotspot/share/jfr/jfr.hpp: 14 hotspot/share/interpreter/interp_masm.hpp: 14 hotspot/share/gc/shared/referencePolicy.hpp: 14 hotspot/share/gc/shared/genOopClosures.inline.hpp: 14 hotspot/share/gc/shared/c1/barrierSetC1.hpp: 14 hotspot/share/gc/serial/defNewGeneration.inline.hpp: 14 hotspot/share/gc/g1/g1ThreadLocalData.hpp: 14 hotspot/share/code/vmreg.inline.hpp: 14 hotspot/share/ci/ciObject.hpp: 14 hotspot/os/linux/os_linux.inline.hpp: 14 hotspot/cpu/x86/nativeInst_x86.hpp: 14 gensrc/jfrfiles/jfrEventIds.hpp: 14 hotspot/share/utilities/utf8.hpp: 13 hotspot/share/utilities/nativeCallStack.hpp: 13 hotspot/share/services/nmtCommon.hpp: 13 hotspot/share/runtime/handshake.hpp: 13 hotspot/share/opto/live.hpp: 13 hotspot/share/opto/chaitin.hpp: 13 hotspot/share/oops/constMethod.hpp: 13 hotspot/share/memory/binaryTreeDictionary.hpp: 13 hotspot/share/memory/arena.hpp: 13 hotspot/share/jvmci/jvmci_globals.hpp: 13 hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.hpp: 13 hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp: 13 hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceIdEpoch.hpp: 13 hotspot/share/gc/shared/weakProcessor.hpp: 13 hotspot/share/gc/shared/preservedMarks.inline.hpp: 13 hotspot/share/gc/parallel/psYoungGen.hpp: 13 hotspot/share/gc/parallel/psCardTable.hpp: 13 hotspot/share/gc/parallel/parMarkBitMap.inline.hpp: 13 hotspot/share/gc/epsilon/epsilonBarrierSet.hpp: 13 hotspot/share/gc/cms/compactibleFreeListSpace.hpp: 13 hotspot/share/compiler/compilerDefinitions.hpp: 13 hotspot/share/c1/c1_MacroAssembler.hpp: 13 hotspot/cpu/x86/register_x86.hpp: 13 java.base/share/native/libjimage/jimage.hpp: 12 hotspot/share/utilities/decoder.hpp: 12 hotspot/share/utilities/constantTag.hpp: 12 hotspot/share/services/virtualMemoryTracker.hpp: 12 hotspot/share/runtime/vframe_hp.hpp: 12 hotspot/share/runtime/sweeper.hpp: 12 hotspot/share/runtime/registerMap.hpp: 12 hotspot/share/prims/nativeLookup.hpp: 12 hotspot/share/prims/jvm_misc.hpp: 12 hotspot/share/opto/idealGraphPrinter.hpp: 12 hotspot/share/opto/c2compiler.hpp: 12 hotspot/share/oops/methodCounters.hpp: 12 hotspot/share/oops/constantPool.inline.hpp: 12 hotspot/share/oops/accessBackend.hpp: 12 hotspot/share/metaprogramming/removeReference.hpp: 12 hotspot/share/metaprogramming/removePointer.hpp: 12 hotspot/share/metaprogramming/isVolatile.hpp: 12 hotspot/share/metaprogramming/isSame.hpp: 12 hotspot/share/metaprogramming/isPointer.hpp: 12 hotspot/share/metaprogramming/decay.hpp: 12 hotspot/share/memory/metaspace/metabase.hpp: 12 hotspot/share/memory/freeList.hpp: 12 hotspot/share/logging/logTag_ext.hpp: 12 hotspot/share/logging/logDecorations.hpp: 12 hotspot/share/jfr/support/jfrTraceIdExtension.hpp: 12 hotspot/share/jfr/instrumentation/jfrEventClassTransformer.hpp: 12 hotspot/share/interpreter/invocationCounter.hpp: 12 hotspot/share/interpreter/bytecodeInterpreter.hpp: 12 hotspot/share/gc/shared/generationSpec.hpp: 12 hotspot/share/gc/shared/gcStats.hpp: 12 hotspot/share/gc/shared/gcPolicyCounters.hpp: 12 hotspot/share/gc/shared/barrierSetConfig.hpp: 12 hotspot/share/gc/parallel/psScavenge.hpp: 12 hotspot/share/gc/cms/parNewGeneration.hpp: 12 hotspot/share/code/vtableStubs.hpp: 12 hotspot/share/code/compressedStream.hpp: 12 hotspot/share/code/compiledMethod.inline.hpp: 12 hotspot/share/code/compiledMethod.hpp: 12 hotspot/share/classfile/verificationType.hpp: 12 hotspot/share/ci/ciSymbol.hpp: 12 hotspot/share/ci/ciInstanceKlass.hpp: 12 hotspot/share/ci/ciConstant.hpp: 12 hotspot/os_cpu/linux_x86/atomic_linux_x86.hpp: 12 gensrc/adfiles/ad_x86.hpp: 12 hotspot/share/utilities/count_trailing_zeros.hpp: 11 hotspot/share/runtime/threadHeapSampler.hpp: 11 hotspot/share/runtime/rtmLocking.hpp: 11 hotspot/share/opto/vectornode.hpp: 11 hotspot/share/opto/locknode.hpp: 11 hotspot/share/opto/intrinsicnode.hpp: 11 hotspot/share/opto/arraycopynode.hpp: 11 hotspot/share/oops/fieldStreams.hpp: 11 hotspot/share/metaprogramming/isConst.hpp: 11 hotspot/share/logging/logOutputList.hpp: 11 hotspot/share/logging/logMessageBuffer.hpp: 11 hotspot/share/jvmci/jvmciEnv.hpp: 11 hotspot/share/jfr/support/jfrKlassExtension.hpp: 11 hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.hpp: 11 hotspot/share/interpreter/templateInterpreter.hpp: 11 hotspot/share/gc/z/zRuntimeWorkers.hpp: 11 hotspot/share/gc/z/zInitialize.hpp: 11 hotspot/share/gc/z/zCollectorPolicy.hpp: 11 hotspot/share/gc/shared/oopStorage.inline.hpp: 11 hotspot/share/gc/shared/cardGeneration.hpp: 11 hotspot/share/gc/serial/cSpaceCounters.hpp: 11 hotspot/share/classfile/systemDictionaryShared.hpp: 11 hotspot/share/classfile/modules.hpp: 11 hotspot/share/classfile/classFileParser.hpp: 11 hotspot/share/ci/ciObjArrayKlass.hpp: 11 hotspot/share/ci/ciCallSite.hpp: 11 hotspot/share/utilities/stringUtils.hpp: 10 hotspot/share/utilities/linkedlist.hpp: 10 hotspot/share/services/lowMemoryDetector.hpp: 10 hotspot/share/services/allocationSite.hpp: 10 hotspot/share/runtime/unhandledOops.hpp: 10 hotspot/share/runtime/threadStatisticalInfo.hpp: 10 hotspot/share/runtime/threadLocalStorage.hpp: 10 hotspot/share/runtime/park.hpp: 10 hotspot/share/runtime/javaFrameAnchor.hpp: 10 hotspot/share/runtime/flags/jvmFlagConstraintList.hpp: 10 hotspot/share/prims/jvmtiRedefineClasses.hpp: 10 hotspot/share/opto/adlcVMDeps.hpp: 10 hotspot/share/opto/ad.hpp: 10 hotspot/share/memory/metaspace/metaspaceCommon.hpp: 10 hotspot/share/jvmci/vmSymbols_jvmci.hpp: 10 hotspot/share/jfr/utilities/jfrRefCountPointer.hpp: 10 hotspot/share/jfr/support/jfrThreadExtension.hpp: 10 hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceIdMacros.hpp: 10 hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceIdBits.inline.hpp: 10 hotspot/share/jfr/leakprofiler/chains/edge.hpp: 10 hotspot/share/interpreter/bytecodeTracer.hpp: 10 hotspot/share/interpreter/abstractInterpreter.hpp: 10 hotspot/share/gc/z/zMessagePort.hpp: 10 hotspot/share/gc/z/zFuture.hpp: 10 hotspot/share/gc/z/zDriver.hpp: 10 hotspot/share/gc/z/zDirector.hpp: 10 hotspot/share/gc/z/zCollectedHeap.hpp: 10 hotspot/share/gc/shared/gcArguments.hpp: 10 hotspot/share/gc/parallel/spaceCounters.hpp: 10 hotspot/share/gc/parallel/psParallelCompact.inline.hpp: 10 hotspot/share/gc/parallel/psParallelCompact.hpp: 10 hotspot/share/gc/parallel/psAdaptiveSizePolicy.hpp: 10 hotspot/share/gc/parallel/gcTaskManager.hpp: 10 hotspot/share/gc/g1/g1SATBMarkQueueSet.hpp: 10 hotspot/share/gc/cms/promotionInfo.hpp: 10 hotspot/share/gc/cms/cmsHeap.hpp: 10 hotspot/share/code/oopRecorder.hpp: 10 hotspot/share/classfile/verifier.hpp: 10 hotspot/share/classfile/altHashing.hpp: 10 hotspot/share/ci/ciTypeArrayKlass.hpp: 10 hotspot/share/ci/ciNullObject.hpp: 10 hotspot/share/c1/c1_LIRAssembler.hpp: 10 hotspot/share/c1/c1_Defs.hpp: 10 hotspot/os/posix/semaphore_posix.hpp: 10 hotspot/os/linux/osThread_linux.hpp: 10 hotspot/os_cpu/linux_x86/orderAccess_linux_x86.hpp: 10 hotspot/os_cpu/linux_x86/bytes_linux_x86.inline.hpp: 10 hotspot/cpu/x86/vmreg_x86.hpp: 10 hotspot/cpu/x86/stubRoutines_x86.hpp: 10 hotspot/cpu/x86/registerMap_x86.hpp: 10 hotspot/cpu/x86/javaFrameAnchor_x86.hpp: 10 hotspot/cpu/x86/frame_x86.hpp: 10 hotspot/cpu/x86/c1_LIRAssembler_x86.hpp: 10 hotspot/cpu/x86/bytes_x86.hpp: 10 java.base/jvmticmlr.h: 9 hotspot/share/utilities/elfFile.hpp: 9 hotspot/share/opto/idealKit.hpp: 9 hotspot/share/memory/metaspace/metachunk.hpp: 9 hotspot/share/jvmci/systemDictionary_jvmci.hpp: 9 hotspot/share/jfr/utilities/jfrHashtable.hpp: 9 hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.hpp: 9 hotspot/share/interpreter/cppInterpreter.hpp: 9 hotspot/share/gc/z/zResurrection.hpp: 9 hotspot/share/gc/z/zAddress.hpp: 9 hotspot/share/gc/shared/oopStorageParState.inline.hpp: 9 hotspot/share/gc/shared/gcThreadLocalData.hpp: 9 hotspot/share/gc/shared/gcConfig.hpp: 9 hotspot/share/gc/g1/g1StringDedupQueue.hpp: 9 hotspot/share/gc/g1/g1FullGCOopClosures.hpp: 9 hotspot/share/gc/g1/g1FullGCMarker.hpp: 9 hotspot/share/gc/g1/g1BarrierSet.inline.hpp: 9 hotspot/share/gc/cms/concurrentMarkSweepThread.hpp: 9 hotspot/share/ci/ciMethodHandle.hpp: 9 hotspot/share/ci/ciCallProfile.hpp: 9 hotspot/share/ci/ciArrayKlass.hpp: 9 hotspot/share/ci/ciArray.hpp: 9 hotspot/share/c1/c1_InstructionPrinter.hpp: 9 hotspot/os_cpu/linux_x86/thread_linux_x86.hpp: 9 hotspot/cpu/x86/templateTable_x86.hpp: 9 hotspot/cpu/x86/relocInfo_x86.hpp: 9 hotspot/cpu/x86/codeBuffer_x86.hpp: 9 hotspot/cpu/x86/assembler_x86.hpp: 9 hotspot/share/runtime/objectMonitor.inline.hpp: 8 hotspot/share/runtime/flags/jvmFlagWriteableList.hpp: 8 hotspot/share/runtime/flags/jvmFlagRangeList.hpp: 8 hotspot/share/opto/optoreg.hpp: 8 hotspot/share/oops/accessBackend.inline.hpp: 8 hotspot/share/libadt/dict.hpp: 8 hotspot/share/jfr/leakprofiler/leakProfiler.hpp: 8 hotspot/share/gc/z/zOopClosures.inline.hpp: 8 hotspot/share/gc/z/zOopClosures.hpp: 8 hotspot/share/gc/shared/ptrQueue.hpp: 8 hotspot/share/gc/shared/modRefBarrierSet.hpp: 8 hotspot/share/gc/shared/concurrentGCPhaseManager.hpp: 8 hotspot/share/gc/shared/cardTableBarrierSet.inline.hpp: 8 hotspot/share/gc/shared/accessBarrierSupport.inline.hpp: 8 hotspot/share/gc/parallel/psPromotionLAB.hpp: 8 hotspot/share/gc/parallel/psMarkSweep.hpp: 8 hotspot/share/gc/g1/g1RootProcessor.hpp: 8 hotspot/share/gc/g1/g1PageBasedVirtualSpace.hpp: 8 hotspot/share/gc/g1/g1FullGCScope.hpp: 8 hotspot/share/gc/g1/g1FullGCCompactionPoint.hpp: 8 hotspot/share/gc/g1/g1ConcurrentRefine.hpp: 8 hotspot/share/compiler/methodMatcher.hpp: 8 hotspot/share/classfile/classLoaderExt.hpp: 8 hotspot/share/ci/ciObjArray.hpp: 8 hotspot/share/c1/c1_ValueSet.hpp: 8 hotspot/os_cpu/linux_x86/gc/z/zGlobals_linux_x86.hpp: 8 hotspot/share/utilities/fakeRttiSupport.hpp: 7 hotspot/share/services/runtimeService.hpp: 7 hotspot/share/services/memBaseline.hpp: 7 hotspot/share/services/classLoadingService.hpp: 7 hotspot/share/services/attachListener.hpp: 7 hotspot/share/runtime/threadSMR.inline.hpp: 7 hotspot/share/prims/resolvedMethodTable.hpp: 7 hotspot/share/prims/jvmtiEventController.inline.hpp: 7 hotspot/share/prims/forte.hpp: 7 hotspot/share/opto/macro.hpp: 7 hotspot/share/opto/callGenerator.hpp: 7 hotspot/share/memory/metaspace/virtualSpaceNode.hpp: 7 hotspot/share/memory/metaspace/metaspaceStatistics.hpp: 7 hotspot/share/memory/metaspaceCounters.hpp: 7 hotspot/share/jfr/recorder/service/jfrPostBox.hpp: 7 hotspot/share/jfr/recorder/jfrEventSetting.inline.hpp: 7 hotspot/share/jfr/leakprofiler/utilities/unifiedOop.hpp: 7 hotspot/share/jfr/leakprofiler/chains/edgeStore.hpp: 7 hotspot/share/interpreter/bytecodeInterpreter.inline.hpp: 7 hotspot/share/gc/z/zTask.hpp: 7 hotspot/share/gc/z/zResurrection.inline.hpp: 7 hotspot/share/gc/z/zOop.hpp: 7 hotspot/share/gc/z/zBarrierSet.inline.hpp: 7 hotspot/share/gc/z/zBarrierSetAssembler.hpp: 7 hotspot/share/gc/z/zBarrier.hpp: 7 hotspot/share/gc/z/c2/zBarrierSetC2.hpp: 7 hotspot/share/gc/shared/modRefBarrierSet.inline.hpp: 7 hotspot/share/gc/shared/barrierSetConfig.inline.hpp: 7 hotspot/share/gc/shared/accessBarrierSupport.hpp: 7 hotspot/share/gc/serial/tenuredGeneration.hpp: 7 hotspot/share/gc/parallel/psPromotionManager.hpp: 7 hotspot/share/gc/parallel/psCompactionManager.hpp: 7 hotspot/share/gc/parallel/generationSizer.hpp: 7 hotspot/share/gc/g1/g1FullGCTask.hpp: 7 hotspot/share/gc/g1/g1FullGCOopClosures.inline.hpp: 7 hotspot/share/gc/g1/g1FullGCMarker.inline.hpp: 7 hotspot/share/gc/g1/g1FullCollector.hpp: 7 hotspot/share/gc/g1/g1ConcurrentMarkThread.hpp: 7 hotspot/share/gc/g1/g1CardCounts.hpp: 7 hotspot/share/gc/cms/cmsLockVerifier.hpp: 7 hotspot/share/compiler/compileTask.hpp: 7 hotspot/share/code/location.hpp: 7 hotspot/share/code/exceptionHandlerTable.hpp: 7 hotspot/share/classfile/metadataOnStackMark.hpp: 7 hotspot/share/ci/ciReplay.hpp: 7 hotspot/share/ci/ciMetadata.hpp: 7 hotspot/share/ci/ciClassList.hpp: 7 hotspot/share/c1/c1_ValueMap.hpp: 7 hotspot/os_cpu/linux_x86/gc/z/zAddress_linux_x86.inline.hpp: 7 hotspot/cpu/x86/icache_x86.hpp: 7 hotspot/cpu/x86/gc/shared/barrierSetAssembler_x86.hpp: 7 hotspot/cpu/x86/frame_x86.inline.hpp: 7 hotspot/share/utilities/json.hpp: 6 hotspot/share/services/memReporter.hpp: 6 hotspot/share/services/mallocSiteTable.hpp: 6 hotspot/share/services/heapDumper.hpp: 6 hotspot/share/runtime/stackValue.hpp: 6 hotspot/share/runtime/jfieldIDWorkaround.hpp: 6 hotspot/share/prims/privilegedStack.hpp: 6 hotspot/share/prims/jvmtiThreadState.inline.hpp: 6 hotspot/share/opto/mathexactnode.hpp: 6 hotspot/share/opto/countbitsnode.hpp: 6 hotspot/share/oops/verifyOopClosure.hpp: 6 hotspot/share/memory/padded.inline.hpp: 6 hotspot/share/memory/metaspaceShared.inline.hpp: 6 hotspot/share/memory/heapShared.hpp: 6 hotspot/share/jvmci/jvmciCompilerToVM.hpp: 6 hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.hpp: 6 hotspot/share/jfr/recorder/storage/jfrStorage.hpp: 6 hotspot/share/jfr/metadata/jfrSerializer.hpp: 6 hotspot/share/gc/z/zTracer.hpp: 6 hotspot/share/gc/z/zLargePages.hpp: 6 hotspot/share/gc/shared/modRefBarrierSetAssembler.hpp: 6 hotspot/share/gc/shared/jvmFlagConstraintsGC.hpp: 6 hotspot/share/gc/shared/gcArguments.inline.hpp: 6 hotspot/share/gc/shared/c2/modRefBarrierSetC2.hpp: 6 hotspot/share/gc/shared/c1/modRefBarrierSetC1.hpp: 6 hotspot/share/gc/parallel/psPromotionManager.inline.hpp: 6 hotspot/share/gc/parallel/psPromotionLAB.inline.hpp: 6 hotspot/share/gc/parallel/psMarkSweepProxy.hpp: 6 hotspot/share/gc/parallel/psMarkSweepDecorator.hpp: 6 hotspot/share/gc/g1/g1HotCardCache.hpp: 6 hotspot/share/gc/g1/g1HeapRegionTraceType.hpp: 6 hotspot/share/gc/cms/promotionInfo.inline.hpp: 6 hotspot/share/gc/cms/concurrentMarkSweepGeneration.inline.hpp: 6 hotspot/share/gc/cms/compactibleFreeListSpace.inline.hpp: 6 hotspot/share/classfile/javaAssertions.hpp: 6 hotspot/share/ci/ciType.hpp: 6 hotspot/share/ci/ciTypeArray.hpp: 6 hotspot/share/ci/ciSignature.hpp: 6 hotspot/share/ci/ciObjectFactory.hpp: 6 hotspot/share/ci/ciFlags.hpp: 6 hotspot/share/ci/ciConstantPoolCache.hpp: 6 hotspot/share/ci/ciBaseObject.hpp: 6 hotspot/share/c1/c1_FpuStackSim.hpp: 6 hotspot/share/c1/c1_Compiler.hpp: 6 hotspot/cpu/x86/jniTypes_x86.hpp: 6 hotspot/cpu/x86/interp_masm_x86.hpp: 6 hotspot/cpu/x86/gc/shared/modRefBarrierSetAssembler_x86.hpp: 6 hotspot/cpu/x86/c1_FpuStackSim_x86.hpp: 6 hotspot/share/utilities/singleWriterSynchronizer.hpp: 5 hotspot/share/runtime/statSampler.hpp: 5 hotspot/share/runtime/flags/jvmFlagConstraintsRuntime.hpp: 5 hotspot/share/runtime/flags/jvmFlagConstraintsCompiler.hpp: 5 hotspot/share/prims/jvmtiRawMonitor.hpp: 5 hotspot/share/prims/jniFastGetField.hpp: 5 hotspot/share/opto/classes.hpp: 5 hotspot/share/memory/metaspace/metablock.hpp: 5 hotspot/share/memory/metaspace/chunkManager.hpp: 5 hotspot/share/memory/heapShared.inline.hpp: 5 hotspot/share/memory/freeList.inline.hpp: 5 hotspot/share/logging/logOutput.hpp: 5 hotspot/share/logging/logMessage.hpp: 5 hotspot/share/logging/logFileStreamOutput.hpp: 5 hotspot/share/jfr/writers/jfrMemoryWriterHost.hpp: 5 hotspot/share/jfr/writers/jfrJavaEventWriter.hpp: 5 hotspot/share/jfr/writers/jfrEventWriterHost.hpp: 5 hotspot/share/jfr/writers/jfrEncoding.hpp: 5 hotspot/share/jfr/support/jfrFlush.hpp: 5 hotspot/share/jfr/support/jfrEventClass.hpp: 5 hotspot/share/jfr/recorder/stringpool/jfrStringPool.hpp: 5 hotspot/share/jfr/leakprofiler/sampling/objectSampler.hpp: 5 hotspot/share/jfr/leakprofiler/sampling/objectSample.hpp: 5 hotspot/share/jfr/dcmd/jfrDcmds.hpp: 5 hotspot/share/gc/z/zErrno.hpp: 5 hotspot/share/gc/shared/weakProcessor.inline.hpp: 5 hotspot/share/gc/shared/c2/cardTableBarrierSetC2.hpp: 5 hotspot/share/gc/parallel/psTasks.hpp: 5 hotspot/share/gc/parallel/psGCAdaptivePolicyCounters.hpp: 5 hotspot/share/gc/parallel/parallelScavengeHeap.inline.hpp: 5 hotspot/share/gc/parallel/jvmFlagConstraintsParallel.hpp: 5 hotspot/share/gc/parallel/immutableSpace.hpp: 5 hotspot/share/gc/parallel/asPSYoungGen.hpp: 5 hotspot/share/gc/parallel/asPSOldGen.hpp: 5 hotspot/share/gc/g1/jvmFlagConstraintsG1.hpp: 5 hotspot/share/gc/g1/heapRegionTracer.hpp: 5 hotspot/share/gc/g1/g1RootClosures.hpp: 5 hotspot/share/gc/g1/g1ConcurrentMarkThread.inline.hpp: 5 hotspot/share/gc/g1/g1Analytics.hpp: 5 hotspot/share/gc/epsilon/epsilonMonitoringSupport.hpp: 5 hotspot/share/gc/epsilon/epsilonHeap.hpp: 5 hotspot/share/gc/epsilon/epsilonCollectorPolicy.hpp: 5 hotspot/share/gc/cms/jvmFlagConstraintsCMS.hpp: 5 hotspot/share/gc/cms/freeChunk.hpp: 5 hotspot/share/compiler/directivesParser.hpp: 5 hotspot/share/code/dependencyContext.hpp: 5 hotspot/share/classfile/stackMapFrame.hpp: 5 hotspot/share/classfile/placeholders.hpp: 5 hotspot/share/ci/bcEscapeAnalyzer.hpp: 5 hotspot/share/c1/c1_LinearScan.hpp: 5 hotspot/share/c1/c1_Canonicalizer.hpp: 5 hotspot/os_cpu/linux_x86/copy_linux_x86.inline.hpp: 5 hotspot/cpu/x86/methodHandles_x86.hpp: 5 hotspot/cpu/x86/copy_x86.hpp: 5 hotspot/cpu/x86/c1_LinearScan_x86.hpp: 5 hotspot/share/utilities/spinYield.hpp: 4 hotspot/share/utilities/quickSort.hpp: 4 hotspot/share/services/writeableFlags.hpp: 4 hotspot/share/services/gcNotifier.hpp: 4 hotspot/share/runtime/serviceThread.hpp: 4 hotspot/share/runtime/relocator.hpp: 4 hotspot/share/runtime/os_perf.hpp: 4 hotspot/share/prims/whitebox.hpp: 4 hotspot/share/prims/jvmtiTagMap.hpp: 4 hotspot/share/prims/jvmtiManageCapabilities.hpp: 4 hotspot/share/prims/jniCheck.hpp: 4 hotspot/share/opto/replacednodes.hpp: 4 hotspot/share/oops/reflectionAccessorImplKlassHelper.hpp: 4 hotspot/share/memory/metaspace/smallBlocks.hpp: 4 hotspot/share/memory/metaspace/occupancyMap.hpp: 4 hotspot/share/memory/binaryTreeDictionary.inline.hpp: 4 hotspot/share/logging/logSelection.hpp: 4 hotspot/share/libadt/set.hpp: 4 hotspot/share/jvmci/jvmciCodeInstaller.hpp: 4 hotspot/share/jvmci/compilerRuntime.hpp: 4 hotspot/share/jfr/writers/jfrNativeEventWriter.hpp: 4 hotspot/share/jfr/recorder/service/jfrRecorderService.hpp: 4 hotspot/share/jfr/recorder/service/jfrEvent.hpp: 4 hotspot/share/jfr/recorder/repository/jfrRepository.hpp: 4 hotspot/share/jfr/recorder/jfrEventSetting.hpp: 4 hotspot/share/jfr/periodic/jfrOSInterface.hpp: 4 hotspot/share/jfr/leakprofiler/utilities/rootType.hpp: 4 hotspot/share/jfr/leakprofiler/utilities/granularTimer.hpp: 4 hotspot/share/jfr/leakprofiler/checkpoint/objectSampleCheckpoint.hpp: 4 hotspot/share/jfr/leakprofiler/chains/edgeQueue.hpp: 4 hotspot/share/jfr/leakprofiler/chains/bitset.hpp: 4 hotspot/share/interpreter/templateInterpreterGenerator.hpp: 4 hotspot/share/interpreter/rewriter.hpp: 4 hotspot/share/gc/z/zTracer.inline.hpp: 4 hotspot/share/gc/z/zNMethodTable.hpp: 4 hotspot/share/gc/z/zNMethodTableEntry.hpp: 4 hotspot/share/gc/z/zList.inline.hpp: 4 hotspot/share/gc/z/zBarrierSetRuntime.hpp: 4 hotspot/share/gc/shared/workerDataArray.inline.hpp: 4 hotspot/share/gc/shared/threadLocalAllocBuffer.inline.hpp: 4 hotspot/share/gc/shared/softRefGenPolicy.hpp: 4 hotspot/share/gc/shared/gcUtil.inline.hpp: 4 hotspot/share/gc/shared/gcLocker.inline.hpp: 4 hotspot/share/gc/shared/allocTracer.hpp: 4 hotspot/share/gc/serial/serialHeap.hpp: 4 hotspot/share/gc/parallel/psScavenge.inline.hpp: 4 hotspot/share/gc/parallel/psGenerationCounters.hpp: 4 hotspot/share/gc/parallel/psCompactionManager.inline.hpp: 4 hotspot/share/gc/parallel/objectStartArray.inline.hpp: 4 hotspot/share/gc/parallel/gcAdaptivePolicyCounters.hpp: 4 hotspot/share/gc/parallel/adjoiningVirtualSpaces.hpp: 4 hotspot/share/gc/g1/survRateGroup.hpp: 4 hotspot/share/gc/g1/g1ConcurrentRefineThread.hpp: 4 hotspot/share/gc/g1/g1CollectorPolicy.hpp: 4 hotspot/share/gc/cms/yieldingWorkgroup.hpp: 4 hotspot/share/gc/cms/gSpaceCounters.hpp: 4 hotspot/share/gc/cms/cmsOopClosures.hpp: 4 hotspot/share/gc/cms/allocationStats.hpp: 4 hotspot/share/gc/cms/adaptiveFreeList.hpp: 4 hotspot/share/classfile/stackMapTable.hpp: 4 hotspot/share/classfile/resolutionErrors.hpp: 4 hotspot/share/classfile/klassFactory.hpp: 4 hotspot/share/classfile/classListParser.hpp: 4 hotspot/share/ci/compilerInterface.hpp: 4 hotspot/share/ci/ciMemberName.hpp: 4 hotspot/share/c1/c1_CFGPrinter.hpp: 4 hotspot/cpu/x86/disassembler_x86.hpp: 4 hotspot/cpu/x86/c1_FrameMap_x86.hpp: 4 hotspot/cpu/x86/c1_Defs_x86.hpp: 4 gensrc/adfiles/adGlobals_x86.hpp: 4 hotspot/share/utilities/globalCounter.hpp: 3 hotspot/share/utilities/elfFuncDescTable.hpp: 3 hotspot/share/utilities/decoder_elf.hpp: 3 hotspot/share/runtime/vmStructs.hpp: 3 hotspot/share/runtime/semaphore.inline.hpp: 3 hotspot/share/runtime/prefetch.hpp: 3 hotspot/share/runtime/perfData.inline.hpp: 3 hotspot/share/runtime/memprofiler.hpp: 3 hotspot/share/prims/unsafe.hpp: 3 hotspot/share/prims/methodComparator.hpp: 3 hotspot/share/prims/jvmtiExtensions.hpp: 3 hotspot/share/prims/jvmtiCodeBlobEvents.hpp: 3 hotspot/share/prims/jvmtiAgentThread.hpp: 3 hotspot/share/precompiled/precompiled.hpp: 3 hotspot/share/opto/superword.hpp: 3 hotspot/share/opto/phasetype.hpp: 3 hotspot/share/opto/output.hpp: 3 hotspot/share/opto/escape.hpp: 3 hotspot/share/opto/coalesce.hpp: 3 hotspot/share/oops/arrayKlass.inline.hpp: 3 hotspot/share/memory/resourceArea.inline.hpp: 3 hotspot/share/memory/metaspace/virtualSpaceList.hpp: 3 hotspot/share/memory/metaspaceGCThresholdUpdater.hpp: 3 hotspot/share/memory/metaspace/blockFreelist.hpp: 3 hotspot/share/jvmci/vmStructs_jvmci.hpp: 3 hotspot/share/jfr/writers/jfrWriterHost.hpp: 3 hotspot/share/jfr/writers/jfrStorageHost.inline.hpp: 3 hotspot/share/jfr/writers/jfrStorageHost.hpp: 3 hotspot/share/jfr/writers/jfrPosition.inline.hpp: 3 hotspot/share/jfr/writers/jfrPosition.hpp: 3 hotspot/share/jfr/writers/jfrEncoders.hpp: 3 hotspot/share/jfr/utilities/jfrTimeConverter.hpp: 3 hotspot/share/jfr/recorder/storage/jfrVirtualMemory.hpp: 3 hotspot/share/jfr/recorder/storage/jfrStorageUtils.inline.hpp: 3 hotspot/share/jfr/recorder/storage/jfrStorageUtils.hpp: 3 hotspot/share/jfr/recorder/storage/jfrStorageControl.hpp: 3 hotspot/share/jfr/recorder/storage/jfrMemorySpace.inline.hpp: 3 hotspot/share/jfr/recorder/service/jfrRecorderThread.hpp: 3 hotspot/share/jfr/recorder/repository/jfrChunkState.hpp: 3 hotspot/share/jfr/recorder/repository/jfrChunkSizeNotifier.hpp: 3 hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.hpp: 3 hotspot/share/jfr/recorder/checkpoint/jfrMetadataEvent.hpp: 3 hotspot/share/jfr/periodic/jfrThreadCPULoadEvent.hpp: 3 hotspot/share/jfr/periodic/jfrNetworkUtilization.hpp: 3 hotspot/share/jfr/leakprofiler/chains/rootSetClosure.hpp: 3 hotspot/share/jfr/leakprofiler/chains/edgeUtils.hpp: 3 hotspot/share/jfr/leakprofiler/chains/dfsClosure.hpp: 3 hotspot/share/jfr/jni/jfrUpcalls.hpp: 3 hotspot/share/jfr/jni/jfrGetAllEventClasses.hpp: 3 hotspot/share/jfr/instrumentation/jfrJvmtiAgent.hpp: 3 hotspot/share/gc/z/zWorkers.inline.hpp: 3 hotspot/share/gc/z/zStatTLAB.hpp: 3 hotspot/share/gc/z/zLock.inline.hpp: 3 hotspot/share/gc/z/zLargePages.inline.hpp: 3 hotspot/share/gc/z/zArray.inline.hpp: 3 hotspot/share/gc/z/c1/zBarrierSetC1.hpp: 3 hotspot/share/gc/shared/stringdedup/stringDedupThread.hpp: 3 hotspot/share/gc/shared/referenceProcessor.inline.hpp: 3 hotspot/share/gc/shared/objectCountEventSender.hpp: 3 hotspot/share/gc/shared/hSpaceCounters.hpp: 3 hotspot/share/gc/shared/genMemoryPools.hpp: 3 hotspot/share/gc/shared/cardGeneration.inline.hpp: 3 hotspot/share/gc/shared/ageTable.inline.hpp: 3 hotspot/share/gc/serial/tenuredGeneration.inline.hpp: 3 hotspot/share/gc/serial/genMarkSweep.hpp: 3 hotspot/share/gc/parallel/adjoiningGenerations.hpp: 3 hotspot/share/gc/g1/vm_operations_g1.hpp: 3 hotspot/share/gc/g1/heapRegionBounds.inline.hpp: 3 hotspot/share/gc/g1/heapRegionBounds.hpp: 3 hotspot/share/gc/g1/g1YoungRemSetSamplingThread.hpp: 3 hotspot/share/gc/g1/g1MemoryPool.hpp: 3 hotspot/share/gc/g1/g1HeapSizingPolicy.hpp: 3 hotspot/share/gc/g1/g1FullGCReferenceProcessorExecutor.hpp: 3 hotspot/share/gc/g1/g1EvacStats.inline.hpp: 3 hotspot/share/gc/g1/g1CodeBlobClosure.hpp: 3 hotspot/share/gc/g1/g1BarrierSetRuntime.hpp: 3 hotspot/share/gc/g1/g1BarrierSetAssembler.hpp: 3 hotspot/share/gc/g1/c1/g1BarrierSetC1.hpp: 3 hotspot/share/gc/cms/vmCMSOperations.hpp: 3 hotspot/share/gc/cms/cmsCollectorPolicy.hpp: 3 hotspot/share/code/codeHeapState.hpp: 3 hotspot/share/classfile/sharedPathsMiscInfo.hpp: 3 hotspot/share/classfile/loaderConstraints.hpp: 3 hotspot/share/classfile/defaultMethods.hpp: 3 hotspot/share/classfile/classLoaderStats.hpp: 3 hotspot/share/classfile/classLoader.inline.hpp: 3 hotspot/share/ci/ciMethodType.hpp: 3 hotspot/share/c1/c1_GraphBuilder.hpp: 3 hotspot/share/aot/compiledIC_aot.hpp: 3 hotspot/share/aot/aotCompiledMethod.hpp: 3 hotspot/share/aot/aotCodeHeap.hpp: 3 hotspot/os/linux/osContainer_linux.hpp: 3 hotspot/os_cpu/linux_x86/prefetch_linux_x86.inline.hpp: 3 hotspot/cpu/x86/vm_version_ext_x86.hpp: 3 hotspot/cpu/x86/rdtsc_x86.hpp: 3 hotspot/cpu/x86/gc/z/zBarrierSetAssembler_x86.hpp: 3 hotspot/cpu/x86/gc/g1/g1BarrierSetAssembler_x86.hpp: 3 hotspot/cpu/x86/c1_MacroAssembler_x86.hpp: 3 hotspot/cpu/x86/assembler_x86.inline.hpp: 3 gensrc/jfrfiles/jfrEventControl.hpp: 3 gensrc/jfrfiles/jfrEventClasses.hpp: 3 hotspot/share/utilities/intHisto.hpp: 2 hotspot/share/utilities/globalCounter.inline.hpp: 2 hotspot/share/utilities/elfSymbolTable.hpp: 2 hotspot/share/utilities/elfStringTable.hpp: 2 hotspot/share/utilities/concurrentHashTableTasks.inline.hpp: 2 hotspot/share/utilities/concurrentHashTable.inline.hpp: 2 hotspot/share/services/nmtDCmd.hpp: 2 hotspot/share/services/mallocTracker.inline.hpp: 2 hotspot/share/services/dtraceAttacher.hpp: 2 hotspot/share/runtime/tieredThresholdPolicy.hpp: 2 hotspot/share/runtime/sharedRuntimeMath.hpp: 2 hotspot/share/runtime/rframe.hpp: 2 hotspot/share/runtime/jniPeriodicChecker.hpp: 2 hotspot/share/prims/whitebox.inline.hpp: 2 hotspot/share/prims/wbtestmethods/parserTests.hpp: 2 hotspot/share/prims/stackwalk.hpp: 2 hotspot/share/prims/jvmtiGetLoadedClasses.hpp: 2 hotspot/share/prims/jvmtiEnter.inline.hpp: 2 hotspot/share/prims/jvmtiClassFileReconstituter.hpp: 2 hotspot/share/prims/cdsoffsets.hpp: 2 hotspot/share/opto/stringopts.hpp: 2 hotspot/share/memory/metaspaceTracer.hpp: 2 hotspot/share/memory/metaspace/spaceManager.hpp: 2 hotspot/share/memory/metaspace/printMetaspaceInfoKlassClosure.hpp: 2 hotspot/share/memory/metaspace/printCLDMetaspaceInfoClosure.hpp: 2 hotspot/share/memory/metaspace/metaspaceDCmd.hpp: 2 hotspot/share/memory/metaspace/metaDebug.hpp: 2 hotspot/share/memory/guardedMemory.hpp: 2 hotspot/share/logging/logTagSetDescriptions.hpp: 2 hotspot/share/logging/logSelectionList.hpp: 2 hotspot/share/logging/logFileOutput.hpp: 2 hotspot/share/logging/logDiagnosticCommand.hpp: 2 hotspot/share/jfr/writers/jfrBigEndianWriter.hpp: 2 hotspot/share/jfr/utilities/jfrTryLock.hpp: 2 hotspot/share/jfr/utilities/jfrJavaLog.hpp: 2 hotspot/share/jfr/utilities/jfrBigEndian.hpp: 2 hotspot/share/jfr/support/jfrAllocationTracer.hpp: 2 hotspot/share/jfr/recorder/stringpool/jfrStringPoolWriter.hpp: 2 hotspot/share/jfr/recorder/service/jfrMemorySizer.hpp: 2 hotspot/share/jfr/recorder/repository/jfrEmergencyDump.hpp: 2 hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetWriter.hpp: 2 hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSet.hpp: 2 hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.hpp: 2 hotspot/share/jfr/recorder/checkpoint/types/jfrType.hpp: 2 hotspot/share/jfr/recorder/checkpoint/types/jfrThreadState.hpp: 2 hotspot/share/jfr/recorder/checkpoint/types/jfrThreadGroup.hpp: 2 hotspot/share/jfr/periodic/sampling/jfrCallTrace.hpp: 2 hotspot/share/jfr/periodic/jfrThreadDumpEvent.hpp: 2 hotspot/share/jfr/periodic/jfrModuleEvent.hpp: 2 hotspot/share/jfr/leakprofiler/utilities/saveRestore.hpp: 2 hotspot/share/jfr/leakprofiler/sampling/samplePriorityQueue.hpp: 2 hotspot/share/jfr/leakprofiler/sampling/sampleList.hpp: 2 hotspot/share/jfr/leakprofiler/emitEventOperation.hpp: 2 hotspot/share/jfr/leakprofiler/checkpoint/rootResolver.hpp: 2 hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.hpp: 2 hotspot/share/jfr/leakprofiler/checkpoint/objectSampleDescription.hpp: 2 hotspot/share/jfr/leakprofiler/chains/objectSampleMarker.hpp: 2 hotspot/share/jfr/leakprofiler/chains/bfsClosure.hpp: 2 hotspot/share/jfr/jni/jfrJniMethodRegistration.hpp: 2 hotspot/share/interpreter/cppInterpreterGenerator.hpp: 2 hotspot/share/interpreter/bytecodeInterpreterProfiling.hpp: 2 hotspot/share/interpreter/bytecode.inline.hpp: 2 hotspot/share/gc/z/zRelocationSet.inline.hpp: 2 hotspot/share/gc/z/zPreMappedMemory.inline.hpp: 2 hotspot/share/gc/z/zMarkCache.inline.hpp: 2 hotspot/share/gc/z/zMarkCache.hpp: 2 hotspot/share/gc/z/zHeapIterator.hpp: 2 hotspot/share/gc/z/zFuture.inline.hpp: 2 hotspot/share/gc/z/zArguments.hpp: 2 hotspot/share/gc/z/vmStructs_z.hpp: 2 hotspot/share/gc/shared/workerManager.hpp: 2 hotspot/share/gc/shared/stringdedup/stringDedupQueue.inline.hpp: 2 hotspot/share/gc/shared/parallelCleaning.hpp: 2 hotspot/share/gc/shared/memAllocator.hpp: 2 hotspot/share/gc/shared/gcConfiguration.hpp: 2 hotspot/share/gc/shared/cardTableBarrierSetAssembler.hpp: 2 hotspot/share/gc/shared/c1/cardTableBarrierSetC1.hpp: 2 hotspot/share/gc/shared/ageTableTracer.hpp: 2 hotspot/share/gc/serial/serialArguments.hpp: 2 hotspot/share/gc/parallel/vmPSOperations.hpp: 2 hotspot/share/gc/parallel/psMemoryPool.hpp: 2 hotspot/share/gc/parallel/pcTasks.hpp: 2 hotspot/share/gc/parallel/parallelArguments.hpp: 2 hotspot/share/gc/parallel/mutableNUMASpace.hpp: 2 hotspot/share/gc/parallel/gcTaskThread.hpp: 2 hotspot/share/gc/g1/g1StringDedupStat.hpp: 2 hotspot/share/gc/g1/g1IHOPControl.hpp: 2 hotspot/share/gc/g1/g1HeapRegionEventSender.hpp: 2 hotspot/share/gc/g1/g1FullGCPrepareTask.hpp: 2 hotspot/share/gc/g1/g1FullGCMarkTask.hpp: 2 hotspot/share/gc/g1/g1FullGCCompactTask.hpp: 2 hotspot/share/gc/g1/g1FullGCAdjustTask.hpp: 2 hotspot/share/gc/g1/g1CardTable.inline.hpp: 2 hotspot/share/gc/g1/g1Arguments.hpp: 2 hotspot/share/gc/g1/c2/g1BarrierSetC2.hpp: 2 hotspot/share/gc/epsilon/epsilonThreadLocalData.hpp: 2 hotspot/share/gc/epsilon/epsilonMemoryPool.hpp: 2 hotspot/share/gc/epsilon/epsilonArguments.hpp: 2 hotspot/share/gc/cms/cmsGCStats.hpp: 2 hotspot/share/gc/cms/cmsCardTable.hpp: 2 hotspot/share/gc/cms/cmsArguments.hpp: 2 hotspot/share/classfile/stackMapTableFormat.hpp: 2 hotspot/share/classfile/classLoaderHierarchyDCmd.hpp: 2 hotspot/share/classfile/bytecodeAssembler.hpp: 2 hotspot/share/c1/c1_ValueSet.inline.hpp: 2 hotspot/share/c1/c1_RangeCheckElimination.hpp: 2 hotspot/share/c1/c1_Optimizer.hpp: 2 hotspot/os/linux/os_share_linux.hpp: 2 hotspot/os_cpu/linux_x86/gc/z/zBackingPath_linux_x86.hpp: 2 hotspot/cpu/x86/gc/shared/cardTableBarrierSetAssembler_x86.hpp: 2 hotspot/cpu/x86/depChecker_x86.hpp: 2 hotspot/cpu/x86/crc32c.h: 2 gensrc/jfrfiles/jfrPeriodic.hpp: 2 hotspot/share/utilities/internalVMTests.hpp: 1 hotspot/share/prims/jniExport.hpp: 1 hotspot/share/oops/methodData.inline.hpp: 1 hotspot/share/oops/array.inline.hpp: 1 hotspot/share/jvmci/vmStructs_compiler_runtime.hpp: 1 hotspot/share/jfr/utilities/jfrResourceManager.hpp: 1 hotspot/share/jfr/utilities/jfrLogTagSets.hpp: 1 hotspot/share/jfr/support/jfrStackTraceMark.hpp: 1 hotspot/share/jfr/leakprofiler/stopOperation.hpp: 1 hotspot/share/jfr/leakprofiler/startOperation.hpp: 1 hotspot/share/jfr/jni/jfrJniMethod.hpp: 1 hotspot/share/gc/z/zPageCache.inline.hpp: 1 hotspot/share/gc/z/zMessagePort.inline.hpp: 1 hotspot/share/gc/z/zMarkTerminate.inline.hpp: 1 hotspot/share/gc/shared/vmStructs_gc.hpp: 1 hotspot/share/gc/shared/stringdedup/stringDedupThread.inline.hpp: 1 hotspot/share/gc/shared/stringdedup/stringDedup.inline.hpp: 1 hotspot/share/gc/serial/vmStructs_serial.hpp: 1 hotspot/share/gc/serial/serialHeap.inline.hpp: 1 hotspot/share/gc/parallel/vmStructs_parallelgc.hpp: 1 hotspot/share/gc/g1/vmStructs_g1.hpp: 1 hotspot/share/gc/g1/g1SharedClosures.hpp: 1 hotspot/share/gc/g1/g1CodeRootSetTable.hpp: 1 hotspot/share/gc/epsilon/vmStructs_epsilon.hpp: 1 hotspot/share/gc/cms/vmStructs_cms.hpp: 1 hotspot/share/gc/cms/parOopClosures.inline.hpp: 1 hotspot/share/gc/cms/parNewGeneration.inline.hpp: 1 hotspot/share/gc/cms/cmsOopClosures.inline.hpp: 1 hotspot/share/gc/cms/cmsHeap.inline.hpp: 1 hotspot/share/code/relocInfo_ext.hpp: 1 hotspot/share/classfile/dictionary.inline.hpp: 1 hotspot/share/aot/aotLoader.inline.hpp: 1 hotspot/os/linux/vmStructs_linux.hpp: 1 hotspot/os_cpu/linux_x86/vmStructs_linux_x86.hpp: 1 hotspot/os_cpu/linux_x86/os_linux_x86.inline.hpp: 1 hotspot/cpu/x86/vmStructs_x86.hpp: 1 From rkennke at redhat.com Thu Nov 1 11:20:13 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 1 Nov 2018 12:20:13 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> Message-ID: <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. Erik: ok for you too? Thanks, Roman >> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >> >> Hi Erik, >> >> right. Fixed this, and what what Kim mentioned plus a missing include: >> >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >> >> Ok now? > > Looks good. > From lois.foltan at oracle.com Thu Nov 1 12:44:29 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 1 Nov 2018 08:44:29 -0400 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: <08c00d04-d1c7-1d40-c032-55ca45f0324c@oracle.com> Thank you Martin for the review! Lois On 10/30/2018 9:33 PM, Martin Balao wrote: > Hi Lois, > > Thanks for your proposal. I'm not a reviewer but it looks good to me too. > > Kind regards, > Martin.- > > On Mon, Oct 29, 2018 at 4:47 PM, Lois Foltan > wrote: > > On 10/25/2018 11:15 AM, Martin Balao wrote: > >> Hi Lois, >> >> Thanks for having a look at this. >> >> Yes, I have a test that triggers this execution path. >> Unfortunately, this is a security test that I cannot publicly >> share. What I can tell you, though, is that the parent class >> loader is a "delegating class loader" which never defined a class >> and that's the reason why it doesn't have a ClassLoaderData >> instance created. >> >> I'm not sure if this would be needed in other places where the >> idiom is not used. The reason is that there might be any other >> implicit assumptions that makes it impossible. That's why I >> limited the scope of this changeset to the only case that I'm >> sure it's possible to trigger. > > Hi Martin, > > I wrote a test case that reproduces this failure and demonstrates > that invoking SystemDictionary::register_loader() will not work.? > With a non-product (fastdebug) build, the test case shows that > calling register_loader is problematic, a SystemDictionary lock is > currently held and acquiring a ClassLoaderDataGraph lock to create > the new ClassLoaderData causes an out of order locking fatal error. > > Please consider this test case and proposed change: > http://cr.openjdk.java.net/~lfoltan/bug_jdk8212937/webrev/ > > > Thanks, > Lois > >> >> Kind regards, >> Martin.- >> >> On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan >> > wrote: >> >> On 10/24/2018 3:58 PM, Martin Balao wrote: >> >> Hi, >> >> Can I have a review for JDK-8212937 [1]? >> >> Webrev.00: >> >> ? * >> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ >> >> ? * >> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip >> >> >> By obtaining the ClassLoaderData instance through a call to >> SystemDictionary::register_loader, we make sure that >> either an existing >> instance is obtained or a new one created. This same >> idiom has already been >> used in other places where there are no guarantees that a >> previous >> ClassLoaderData instance exists for a given class loader. >> See futher >> information in JDK-8212937 ticket [1]. >> >> Thanks, >> Martin.- >> >> -- >> [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 >> >> >> Hi Martin, >> >> I agree that the same idiom is used in other places, but >> there are also a couple of occurrences of >> java_lang_ClassLoader::parent in >> classfile/classLoaderStats.cpp and >> classfile/classLoaderHierarchyDCmd.cpp that do not.? In order >> to better understand, do you have a test case that >> demonstrates this issue? >> >> Thanks, >> Lois >> >> > > From lois.foltan at oracle.com Thu Nov 1 12:45:47 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Thu, 1 Nov 2018 08:45:47 -0400 Subject: RFR 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: <5976703e-b4b0-d865-1e2f-b793f8cfb03b@oracle.com> Message-ID: <0f251d60-f33c-fbb6-395e-55ef531b22b6@oracle.com> On 10/30/2018 6:40 PM, coleen.phillimore at oracle.com wrote: > > > On 10/30/18 2:13 PM, Lois Foltan wrote: >> On 10/30/2018 11:01 AM, Severin Gehwolf wrote: >> >>> Hi Lois, >>> >>> On Mon, 2018-10-29 at 15:47 -0400, Lois Foltan wrote: >>>> On 10/25/2018 11:15 AM, Martin Balao wrote: >>>> >>>>> Hi Lois, >>>>> >>>>> Thanks for having a look at this. >>>>> >>>>> Yes, I have a test that triggers this execution path. Unfortunately, >>>>> this is a security test that I cannot publicly share. What I can tell >>>>> you, though, is that the parent class loader is a "delegating class >>>>> loader" which never defined a class and that's the reason why it >>>>> doesn't have a ClassLoaderData instance created. >>>>> >>>>> I'm not sure if this would be needed in other places where the idiom >>>>> is not used. The reason is that there might be any other implicit >>>>> assumptions that makes it impossible. That's why I limited the scope >>>>> of this changeset to the only case that I'm sure it's possible to >>>>> trigger. >>>> Hi Martin, >>>> >>>> I wrote a test case that reproduces this failure and demonstrates that >>>> invoking SystemDictionary::register_loader() will not work. With a >>>> non-product (fastdebug) build, the test case shows that calling >>>> register_loader is problematic, a SystemDictionary lock is currently >>>> held and acquiring a ClassLoaderDataGraph lock to create the new >>>> ClassLoaderData causes an out of order locking fatal error. >>>> >>>> Please consider this test case and proposed change: >>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8212937/webrev/ > > This change looks good to me.? I'd prefer not calling > register_loader() when one is only needed to produce an error > message.? This also avoids the lock inversion problem. Thank you Coleen for the review!? Testing: hs-tier1-4 and jdk-tier1-3 have completed successfully. Lois > > Thanks, > Coleen > >>> FWIW, I've tested this webrev with our reproducer and this fix works >>> for us. >> >> Thank you Severin for verifying this! >> Lois >> >>> >>> Thanks, >>> Severin >>> >>>>> Kind regards, >>>>> Martin.- >>>>> >>>>> On Wed, Oct 24, 2018 at 5:52 PM, Lois Foltan >>>> > wrote: >>>>> >>>>> ???? On 10/24/2018 3:58 PM, Martin Balao wrote: >>>>> >>>>> ???????? Hi, >>>>> >>>>> ???????? Can I have a review for JDK-8212937 [1]? >>>>> >>>>> ???????? Webrev.00: >>>>> >>>>> ?????????? * >>>>> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00/ >>>>> >>>>> >>>>> ?????????? * >>>>> http://cr.openjdk.java.net/~mbalao/webrevs/8212937/8212937.webrev.00.zip >>>>> >>>>> >>>>> >>>>> >>>>> ???????? By obtaining the ClassLoaderData instance through a call to >>>>> ???????? SystemDictionary::register_loader, we make sure that >>>>> either an >>>>> ???????? existing >>>>> ???????? instance is obtained or a new one created. This same >>>>> idiom has >>>>> ???????? already been >>>>> ???????? used in other places where there are no guarantees that a >>>>> previous >>>>> ???????? ClassLoaderData instance exists for a given class loader. >>>>> See >>>>> ???????? futher >>>>> ???????? information in JDK-8212937 ticket [1]. >>>>> >>>>> ???????? Thanks, >>>>> ???????? Martin.- >>>>> >>>>> ???????? -- >>>>> ???????? [1] - https://bugs.openjdk.java.net/browse/JDK-8212937 >>>>> >>>>> >>>>> ???? Hi Martin, >>>>> >>>>> ???? I agree that the same idiom is used in other places, but >>>>> there are >>>>> ???? also a couple of occurrences of java_lang_ClassLoader::parent in >>>>> ???? classfile/classLoaderStats.cpp and >>>>> ???? classfile/classLoaderHierarchyDCmd.cpp that do not. In order to >>>>> ???? better understand, do you have a test case that demonstrates >>>>> this >>>>> ???? issue? >>>>> >>>>> ???? Thanks, >>>>> ???? Lois >>>>> >>>>> >>>> >> > From erik.osterlund at oracle.com Thu Nov 1 12:58:41 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 1 Nov 2018 13:58:41 +0100 Subject: RFR: 8212989: Allow CompiledMethod ExceptionCache have unloaded klasses Message-ID: <5BDAF881.3030905@oracle.com> Hi, The ExceptionCaches of compiled methods is traditionally pruned in safepoints during class unloading. This allows the exception cache to have lock-free reads, while performing inserts under a lock, and pruning under a safepoint. With concurrent class unloading, the exception caches need to be cleaned concurrently. To retain the lock-free reads that seem good to keep lock-free, a lock-free cleaning mechanism was introduced. Only one thread per compiled method cleans each exception cache, but that thread races with both single writers (under the ExceptionCache_lock) and any number of readers. The head of the exception cache list is concurrently cleaned by both inserting threads and cleaning threads. This allows having an invariant that no newly prepended entries ever produce next pointers to dead exception caches, that are concurrently removed from the list. As for the internal next pointers, they are only pruned by the one concurrent cleaning thread. This happens concurrent to reads that simply skip over dead entries as they have different Klass* pointers to the one being searched for. The single concurrent cleanup thread does not delete things removed from the list straight away. Instead, they are placed on a purge list that is freed up after a subsequent global handshaking operation. That allows ABA-free CAS instructions in the lock-free paths, and allows safe concurrent reading of entries in the exception cache list. Note that we already incorrectly "relied on memory_order_consume" for the head pointer. But it used a volatile load. Assuming volatile loads retain memory_order_consume semantics is not allowed in HotSpot runtime code, so I changed those loads to use acquire accordingly. Webrev: http://cr.openjdk.java.net/~eosterlund/8212989/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8212989 Thanks, /Erik From erik.osterlund at oracle.com Thu Nov 1 13:52:27 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 1 Nov 2018 14:52:27 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> Message-ID: <5BDB051B.6060703@oracle.com> Hi Roman, Would you mind explaining how you need to override this and why? I'm afraid I did not quite get it from your description in the RFC (which is also why I didn't come up with a solution either). Am I correct that you need to return false if the passed in offset is the cell header offset -8, or is there anything else at all required to that logic? You mentioned something about the high order byte being masked on AArch64, but didn't quite connect the dot to how that is related to this code. Is it? Thanks, /Erik On 2018-11-01 12:20, Roman Kennke wrote: > Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. > > Erik: ok for you too? > > Thanks, > Roman > > >>> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >>> >>> Hi Erik, >>> >>> right. Fixed this, and what what Kim mentioned plus a missing include: >>> >>> Incremental: >>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>> Full: >>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>> >>> Ok now? >> Looks good. >> From sean.mullan at oracle.com Thu Nov 1 16:48:57 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 1 Nov 2018 12:48:57 -0400 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <61f7523d-3fce-023b-f806-aa49a02f6d9c@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <61f7523d-3fce-023b-f806-aa49a02f6d9c@oracle.com> Message-ID: <8c969cca-fab2-4731-3d81-a956c1898677@oracle.com> On 11/1/18 1:29 AM, dean.long at oracle.com wrote: > On 10/31/18 9:39 PM, Bernd Eckenfels wrote: >> http://cr.openjdk.java.net/~dlong/8212605/webrev.1/src/java.base/share/classes/java/security/AccessController.java.udiff.html >> >> >> In checkContext should the security manager be null checked first >> instead of last to optimize for the typical case? (If the side effects >> in that expression are desired it should be documented) > > I was following the example of createWrapper.? The side-effects of > getInnocuousAcc() will only happen once, so the order shouldn't matter > here, except for performance reasons.? I don't have a strong opinion > about the order, but it looks like the typical case for createWrapper > would also be the typical case for checkContext, so maybe they both > should be changed, if indeed a null security manager is the more typical > case. A null SM should be the more common case. I am ok with changing the order so the SM is checked first, but it should be done in both the createWrapper and checkContext methods. Alternatively, we could see if they could be combined to eliminate the duplicate code but that might not be practical from looking at them. --Sean From rkennke at redhat.com Thu Nov 1 16:58:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 1 Nov 2018 17:58:24 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BDB051B.6060703@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> Message-ID: <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> Hi Erik, > Would you mind explaining how you need to override this and why? I'm > afraid I did not quite get it from your description in the RFC (which is > also why I didn't come up with a solution either). > > Am I correct that you need to return false if the passed in offset is > the cell header offset -8, or is there anything else at all required to > that logic? No, it's really just that. Plus take care of it in the case of combined narrow-oops-offset plus -8. > You mentioned something about the high order byte being > masked on AArch64, but didn't quite connect the dot to how that is > related to this code. Is it? Yes, we also need to mask the high order byte in AArch64 because of the way addressing works on that platform, and because -8 flips on those negative bits. Also, the mach5 job came back with FAILED (see below). Can somebody with access check and see what's up? Thanks, Roman Build Details: 2018-11-01-1146402.roman.source 0 Failed Tests Mach5 Tasks Results Summary EXECUTED_WITH_FAILURE: 6 KILLED: 0 UNABLE_TO_RUN: 21 PASSED: 48 FAILED: 0 NA: 0 Build 6 Executed with failure solaris-sparcv9-solaris-sparcv9-build-8 error while building, return value: 2 solaris-sparcv9-debug-solaris-sparcv9-build-9 error while building, return value: 2 windows-x64-windows-x64-build-10 error while building, return value: 2 windows-x64-debug-windows-x64-build-11 error while building, return value: 2 windows-x64-open-windows-x64-build-12 error while building, return value: 2 windows-x64-open-debug-windows-x64-build-13 error while building, return value: 2 2 Not run solaris-sparcv9-install-solaris-sparcv9-build-16 Dependency task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 windows-x64-install-windows-x64-build-17 Dependency task failed: YJftjiBUYc Test 19 Not run tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 Dependency task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 Dependency task failed: mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 Dependency task failed: YJftjiBUYc tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 Dependency task failed: YJftjiBVYc tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 Dependency task failed: YJftjiBVYc tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 Dependency task failed: YJftjiBVYc tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 Dependency task failed: YJftjiBVYc tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 Dependency task failed: YJftjiBVYc tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 Dependency task failed: YJftjiBVYc tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 Dependency task failed: YJftjiBVYc See all 19... > Thanks, > /Erik > > On 2018-11-01 12:20, Roman Kennke wrote: >> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >> >> Erik: ok for you too? >> >> Thanks, >> Roman >> >> >>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >>>> >>>> Hi Erik, >>>> >>>> right. Fixed this, and what what Kim mentioned plus a missing include: >>>> >>>> Incremental: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>> Full: >>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>> >>>> Ok now? >>> Looks good. >>> > From sean.mullan at oracle.com Thu Nov 1 17:01:26 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 1 Nov 2018 13:01:26 -0400 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> Message-ID: <670ff93f-e8e0-d7a0-d410-daa42b5a9a32@oracle.com> Some of the copyrights need to be updated to 2018. All else looks good to me as I had reviewed an earlier version of this before. We have talked about doing this for a while now, so I am finally glad we and are able to pretty much eliminate one of the more common SecurityManager related hot-spots and give a performance boost to applications that don't use the SM as well. --Sean On 10/31/18 6:23 PM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8212605 > http://cr.openjdk.java.net/~dlong/8212605/webrev.1 > > This change implements AccessController.doPrivileged in Java.? This > gives a performance improvement while also being useful to Project Loom > by removing the Java --> native --> Java transition.? One reason > doPrivileged has historically been in native is because of the need to > guarantee the cleanup of the privileged context when doPrivileged > returns.? To do that in Java, the information that > AccessController.getContext needs is pushed onto the Java stack as > arguments to a method that getContext will recognize during its stack > walk.? This allows us to remove the native privileged stack while > guaranteeing that the privileged context goes away when the method returns. > > Tested with tier1-tier3 hotspot and jdk tests and JCK api/java_security > tests.? For the first few rounds of testing, I kept the old native > privileged stack and compared the results of the old and new > implementations for each getContext call, which did catch some early > bugs.? The changes were also examined by internal security experts and > run through additional internal security tests. > > The improvement on this [1] doPrivileged microbenchmark is approximate 50x. > > There is no attempt to optimize getContext() or security permission > checks in this change, however, this is intended to be a first step > towards other possible improvements, for example those proposed here [2]. > > dl > > [1] > http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java > > [2] > http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html > > From ioi.lam at oracle.com Thu Nov 1 18:07:13 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 1 Nov 2018 11:07:13 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure Message-ID: https://bugs.openjdk.java.net/browse/JDK-8213250 http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ When Metaspace::allocate() fails, usually we first try to GC to see if some MetaspaceObjs can be freed. This is an optimization to avoid prematurely expanding the metaspace. However, GC cannot run in the last phase of CDS which runs inside the VM thread. The old code just aborts the CDS dump. The fix is to unconditionally expand the metaspace without doing a GC. (During GC dump time, no classes are unloaded, so even if we force a GC, it would not free up any metaspace anyway). Testing: Before the fix, the test case (jruby) would fail to dump once a couple of times. Now it has dumped hundreds of times without failure. I am also running hs-tier 1/2 Thanks - Ioi From dean.long at oracle.com Thu Nov 1 19:11:02 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 1 Nov 2018 12:11:02 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <670ff93f-e8e0-d7a0-d410-daa42b5a9a32@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <670ff93f-e8e0-d7a0-d410-daa42b5a9a32@oracle.com> Message-ID: <1a5cf1b4-d0ef-1b25-4d69-ecc9f4725c77@oracle.com> On 11/1/18 10:01 AM, Sean Mullan wrote: > Some of the copyrights need to be updated to 2018. > Fixed. > All else looks good to me as I had reviewed an earlier version of this > before. We have talked about doing this for a while now, so I am > finally glad we and are able to pretty much eliminate one of the more > common SecurityManager related hot-spots and give a performance boost > to applications that don't use the SM as well. > Thanks Sean. dl > --Sean > > On 10/31/18 6:23 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8212605 >> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >> >> This change implements AccessController.doPrivileged in Java. This >> gives a performance improvement while also being useful to Project >> Loom by removing the Java --> native --> Java transition.? One reason >> doPrivileged has historically been in native is because of the need >> to guarantee the cleanup of the privileged context when doPrivileged >> returns.? To do that in Java, the information that >> AccessController.getContext needs is pushed onto the Java stack as >> arguments to a method that getContext will recognize during its stack >> walk.? This allows us to remove the native privileged stack while >> guaranteeing that the privileged context goes away when the method >> returns. >> >> Tested with tier1-tier3 hotspot and jdk tests and JCK >> api/java_security tests.? For the first few rounds of testing, I kept >> the old native privileged stack and compared the results of the old >> and new implementations for each getContext call, which did catch >> some early bugs.? The changes were also examined by internal security >> experts and run through additional internal security tests. >> >> The improvement on this [1] doPrivileged microbenchmark is >> approximate 50x. >> >> There is no attempt to optimize getContext() or security permission >> checks in this change, however, this is intended to be a first step >> towards other possible improvements, for example those proposed here >> [2]. >> >> dl >> >> [1] >> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >> >> [2] >> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >> >> From dean.long at oracle.com Thu Nov 1 19:21:07 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 1 Nov 2018 12:21:07 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <8c969cca-fab2-4731-3d81-a956c1898677@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <61f7523d-3fce-023b-f806-aa49a02f6d9c@oracle.com> <8c969cca-fab2-4731-3d81-a956c1898677@oracle.com> Message-ID: <8a485f73-e942-12e8-9c7d-0944ff0af2de@oracle.com> On 11/1/18 9:48 AM, Sean Mullan wrote: > On 11/1/18 1:29 AM, dean.long at oracle.com wrote: >> On 10/31/18 9:39 PM, Bernd Eckenfels wrote: >>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1/src/java.base/share/classes/java/security/AccessController.java.udiff.html >>> >>> >>> In checkContext should the security manager be null checked first >>> instead of last to optimize for the typical case? (If the side >>> effects in that expression are desired it should be documented) >> >> I was following the example of createWrapper.? The side-effects of >> getInnocuousAcc() will only happen once, so the order shouldn't >> matter here, except for performance reasons.? I don't have a strong >> opinion about the order, but it looks like the typical case for >> createWrapper would also be the typical case for checkContext, so >> maybe they both should be changed, if indeed a null security manager >> is the more typical case. > > A null SM should be the more common case. I am ok with changing the > order so the SM is checked first, but it should be done in both the > createWrapper and checkContext methods. Alternatively, we could see if > they could be combined to eliminate the duplicate code but that might > not be practical from looking at them. > I don't see a way to do it without using a lambda or doing extra work in one version, so I just changed the order for now.? I also replaced getCallerPD with the faster getProtectionDomain and removed a stale comment about impliesCreateAccessControlContext being called by the VM.? It should be safe to remove now, but I left it in to minimize changes. http://cr.openjdk.java.net/~dlong/8212605/webrev.3.incr/ http://cr.openjdk.java.net/~dlong/8212605/webrev.3/ dl > --Sean From sean.mullan at oracle.com Thu Nov 1 19:39:41 2018 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 1 Nov 2018 15:39:41 -0400 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <8a485f73-e942-12e8-9c7d-0944ff0af2de@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <61f7523d-3fce-023b-f806-aa49a02f6d9c@oracle.com> <8c969cca-fab2-4731-3d81-a956c1898677@oracle.com> <8a485f73-e942-12e8-9c7d-0944ff0af2de@oracle.com> Message-ID: <72357551-9393-d3e5-ced1-98452354ae9b@oracle.com> On 11/1/18 3:21 PM, dean.long at oracle.com wrote: > On 11/1/18 9:48 AM, Sean Mullan wrote: >> On 11/1/18 1:29 AM, dean.long at oracle.com wrote: >>> On 10/31/18 9:39 PM, Bernd Eckenfels wrote: >>>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1/src/java.base/share/classes/java/security/AccessController.java.udiff.html >>>> >>>> >>>> In checkContext should the security manager be null checked first >>>> instead of last to optimize for the typical case? (If the side >>>> effects in that expression are desired it should be documented) >>> >>> I was following the example of createWrapper.? The side-effects of >>> getInnocuousAcc() will only happen once, so the order shouldn't >>> matter here, except for performance reasons.? I don't have a strong >>> opinion about the order, but it looks like the typical case for >>> createWrapper would also be the typical case for checkContext, so >>> maybe they both should be changed, if indeed a null security manager >>> is the more typical case. >> >> A null SM should be the more common case. I am ok with changing the >> order so the SM is checked first, but it should be done in both the >> createWrapper and checkContext methods. Alternatively, we could see if >> they could be combined to eliminate the duplicate code but that might >> not be practical from looking at them. >> > > I don't see a way to do it without using a lambda or doing extra work in > one version, so I just changed the order for now. Yes, I was thinking about using a lambda too, but I'm a bit wary of using lambdas in the security checking code. Best to keep the methods separate for now. > I also replaced > getCallerPD with the faster getProtectionDomain and removed a stale > comment about impliesCreateAccessControlContext being called by the VM. > It should be safe to remove now, but I left it in to minimize changes. I would just remove it, so we don't forget about it later. Looks good otherwise. --Sean > http://cr.openjdk.java.net/~dlong/8212605/webrev.3.incr/ > http://cr.openjdk.java.net/~dlong/8212605/webrev.3/ > > dl > >> --Sean > From vladimir.kozlov at oracle.com Thu Nov 1 20:14:00 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 1 Nov 2018 13:14:00 -0700 Subject: RFR: 8212989: Allow CompiledMethod ExceptionCache have unloaded klasses In-Reply-To: <5BDAF881.3030905@oracle.com> References: <5BDAF881.3030905@oracle.com> Message-ID: Hi Erik, It seems these changed are not based on latest sources. I don't see call to CodeCache::do_unloading in CodeCache::do_unloading() which is present in latest code: http://hg.openjdk.java.net/jdk/jdk/file/5ea020bcaa0d/src/hotspot/share/code/codeCache.cpp#l683 May be make some methods private, like CompiledMethod::exception_cache_acquire(). It seems you remove the only case when release_set_exception_cache() is called. I don't see where purge_exception_caches() is used. Next sentence seems incomplete: + // unlinked. That is also when the CodeCache::exception_cache_free_list() Why you use exception_cache_acquire() in clean_exception_cache() and simple exception_cache() in add_exception_cache_entry()? Thanks, Vladimir On 11/1/18 5:58 AM, Erik ?sterlund wrote: > Hi, > > The ExceptionCaches of compiled methods is traditionally pruned in safepoints during class unloading. This allows the > exception cache to have lock-free reads, while performing inserts under a lock, and pruning under a safepoint. With > concurrent class unloading, the exception caches need to be cleaned concurrently. > > To retain the lock-free reads that seem good to keep lock-free, a lock-free cleaning mechanism was introduced. Only one > thread per compiled method cleans each exception cache, but that thread races with both single writers (under the > ExceptionCache_lock) and any number of readers. > > The head of the exception cache list is concurrently cleaned by both inserting threads and cleaning threads. This allows > having an invariant that no newly prepended entries ever produce next pointers to dead exception caches, that are > concurrently removed from the list. As for the internal next pointers, they are only pruned by the one concurrent > cleaning thread. This happens concurrent to reads that simply skip over dead entries as they have different Klass* > pointers to the one being searched for. > > The single concurrent cleanup thread does not delete things removed from the list straight away. Instead, they are > placed on a purge list that is freed up after a subsequent global handshaking operation. That allows ABA-free CAS > instructions in the lock-free paths, and allows safe concurrent reading of entries in the exception cache list. > > Note that we already incorrectly "relied on memory_order_consume" for the head pointer. But it used a volatile load. > Assuming volatile loads retain memory_order_consume semantics is not allowed in HotSpot runtime code, so I changed those > loads to use acquire accordingly. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212989/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212989 > > Thanks, > /Erik From mandy.chung at oracle.com Thu Nov 1 20:45:37 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 1 Nov 2018 13:45:37 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <43be67e3-c6a5-3ad2-d835-e9c32f90374f@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <363cd5f9-e143-79cc-9698-6c8cb213ef1e@oracle.com> <43be67e3-c6a5-3ad2-d835-e9c32f90374f@oracle.com> Message-ID: On 11/1/18 1:18 AM, Vladimir Ivanov wrote: > >> I think it's a good idea, but I believe it would require a CSR >> request. Do you mind if I file a separate issue for >> jdk.internal.vm.annotation.Hidden? > > Sure. > > Most of the annotations in jdk.internal.vm.annotation were originally > located in java.lang.invoke. Not sure CSR will be required in this > particular case. > @Hidden is internal and no CSR is needed. FYI.? JDK-8212620 is the RFE to consider providing a standard mechanism to hide frames from stack trace. Mandy > Best regards, > Vladimir Ivanov > >> On 10/31/18 6:11 PM, Vladimir Ivanov wrote: >>> Dean, >>> >>> src/java.base/share/classes/java/security/AccessController.java: >>> +??? /** >>> +???? * Internal marker for hidden implementation frames. >>> +???? */ >>> +??? /*non-public*/ >>> +??? @Target(ElementType.METHOD) >>> +??? @Retention(RetentionPolicy.RUNTIME) >>> +??? @interface Hidden { >>> +??? } >>> >>> You declare @Hidden, but then map it to _method_Hidden along with >>> @Hidden from java.lang.invoke.LambdaForm. >>> >>> What do you think about moving LambdaForm.Hidden to >>> jdk.internal.vm.annotation instead and share among all usages? >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> On 31/10/2018 15:23, dean.long at oracle.com wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8212605 >>>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >>>> >>>> This change implements AccessController.doPrivileged in Java. This >>>> gives a performance improvement while also being useful to Project >>>> Loom by removing the Java --> native --> Java transition.? One >>>> reason doPrivileged has historically been in native is because of >>>> the need to guarantee the cleanup of the privileged context when >>>> doPrivileged returns.? To do that in Java, the information that >>>> AccessController.getContext needs is pushed onto the Java stack as >>>> arguments to a method that getContext will recognize during its >>>> stack walk.? This allows us to remove the native privileged stack >>>> while guaranteeing that the privileged context goes away when the >>>> method returns. >>>> >>>> Tested with tier1-tier3 hotspot and jdk tests and JCK >>>> api/java_security tests.? For the first few rounds of testing, I >>>> kept the old native privileged stack and compared the results of >>>> the old and new implementations for each getContext call, which did >>>> catch some early bugs.? The changes were also examined by internal >>>> security experts and run through additional internal security tests. >>>> >>>> The improvement on this [1] doPrivileged microbenchmark is >>>> approximate 50x. >>>> >>>> There is no attempt to optimize getContext() or security permission >>>> checks in this change, however, this is intended to be a first step >>>> towards other possible improvements, for example those proposed >>>> here [2]. >>>> >>>> dl >>>> >>>> [1] >>>> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >>>> >>>> [2] >>>> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >>>> >>>> >> From igor.ignatyev at oracle.com Thu Nov 1 21:15:55 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 1 Nov 2018 14:15:55 -0700 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags Message-ID: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html > 174 lines changed: 0 ins; 170 del; 4 mod; Hi all, could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html testing: tier1, build all regular platforms [1] https://bugs.openjdk.java.net/browse/JDK-8177708 [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html Thanks, -- Igor From dean.long at oracle.com Thu Nov 1 21:34:33 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 1 Nov 2018 14:34:33 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <363cd5f9-e143-79cc-9698-6c8cb213ef1e@oracle.com> <43be67e3-c6a5-3ad2-d835-e9c32f90374f@oracle.com> Message-ID: <4452ed95-c282-0ba7-340a-4566cfbb0095@oracle.com> On 11/1/18 1:45 PM, Mandy Chung wrote: > > > On 11/1/18 1:18 AM, Vladimir Ivanov wrote: >> >>> I think it's a good idea, but I believe it would require a CSR >>> request. Do you mind if I file a separate issue for >>> jdk.internal.vm.annotation.Hidden? >> >> Sure. >> >> Most of the annotations in jdk.internal.vm.annotation were originally >> located in java.lang.invoke. Not sure CSR will be required in this >> particular case. >> > > @Hidden is internal and no CSR is needed. > > FYI.? JDK-8212620 is the RFE to consider providing a standard > mechanism to hide frames from stack trace. > OK, I already filed JDK-8213234 but I think I should just close it as a duplicate of JDK-8212620. dl > Mandy > >> Best regards, >> Vladimir Ivanov >> >>> On 10/31/18 6:11 PM, Vladimir Ivanov wrote: >>>> Dean, >>>> >>>> src/java.base/share/classes/java/security/AccessController.java: >>>> +??? /** >>>> +???? * Internal marker for hidden implementation frames. >>>> +???? */ >>>> +??? /*non-public*/ >>>> +??? @Target(ElementType.METHOD) >>>> +??? @Retention(RetentionPolicy.RUNTIME) >>>> +??? @interface Hidden { >>>> +??? } >>>> >>>> You declare @Hidden, but then map it to _method_Hidden along with >>>> @Hidden from java.lang.invoke.LambdaForm. >>>> >>>> What do you think about moving LambdaForm.Hidden to >>>> jdk.internal.vm.annotation instead and share among all usages? >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> On 31/10/2018 15:23, dean.long at oracle.com wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8212605 >>>>> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >>>>> >>>>> This change implements AccessController.doPrivileged in Java. This >>>>> gives a performance improvement while also being useful to Project >>>>> Loom by removing the Java --> native --> Java transition.? One >>>>> reason doPrivileged has historically been in native is because of >>>>> the need to guarantee the cleanup of the privileged context when >>>>> doPrivileged returns.? To do that in Java, the information that >>>>> AccessController.getContext needs is pushed onto the Java stack as >>>>> arguments to a method that getContext will recognize during its >>>>> stack walk.? This allows us to remove the native privileged stack >>>>> while guaranteeing that the privileged context goes away when the >>>>> method returns. >>>>> >>>>> Tested with tier1-tier3 hotspot and jdk tests and JCK >>>>> api/java_security tests.? For the first few rounds of testing, I >>>>> kept the old native privileged stack and compared the results of >>>>> the old and new implementations for each getContext call, which >>>>> did catch some early bugs.? The changes were also examined by >>>>> internal security experts and run through additional internal >>>>> security tests. >>>>> >>>>> The improvement on this [1] doPrivileged microbenchmark is >>>>> approximate 50x. >>>>> >>>>> There is no attempt to optimize getContext() or security >>>>> permission checks in this change, however, this is intended to be >>>>> a first step towards other possible improvements, for example >>>>> those proposed here [2]. >>>>> >>>>> dl >>>>> >>>>> [1] >>>>> http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java >>>>> >>>>> [2] >>>>> http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html >>>>> >>>>> >>> > From mandy.chung at oracle.com Thu Nov 1 21:42:45 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 1 Nov 2018 14:42:45 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <4452ed95-c282-0ba7-340a-4566cfbb0095@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <363cd5f9-e143-79cc-9698-6c8cb213ef1e@oracle.com> <43be67e3-c6a5-3ad2-d835-e9c32f90374f@oracle.com> <4452ed95-c282-0ba7-340a-4566cfbb0095@oracle.com> Message-ID: <595b46eb-bd26-7207-77a7-c76c5fc9719f@oracle.com> On 11/1/18 2:34 PM, dean.long at oracle.com wrote: > >> @Hidden is internal and no CSR is needed. >> >> FYI.? JDK-8212620 is the RFE to consider providing a standard >> mechanism to hide frames from stack trace. >> > > OK, I already filed JDK-8213234 but I think I should just close it as > a duplicate of JDK-8212620. > JDK-8212620 likely does not make JDK 12.? I assume Vladimir suggests to do the rename with your fix (or a follow up). Mandy From dean.long at oracle.com Thu Nov 1 21:52:05 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Thu, 1 Nov 2018 14:52:05 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <72357551-9393-d3e5-ced1-98452354ae9b@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <61f7523d-3fce-023b-f806-aa49a02f6d9c@oracle.com> <8c969cca-fab2-4731-3d81-a956c1898677@oracle.com> <8a485f73-e942-12e8-9c7d-0944ff0af2de@oracle.com> <72357551-9393-d3e5-ced1-98452354ae9b@oracle.com> Message-ID: On 11/1/18 12:39 PM, Sean Mullan wrote: >> I also replaced getCallerPD with the faster getProtectionDomain and >> removed a stale comment about impliesCreateAccessControlContext being >> called by the VM. It should be safe to remove now, but I left it in >> to minimize changes. > > I would just remove it, so we don't forget about it later. OK, I removed it: http://cr.openjdk.java.net/~dlong/8212605/webrev.4.incr/ dl From erik.osterlund at oracle.com Thu Nov 1 22:14:27 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 1 Nov 2018 23:14:27 +0100 Subject: RFR: 8212989: Allow CompiledMethod ExceptionCache have unloaded klasses In-Reply-To: References: <5BDAF881.3030905@oracle.com> Message-ID: <604b581e-00f2-24df-b06f-738bdefc593a@oracle.com> Hi Vladimir, Thanks for the quick reply. On 2018-11-01 21:14, Vladimir Kozlov wrote: > Hi Erik, > > It seems these? changed are not based on latest sources. I don't see > call to CodeCache::do_unloading in CodeCache::do_unloading() which is > present in latest code: > > http://hg.openjdk.java.net/jdk/jdk/file/5ea020bcaa0d/src/hotspot/share/code/codeCache.cpp#l683 > Yeah this code is taken from the context of the future. Here is the exact same patch rebased against jdk-jdk: http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01/ And new webrev with your suggestions: http://cr.openjdk.java.net/~eosterlund/8212989/webrev.02/ And incremental of that to the jdk-jdk rebase: http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01_02/ > > May be make some methods private, like > CompiledMethod::exception_cache_acquire(). Sure. Made the accessors protected, as some are used in nmethod too and it seems fine for subclasses to access. > It seems you remove the only case when release_set_exception_cache() is > called. That is true. Removed. > I don't see where purge_exception_caches() is used. It's only called by a future version of ZGC from its new concurrent class unloading feature that is currently baking in the ZGC repository. The release_exception_cache member function deletes entries synchronously if cleanup is triggered in a safepoint (all GCs except ZGC), and otherwise defers deletion to the purge_exception_caches() method, that is expected to happen one global handshake operation after the exception caches have been unlinked by cleanup. > Next sentence seems incomplete: > > +? // unlinked. That is also when the > CodeCache::exception_cache_free_list() Fixed. > Why you use exception_cache_acquire() in clean_exception_cache() and > simple exception_cache() in add_exception_cache_entry()? The reason is that in the context of insertion, you are holding the lock protecting against concurrent insertions. Therefore, entries you load are either entries that were protected by the same lock (other insertions), or entries that are as old or older than entries that have died a whole safepoint operation earlier (due to prepending inserts), which makes its memory state stable. Therefore, no acquire is required when reading the head in the insertion path. Thanks, /Erik > Thanks, > Vladimir > > On 11/1/18 5:58 AM, Erik ?sterlund wrote: >> Hi, >> >> The ExceptionCaches of compiled methods is traditionally pruned in >> safepoints during class unloading. This allows the exception cache to >> have lock-free reads, while performing inserts under a lock, and >> pruning under a safepoint. With concurrent class unloading, the >> exception caches need to be cleaned concurrently. >> >> To retain the lock-free reads that seem good to keep lock-free, a >> lock-free cleaning mechanism was introduced. Only one thread per >> compiled method cleans each exception cache, but that thread races >> with both single writers (under the ExceptionCache_lock) and any >> number of readers. >> >> The head of the exception cache list is concurrently cleaned by both >> inserting threads and cleaning threads. This allows having an >> invariant that no newly prepended entries ever produce next pointers >> to dead exception caches, that are concurrently removed from the list. >> As for the internal next pointers, they are only pruned by the one >> concurrent cleaning thread. This happens concurrent to reads that >> simply skip over dead entries as they have different Klass* pointers >> to the one being searched for. >> >> The single concurrent cleanup thread does not delete things removed >> from the list straight away. Instead, they are placed on a purge list >> that is freed up after a subsequent global handshaking operation. That >> allows ABA-free CAS instructions in the lock-free paths, and allows >> safe concurrent reading of entries in the exception cache list. >> >> Note that we already incorrectly "relied on memory_order_consume" for >> the head pointer. But it used a volatile load. Assuming volatile loads >> retain memory_order_consume semantics is not allowed in HotSpot >> runtime code, so I changed those loads to use acquire accordingly. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212989 >> >> Thanks, >> /Erik From erik.joelsson at oracle.com Thu Nov 1 22:22:58 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 1 Nov 2018 15:22:58 -0700 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> Message-ID: <49fc3851-5b6b-0ca3-bd9d-6dc88237d2ca@oracle.com> Build changes look good. /Erik On 2018-11-01 14:15, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >> 174 lines changed: 0 ins; 170 del; 4 mod; > Hi all, > > could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? > > 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 > webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html > > testing: tier1, build all regular platforms > [1] https://bugs.openjdk.java.net/browse/JDK-8177708 > [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html > [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html > > Thanks, > -- Igor From ecki at zusammenkunft.net Thu Nov 1 04:39:08 2018 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Thu, 1 Nov 2018 04:39:08 +0000 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com>, <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> Message-ID: http://cr.openjdk.java.net/~dlong/8212605/webrev.1/src/java.base/share/classes/java/security/AccessController.java.udiff.html In checkContext should the security manager be null checked first instead of last to optimize for the typical case? (If the side effects in that expression are desired it should be documented) I find the tail call optimization comment in wrapException adds only confusion to an otherwise clear helper. But maybe it?s just me who does not understand it. Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: core-libs-dev im Auftrag von Vladimir Ivanov Gesendet: Donnerstag, November 1, 2018 5:11 AM An: dean.long at oracle.com; security-dev at openjdk.java.net; core-libs-dev Libs; hotspot-dev developers Betreff: Re: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged Dean, src/java.base/share/classes/java/security/AccessController.java: + /** + * Internal marker for hidden implementation frames. + */ + /*non-public*/ + @Target(ElementType.METHOD) + @Retention(RetentionPolicy.RUNTIME) + @interface Hidden { + } You declare @Hidden, but then map it to _method_Hidden along with @Hidden from java.lang.invoke.LambdaForm. What do you think about moving LambdaForm.Hidden to jdk.internal.vm.annotation instead and share among all usages? Best regards, Vladimir Ivanov On 31/10/2018 15:23, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8212605 > http://cr.openjdk.java.net/~dlong/8212605/webrev.1 > > This change implements AccessController.doPrivileged in Java. This > gives a performance improvement while also being useful to Project Loom > by removing the Java --> native --> Java transition. One reason > doPrivileged has historically been in native is because of the need to > guarantee the cleanup of the privileged context when doPrivileged > returns. To do that in Java, the information that > AccessController.getContext needs is pushed onto the Java stack as > arguments to a method that getContext will recognize during its stack > walk. This allows us to remove the native privileged stack while > guaranteeing that the privileged context goes away when the method returns. > > Tested with tier1-tier3 hotspot and jdk tests and JCK api/java_security > tests. For the first few rounds of testing, I kept the old native > privileged stack and compared the results of the old and new > implementations for each getContext call, which did catch some early > bugs. The changes were also examined by internal security experts and > run through additional internal security tests. > > The improvement on this [1] doPrivileged microbenchmark is approximate 50x. > > There is no attempt to optimize getContext() or security permission > checks in this change, however, this is intended to be a first step > towards other possible improvements, for example those proposed here [2]. > > dl > > [1] > http://hg.openjdk.java.net/code-tools/jmh-jdk-microbenchmarks/file/fc4783360f58/src/main/java/org/openjdk/bench/java/security/DoPrivileged.java > > [2] > http://mail.openjdk.java.net/pipermail/security-dev/2017-December/016627.html > > From david.holmes at oracle.com Thu Nov 1 23:16:20 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 2 Nov 2018 09:16:20 +1000 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> Message-ID: Hi Igor, There's no point having empty test_log macros that do nothing. The macro and all uses should just be deleted ... unless you plan on adding some other form of logging for this? Thanks, David On 2/11/2018 7:15 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >> 174 lines changed: 0 ins; 170 del; 4 mod; > > Hi all, > > could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? > > 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 > webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html > > testing: tier1, build all regular platforms > [1] https://bugs.openjdk.java.net/browse/JDK-8177708 > [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html > [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html > > Thanks, > -- Igor > From igor.ignatyev at oracle.com Thu Nov 1 23:23:03 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 1 Nov 2018 16:23:03 -0700 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> Message-ID: <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> Hi David, removing usage of test_log will just mix "unneeded" changes w/ this clean up. as TestReservedSpace_test, TestReserveMemorySpecial_test, TestVirtualSpace_test, and TestMetaspaceUtils_test are to be removed by 8213269[*], I don't think we need to pay much attention to their code. [*] https://bugs.openjdk.java.net/browse/JDK-8213269 -- Igor > On Nov 1, 2018, at 4:16 PM, David Holmes wrote: > > Hi Igor, > > There's no point having empty test_log macros that do nothing. The macro and all uses should just be deleted ... unless you plan on adding some other form of logging for this? > > Thanks, > David > > On 2/11/2018 7:15 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>> 174 lines changed: 0 ins; 170 del; 4 mod; >> Hi all, >> could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? >> 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 >> webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >> testing: tier1, build all regular platforms >> [1] https://bugs.openjdk.java.net/browse/JDK-8177708 >> [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >> [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html >> Thanks, >> -- Igor From igor.ignatyev at oracle.com Thu Nov 1 23:37:28 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 1 Nov 2018 16:37:28 -0700 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> Message-ID: <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> on a second though, removing these macros isn't that big, here is an incremental webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.0-1 . besides removing test_log, it also include fixes in build and doc needed due to rebasing. Erik, could you please re-review build changes? http://cr.openjdk.java.net/~iignatyev//8213058/webrev.01/ is the whole webrev. Thanks, -- Igor > On Nov 1, 2018, at 4:23 PM, Igor Ignatyev wrote: > > Hi David, > > removing usage of test_log will just mix "unneeded" changes w/ this clean up. as TestReservedSpace_test, TestReserveMemorySpecial_test, TestVirtualSpace_test, and TestMetaspaceUtils_test are to be removed by 8213269[*], I don't think we need to pay much attention to their code. > > [*] https://bugs.openjdk.java.net/browse/JDK-8213269 > > -- Igor > >> On Nov 1, 2018, at 4:16 PM, David Holmes wrote: >> >> Hi Igor, >> >> There's no point having empty test_log macros that do nothing. The macro and all uses should just be deleted ... unless you plan on adding some other form of logging for this? >> >> Thanks, >> David >> >> On 2/11/2018 7:15 AM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>> 174 lines changed: 0 ins; 170 del; 4 mod; >>> Hi all, >>> could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? >>> 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 >>> webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>> testing: tier1, build all regular platforms >>> [1] https://bugs.openjdk.java.net/browse/JDK-8177708 >>> [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>> [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html >>> Thanks, >>> -- Igor > From david.holmes at oracle.com Thu Nov 1 23:40:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 2 Nov 2018 09:40:45 +1000 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> Message-ID: <5c8dc3a7-b172-674d-3d1f-c367e5b7b220@oracle.com> On 2/11/2018 9:37 AM, Igor Ignatyev wrote: > on a second though, removing these macros isn't that big, here is an > incremental webrev: > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.0-1 :) Thanks. Looks good! David > . besides > removing test_log, it also include fixes in build and doc needed due to > rebasing. > > Erik, could you please re-review build changes? > > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.01/ > ?is the > whole webrev. > > Thanks, > -- Igor > > >> On Nov 1, 2018, at 4:23 PM, Igor Ignatyev > > wrote: >> >> Hi David, >> >> removing usage of test_log will just mix "unneeded" changes w/ this >> clean up. as TestReservedSpace_test, TestReserveMemorySpecial_test, >> TestVirtualSpace_test, and TestMetaspaceUtils_test are to be removed >> by 8213269[*], I don't think we need to pay much attention to their code. >> >> [*] https://bugs.openjdk.java.net/browse/JDK-8213269 >> >> >> -- Igor >> >>> On Nov 1, 2018, at 4:16 PM, David Holmes >> > wrote: >>> >>> Hi Igor, >>> >>> There's no point having empty test_log macros that do nothing. The >>> macro and all uses should just be deleted ... unless you plan on >>> adding some other form of logging for this? >>> >>> Thanks, >>> David >>> >>> On 2/11/2018 7:15 AM, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>> >>>>> 174 lines changed: 0 ins; 170 del; 4 mod; >>>> Hi all, >>>> could you please review this small clean up which removes >>>> ExecuteInternalVMTests and VerboseInternalVMTests flags and related >>>> make targets and tests? >>>> 8177708[1-3] is to convert the last of internal vm tests, so the >>>> whole InternalVMTests can be removed. >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 >>>> webrev: >>>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>> testing: tier1, build all regular platforms >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8177708 >>>> [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>> [3] >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html >>>> Thanks, >>>> -- Igor >> > From erik.joelsson at oracle.com Fri Nov 2 00:35:00 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 1 Nov 2018 17:35:00 -0700 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> Message-ID: <96892200-6e48-a70f-4494-6be02324cdea@oracle.com> Looks good. /Erik On 2018-11-01 16:37, Igor Ignatyev wrote: > on a second though, removing these macros isn't that big, here is an > incremental webrev: > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.0-1 > . > besides removing test_log, it also include fixes in build and doc > needed due to rebasing. > > Erik, could you please re-review build changes? > > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.01/ > ?is the > whole webrev. > > Thanks, > -- Igor > > >> On Nov 1, 2018, at 4:23 PM, Igor Ignatyev > > wrote: >> >> Hi David, >> >> removing usage of test_log will just mix "unneeded" changes w/ this >> clean up. as TestReservedSpace_test, TestReserveMemorySpecial_test, >> TestVirtualSpace_test, and TestMetaspaceUtils_test are to be removed >> by 8213269[*], I don't think we need to pay much attention to their code. >> >> [*] https://bugs.openjdk.java.net/browse/JDK-8213269 >> >> >> -- Igor >> >>> On Nov 1, 2018, at 4:16 PM, David Holmes >> > wrote: >>> >>> Hi Igor, >>> >>> There's no point having empty test_log macros that do nothing. The >>> macro and all uses should just be deleted ... unless you plan on >>> adding some other form of logging for this? >>> >>> Thanks, >>> David >>> >>> On 2/11/2018 7:15 AM, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>> >>>>> 174 lines changed: 0 ins; 170 del; 4 mod; >>>> Hi all, >>>> could you please review this small clean up which removes >>>> ExecuteInternalVMTests and VerboseInternalVMTests flags and related >>>> make targets and tests? >>>> 8177708[1-3] is to convert the last of internal vm tests, so the >>>> whole InternalVMTests can be removed. >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 >>>> webrev: >>>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>> testing: tier1, build all regular platforms >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8177708 >>>> [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>> [3] >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html >>>> Thanks, >>>> -- Igor >> > From mandy.chung at oracle.com Fri Nov 2 04:16:32 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 1 Nov 2018 21:16:32 -0700 Subject: Review request: JDK-8211122: Reduce the number of internal classes made accessible to jdk.unsupported Message-ID: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> This patch includes the following changes that will reduce the number of internal classes made accessible to jdk.unsupported module via qualified exports. 1. move shared secrets to a new jdk.internal.access package. jdk.internal.misc package has been a dumping ground for various kinds of internal APIs and many of which were moved to an appropriate package. This is a follow-up clean up that moves the shared secrets and JavaXXXAccess interfaces to jdk.internal.access package. 2. add a wrapper class jdk.internal.misc.FileSystemOption to expose sun.nio.fs.ExtendedOptions for com.sun.nio.file to access This eliminates the qualified exports of sun.nio.fs to jdk.unsupported. 3. Refactor the implementation of sun.misc.Unsafe::invokeCleaner to jdk.internal.misc.Unsafe. This eliminates the qualified exports of jdk.internal.ref and sun.nio.ch to jdk.unsupported. Webrev: http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.02 The change is straight forward although quite many files are modified. Majority of the changes is import statement change or test @modules change due to the rename. I ran hs-tier1-3 and jdk-tier1-3 tests. We considered the alternative to define a wrapper class for everything that jdk.unsupported module depends on in a dedicated package e.g. jdk.internal.unsupported.? It would need a wrapper for Unsafe, Signal, SignalHandler and ExtendedOptions.? It means that it would have 3 classes of each - two similar/duplicated implementations (one in sun.misc and one in jdk.internal.unsupported) and one in jdk.internal.misc ([1] is the prototype).? Only a limited number of files are touched but I think the proposed approach is cleaner. Thanks Mandy [1] http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.01/ From ioi.lam at oracle.com Fri Nov 2 04:47:30 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 1 Nov 2018 21:47:30 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: References: Message-ID: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> Hi, I've revised the webrev to get to the the root of the problem -- we shouldn't allocate metaspace objects from within the VM Thread! http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ The original bug that caused the "!(DumpSharedSpaces && THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was ??? https://bugs.openjdk.java.net/browse/JDK-8196626 ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock (not allowed to block on)" The fix in JDK-8196626 is just a band-aid solution -- avoiding the GC. The proper fix is not to do any allocation inside the VM thread. So instead of calling finalize_verification_constraints, which allocates MetaspaceObjs, very late in the CDS dumping process, now we call be as soon as a class has been verified. Thanks - Ioi On 11/1/18 11:07 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8213250 > http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ > > > When Metaspace::allocate() fails, usually we first try to GC to see if > some MetaspaceObjs can be freed. This is an optimization to avoid > prematurely > expanding the metaspace. > > However, GC cannot run in the last phase of CDS which runs inside the > VM thread. > The old code just aborts the CDS dump. The fix is to unconditionally > expand the > metaspace without doing a GC. > > (During GC dump time, no classes are unloaded, so even if we force a > GC, it would > not free up any metaspace anyway). > > Testing: > > Before the fix, the test case (jruby) would fail to dump once a couple > of times. Now > it has dumped hundreds of times without failure. > > I am also running hs-tier 1/2 > > Thanks > - Ioi From magnus.ihse.bursie at oracle.com Fri Nov 2 08:25:25 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 2 Nov 2018 09:25:25 +0100 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> Message-ID: <26c12d82-c751-ca46-ff44-2eb09f406c9d@oracle.com> Hi Igor, On 2018-11-01 22:15, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html What release are you targeting? This does not look like a patch that can be applied to jdk/jdk, at least not after JDK-8210958. When removing the "hotspot-internal" test, please also update make/common/FindTests.gmk, so it does not get added to ALL_NAMED_TESTS. Also, please update the test documentation at doc/testing.md and testing.html. (Update the markdown file, then you can regenerate the html file using "make update-build-docs".) /Magnus >> 174 lines changed: 0 ins; 170 del; 4 mod; > Hi all, > > could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? > > 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 > webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html > > testing: tier1, build all regular platforms > [1] https://bugs.openjdk.java.net/browse/JDK-8177708 > [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html > [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html > > Thanks, > -- Igor From magnus.ihse.bursie at oracle.com Fri Nov 2 08:27:42 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 2 Nov 2018 09:27:42 +0100 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> Message-ID: On 2018-11-02 00:37, Igor Ignatyev wrote: > on a second though, removing these macros isn't that big, here is an incremental webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.0-1 . besides removing test_log, it also include fixes in build and doc needed due to rebasing. > > Erik, could you please re-review build changes? > > http://cr.openjdk.java.net/~iignatyev//8213058/webrev.01/ is the whole webrev. Sorry, I missed your updated webrev. You still need to fix FindTests.gmk, though. /Magnus > > Thanks, > -- Igor > > >> On Nov 1, 2018, at 4:23 PM, Igor Ignatyev wrote: >> >> Hi David, >> >> removing usage of test_log will just mix "unneeded" changes w/ this clean up. as TestReservedSpace_test, TestReserveMemorySpecial_test, TestVirtualSpace_test, and TestMetaspaceUtils_test are to be removed by 8213269[*], I don't think we need to pay much attention to their code. >> >> [*] https://bugs.openjdk.java.net/browse/JDK-8213269 >> >> -- Igor >> >>> On Nov 1, 2018, at 4:16 PM, David Holmes wrote: >>> >>> Hi Igor, >>> >>> There's no point having empty test_log macros that do nothing. The macro and all uses should just be deleted ... unless you plan on adding some other form of logging for this? >>> >>> Thanks, >>> David >>> >>> On 2/11/2018 7:15 AM, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>>> 174 lines changed: 0 ins; 170 del; 4 mod; >>>> Hi all, >>>> could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? >>>> 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>> testing: tier1, build all regular platforms >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8177708 >>>> [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>> [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html >>>> Thanks, >>>> -- Igor From shade at redhat.com Fri Nov 2 09:17:21 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 2 Nov 2018 10:17:21 +0100 Subject: RFR: 6735527: Bitmap - speed up searches In-Reply-To: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> References: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> Message-ID: <1f7893b7-3362-e25d-23fd-e708718c24d8@redhat.com> On 10/27/2018 02:18 AM, Kim Barrett wrote: > Webrev: > http://cr.openjdk.java.net/~kbarrett/6735527/open.00/ I like this patch. Shenandoah concurrent phase times have improved a little bit with it, especially when the heap is very sparse. -Aleksey From daniel.fuchs at oracle.com Fri Nov 2 10:18:18 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 2 Nov 2018 10:18:18 +0000 Subject: Review request: JDK-8211122: Reduce the number of internal classes made accessible to jdk.unsupported In-Reply-To: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> References: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> Message-ID: <050a45ec-5074-477f-b3c4-b6e1f1814444@oracle.com> Hi Mandy, I won't comment on the approach, though [1] looks more cryptic to me as it includes suspicious native code changes. I skimmed through the patch file at http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.02/open.patch and haven't noticed anything wrong at first sight. The changes to java.util.logging & associated tests look fine. best regards, -- daniel On 02/11/2018 04:16, Mandy Chung wrote: > This patch includes the following changes that will reduce the > number of internal classes made accessible to jdk.unsupported > module via qualified exports. > > 1. move shared secrets to a new jdk.internal.access package. > > jdk.internal.misc package has been a dumping ground for various > kinds of internal APIs and many of which were moved to an appropriate > package. This is a follow-up clean up that moves the shared secrets > and JavaXXXAccess interfaces to jdk.internal.access package. > > 2. add a wrapper class jdk.internal.misc.FileSystemOption to expose > sun.nio.fs.ExtendedOptions for com.sun.nio.file to access > > This eliminates the qualified exports of sun.nio.fs to jdk.unsupported. > > 3. Refactor the implementation of sun.misc.Unsafe::invokeCleaner > to jdk.internal.misc.Unsafe. > > This eliminates the qualified exports of jdk.internal.ref and > sun.nio.ch to jdk.unsupported. > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.02 > > The change is straight forward although quite many files are modified. > Majority of the changes is import statement change or test @modules > change due to the rename. > > I ran hs-tier1-3 and jdk-tier1-3 tests. > > We considered the alternative to define a wrapper class for everything > that jdk.unsupported module depends on in a dedicated package > e.g. jdk.internal.unsupported.? It would need a wrapper for Unsafe, > Signal, SignalHandler and ExtendedOptions.? It means that it would > have 3 classes of each - two similar/duplicated implementations > (one in sun.misc and one in jdk.internal.unsupported) and one > in jdk.internal.misc ([1] is the prototype).? Only a limited number > of files are touched but I think the proposed approach is cleaner. > > Thanks > Mandy > [1] http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.01/ From erik.osterlund at oracle.com Fri Nov 2 11:51:00 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 2 Nov 2018 12:51:00 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> Message-ID: <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> Hi Roman, On 2018-11-01 17:58, Roman Kennke wrote: > Hi Erik, > >> Would you mind explaining how you need to override this and why? I'm >> afraid I did not quite get it from your description in the RFC (which is >> also why I didn't come up with a solution either). >> >> Am I correct that you need to return false if the passed in offset is >> the cell header offset -8, or is there anything else at all required to >> that logic? > > No, it's really just that. Plus take care of it in the case of combined > narrow-oops-offset plus -8. >> You mentioned something about the high order byte being >> masked on AArch64, but didn't quite connect the dot to how that is >> related to this code. Is it? > > Yes, we also need to mask the high order byte in AArch64 because of the > way addressing works on that platform, and because -8 flips on those > negative bits. Okay. Looking at https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html that you posted before, assuming this is how it will be extended. So in the case with a heap base, the resulting address will never be negative, and you know it will trap (presuming there is an mprotected page at offset -8 from the heap base pointer. So it seems in this case, the check could have just been offset == -8. So let's say we access null -8 without a heap base then. The address becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address masking of the high order byte. I presume that is what the AArch64 code is dealing with. But I don't quite understand why. Addresses starting with 0x00FF are not valid in user space so we never need to worry about any memory being mapped in there accidentally. So I'm not sure what this code protects against. So that takes me back to my original intuition: isn't this the same as checking at the very top: if (offset == BrooksPointer::byte_offset()) return false; If it really is as simple as just that, and I did not miss something not obvious, then perhaps we would be better off abstracting a cell header check here, as opposed to wrapping the whole thing in a virtual member function. Especially since there are now a number of these occurrences where some basic knowledge about cell header size that is 0 for all GCs except Shenandoah where it is 8, leaves us wrapping a whole bunch of stuff behind a GC interface, only to handle that one difference. While I don't know exactly what this cell abstraction would reasonably look like in its full shape, perhaps we could start with something simple like a virtual member function size_t cell_header_size() on CollectedHeap with a suitable comment explaining that a cell header is a GC thing we sometimes put above the object header. And then see if there are more related functions that lead us to a helper class for cells or something like that. I'm open to suggestions of course. Thought I'd check with you if this sounds sane to you. Of course if my assumption is wrong that this check could be written much simpler than it looks like, then I suppose I need to give up on that idea. It all boils down to that really. Thanks, /Erik > Also, the mach5 job came back with FAILED (see below). Can somebody with > access check and see what's up? > > Thanks, > Roman > > > Build Details: 2018-11-01-1146402.roman.source > 0 Failed Tests > Mach5 Tasks Results Summary > > EXECUTED_WITH_FAILURE: 6 > KILLED: 0 > UNABLE_TO_RUN: 21 > PASSED: 48 > FAILED: 0 > NA: 0 > Build > > 6 Executed with failure > solaris-sparcv9-solaris-sparcv9-build-8 error while > building, return value: 2 > solaris-sparcv9-debug-solaris-sparcv9-build-9 error while > building, return value: 2 > windows-x64-windows-x64-build-10 error while building, > return value: 2 > windows-x64-debug-windows-x64-build-11 error while building, > return value: 2 > windows-x64-open-windows-x64-build-12 error while building, > return value: 2 > windows-x64-open-debug-windows-x64-build-13 error while > building, return value: 2 > 2 Not run > solaris-sparcv9-install-solaris-sparcv9-build-16 Dependency > task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 > windows-x64-install-windows-x64-build-17 Dependency task > failed: YJftjiBUYc > > Test > > 19 Not run > > tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 > Dependency task failed: > mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 > > tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 > Dependency task failed: > mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 > > tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 > Dependency task failed: YJftjiBUYc > > tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 > Dependency task failed: YJftjiBVYc > > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 Dependency > task failed: YJftjiBVYc > > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 Dependency > task failed: YJftjiBVYc > > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 Dependency > task failed: YJftjiBVYc > > tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 > Dependency task failed: YJftjiBVYc > > tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 > Dependency task failed: YJftjiBVYc > > tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 > Dependency task failed: YJftjiBVYc > See all 19... > > > >> Thanks, >> /Erik >> >> On 2018-11-01 12:20, Roman Kennke wrote: >>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>> >>> Erik: ok for you too? >>> >>> Thanks, >>> Roman >>> >>> >>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >>>>> >>>>> Hi Erik, >>>>> >>>>> right. Fixed this, and what what Kim mentioned plus a missing include: >>>>> >>>>> Incremental: >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>> Full: >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>> >>>>> Ok now? >>>> Looks good. >>>> >> > From mandy.chung at oracle.com Fri Nov 2 15:44:20 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 2 Nov 2018 08:44:20 -0700 Subject: Review request: JDK-8211122: Reduce the number of internal classes made accessible to jdk.unsupported In-Reply-To: <050a45ec-5074-477f-b3c4-b6e1f1814444@oracle.com> References: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> <050a45ec-5074-477f-b3c4-b6e1f1814444@oracle.com> Message-ID: On 11/2/18 3:18 AM, Daniel Fuchs wrote: > Hi Mandy, > > I won't comment on the approach, though [1] looks more > cryptic to me as it includes suspicious native code > changes. > The native code change should have been separated from this fix.? It attempts to remove jdk.internal.misc.Unsafe::getLoadAvg.? I cleaned up and update webrev.01 in place (please reload to check out). > I skimmed through the patch file at > http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.02/open.patch > > and haven't noticed anything wrong at first sight. > > The changes to java.util.logging & associated tests look fine. > Thanks Mandy > best regards, > > -- daniel > > On 02/11/2018 04:16, Mandy Chung wrote: >> This patch includes the following changes that will reduce the >> number of internal classes made accessible to jdk.unsupported >> module via qualified exports. >> >> 1. move shared secrets to a new jdk.internal.access package. >> >> jdk.internal.misc package has been a dumping ground for various >> kinds of internal APIs and many of which were moved to an appropriate >> package. This is a follow-up clean up that moves the shared secrets >> and JavaXXXAccess interfaces to jdk.internal.access package. >> >> 2. add a wrapper class jdk.internal.misc.FileSystemOption to expose >> sun.nio.fs.ExtendedOptions for com.sun.nio.file to access >> >> This eliminates the qualified exports of sun.nio.fs to jdk.unsupported. >> >> 3. Refactor the implementation of sun.misc.Unsafe::invokeCleaner >> to jdk.internal.misc.Unsafe. >> >> This eliminates the qualified exports of jdk.internal.ref and >> sun.nio.ch to jdk.unsupported. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.02 >> >> The change is straight forward although quite many files are modified. >> Majority of the changes is import statement change or test @modules >> change due to the rename. >> >> I ran hs-tier1-3 and jdk-tier1-3 tests. >> >> We considered the alternative to define a wrapper class for everything >> that jdk.unsupported module depends on in a dedicated package >> e.g. jdk.internal.unsupported.? It would need a wrapper for Unsafe, >> Signal, SignalHandler and ExtendedOptions.? It means that it would >> have 3 classes of each - two similar/duplicated implementations >> (one in sun.misc and one in jdk.internal.unsupported) and one >> in jdk.internal.misc ([1] is the prototype).? Only a limited number >> of files are touched but I think the proposed approach is cleaner. >> >> Thanks >> Mandy >> [1] http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.01/ > From Alan.Bateman at oracle.com Fri Nov 2 16:16:07 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 2 Nov 2018 16:16:07 +0000 Subject: Review request: JDK-8211122: Reduce the number of internal classes made accessible to jdk.unsupported In-Reply-To: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> References: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> Message-ID: On 02/11/2018 04:16, Mandy Chung wrote: > This patch includes the following changes that will reduce the > number of internal classes made accessible to jdk.unsupported > module via qualified exports. > > 1. move shared secrets to a new jdk.internal.access package. > > jdk.internal.misc package has been a dumping ground for various > kinds of internal APIs and many of which were moved to an appropriate > package. This is a follow-up clean up that moves the shared secrets > and JavaXXXAccess interfaces to jdk.internal.access package. > > 2. add a wrapper class jdk.internal.misc.FileSystemOption to expose > sun.nio.fs.ExtendedOptions for com.sun.nio.file to access > > This eliminates the qualified exports of sun.nio.fs to jdk.unsupported. > > 3. Refactor the implementation of sun.misc.Unsafe::invokeCleaner > to jdk.internal.misc.Unsafe. > > This eliminates the qualified exports of jdk.internal.ref and > sun.nio.ch to jdk.unsupported. I skimmed through the changes and I think they are okay. A minor comment on SharedSecrets is that it might be a bit clearer import Unsafe. Also in FileSystemOption there is a typo in the javadoc for the register method, I assume you mean "register as an OpenOption". -Alan From mandy.chung at oracle.com Fri Nov 2 16:23:47 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 2 Nov 2018 09:23:47 -0700 Subject: Review request: JDK-8211122: Reduce the number of internal classes made accessible to jdk.unsupported In-Reply-To: References: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> Message-ID: On 11/2/18 9:16 AM, Alan Bateman wrote: > On 02/11/2018 04:16, Mandy Chung wrote: >> This patch includes the following changes that will reduce the >> number of internal classes made accessible to jdk.unsupported >> module via qualified exports. >> >> 1. move shared secrets to a new jdk.internal.access package. >> >> jdk.internal.misc package has been a dumping ground for various >> kinds of internal APIs and many of which were moved to an appropriate >> package. This is a follow-up clean up that moves the shared secrets >> and JavaXXXAccess interfaces to jdk.internal.access package. >> >> 2. add a wrapper class jdk.internal.misc.FileSystemOption to expose >> sun.nio.fs.ExtendedOptions for com.sun.nio.file to access >> >> This eliminates the qualified exports of sun.nio.fs to jdk.unsupported. >> >> 3. Refactor the implementation of sun.misc.Unsafe::invokeCleaner >> to jdk.internal.misc.Unsafe. >> >> This eliminates the qualified exports of jdk.internal.ref and >> sun.nio.ch to jdk.unsupported. > I skimmed through the changes and I think they are okay. > > A minor comment on SharedSecrets is that it might be a bit clearer > import Unsafe. Also in FileSystemOption there is a typo in the javadoc > for the register method, I assume you mean "register as an OpenOption". Fixed in my local repo. Thanks for the review. Mandy From igor.ignatyev at oracle.com Fri Nov 2 17:03:45 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 2 Nov 2018 10:03:45 -0700 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> Message-ID: <75B5FC75-3EF8-4ABE-9901-3E7B487D4DB5@oracle.com> Hi Magnus, > You still need to fix FindTests.gmk, though. thanks for spotting that. is [*] enough, or did I miss something else? -- Igor [*] > diff -r c6d128f60997 make/common/FindTests.gmk > --- a/make/common/FindTests.gmk Thu Nov 01 16:33:43 2018 -0700 > +++ b/make/common/FindTests.gmk Fri Nov 02 10:02:22 2018 -0700 > @@ -79,7 +79,7 @@ > ALL_NAMED_TESTS += $(addprefix make-, $(MAKE_TEST_TARGETS)) > > # Add special tests > -ALL_NAMED_TESTS += hotspot-internal failure-handler make > +ALL_NAMED_TESTS += failure-handler make > > ################################################################################ > On Nov 2, 2018, at 1:27 AM, Magnus Ihse Bursie wrote: > > > On 2018-11-02 00:37, Igor Ignatyev wrote: >> on a second though, removing these macros isn't that big, here is an incremental webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.0-1 . besides removing test_log, it also include fixes in build and doc needed due to rebasing. >> >> Erik, could you please re-review build changes? >> >> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.01/ is the whole webrev. > > Sorry, I missed your updated webrev. You still need to fix FindTests.gmk, though. > > /Magnus > > > >> >> Thanks, >> -- Igor >> >> >>> On Nov 1, 2018, at 4:23 PM, Igor Ignatyev wrote: >>> >>> Hi David, >>> >>> removing usage of test_log will just mix "unneeded" changes w/ this clean up. as TestReservedSpace_test, TestReserveMemorySpecial_test, TestVirtualSpace_test, and TestMetaspaceUtils_test are to be removed by 8213269[*], I don't think we need to pay much attention to their code. >>> >>> [*] https://bugs.openjdk.java.net/browse/JDK-8213269 >>> >>> -- Igor >>> >>>> On Nov 1, 2018, at 4:16 PM, David Holmes wrote: >>>> >>>> Hi Igor, >>>> >>>> There's no point having empty test_log macros that do nothing. The macro and all uses should just be deleted ... unless you plan on adding some other form of logging for this? >>>> >>>> Thanks, >>>> David >>>> >>>> On 2/11/2018 7:15 AM, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>>>> 174 lines changed: 0 ins; 170 del; 4 mod; >>>>> Hi all, >>>>> could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? >>>>> 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 >>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>>> testing: tier1, build all regular platforms >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8177708 >>>>> [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>>> [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html >>>>> Thanks, >>>>> -- Igor > From magnus.ihse.bursie at oracle.com Fri Nov 2 18:36:54 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 2 Nov 2018 19:36:54 +0100 Subject: RFR(S): remove ExecuteInternalVMTests and VerboseInternalVMTests flags In-Reply-To: <75B5FC75-3EF8-4ABE-9901-3E7B487D4DB5@oracle.com> References: <3B4A0F91-F037-4674-B05B-F9EFA332F05C@oracle.com> <2EEDEA11-CA41-4A6B-9B2F-049317A44142@oracle.com> <7BBE4756-1DCE-485A-AA7B-4111615A6407@oracle.com> <75B5FC75-3EF8-4ABE-9901-3E7B487D4DB5@oracle.com> Message-ID: <7BF97259-A0F4-4A6A-90F7-BF13E74DC01B@oracle.com> > 2 nov. 2018 kl. 18:03 skrev Igor Ignatyev : > > Hi Magnus, > >> You still need to fix FindTests.gmk, though. > thanks for spotting that. is [*] enough, or did I miss something else? This looks enough. Thanks! /Magnus > > -- Igor > > [*] >> diff -r c6d128f60997 make/common/FindTests.gmk >> --- a/make/common/FindTests.gmk Thu Nov 01 16:33:43 2018 -0700 >> +++ b/make/common/FindTests.gmk Fri Nov 02 10:02:22 2018 -0700 >> @@ -79,7 +79,7 @@ >> ALL_NAMED_TESTS += $(addprefix make-, $(MAKE_TEST_TARGETS)) >> >> # Add special tests >> -ALL_NAMED_TESTS += hotspot-internal failure-handler make >> +ALL_NAMED_TESTS += failure-handler make >> >> ################################################################################ > > >> On Nov 2, 2018, at 1:27 AM, Magnus Ihse Bursie wrote: >> >> >>> On 2018-11-02 00:37, Igor Ignatyev wrote: >>> on a second though, removing these macros isn't that big, here is an incremental webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.0-1 . besides removing test_log, it also include fixes in build and doc needed due to rebasing. >>> >>> Erik, could you please re-review build changes? >>> >>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.01/ is the whole webrev. >> >> Sorry, I missed your updated webrev. You still need to fix FindTests.gmk, though. >> >> /Magnus >> >> >> >>> >>> Thanks, >>> -- Igor >>> >>> >>>> On Nov 1, 2018, at 4:23 PM, Igor Ignatyev wrote: >>>> >>>> Hi David, >>>> >>>> removing usage of test_log will just mix "unneeded" changes w/ this clean up. as TestReservedSpace_test, TestReserveMemorySpecial_test, TestVirtualSpace_test, and TestMetaspaceUtils_test are to be removed by 8213269[*], I don't think we need to pay much attention to their code. >>>> >>>> [*] https://bugs.openjdk.java.net/browse/JDK-8213269 >>>> >>>> -- Igor >>>> >>>>> On Nov 1, 2018, at 4:16 PM, David Holmes wrote: >>>>> >>>>> Hi Igor, >>>>> >>>>> There's no point having empty test_log macros that do nothing. The macro and all uses should just be deleted ... unless you plan on adding some other form of logging for this? >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> On 2/11/2018 7:15 AM, Igor Ignatyev wrote: >>>>>> http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>>>>> 174 lines changed: 0 ins; 170 del; 4 mod; >>>>>> Hi all, >>>>>> could you please review this small clean up which removes ExecuteInternalVMTests and VerboseInternalVMTests flags and related make targets and tests? >>>>>> 8177708[1-3] is to convert the last of internal vm tests, so the whole InternalVMTests can be removed. >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213058 >>>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8213058/webrev.00/index.html >>>>>> testing: tier1, build all regular platforms >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8177708 >>>>>> [2] http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>>>> [3] http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-October/030633.html >>>>>> Thanks, >>>>>> -- Igor >> > From calvin.cheung at oracle.com Fri Nov 2 19:30:34 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Fri, 02 Nov 2018 12:30:34 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> Message-ID: <5BDCA5DA.5000800@oracle.com> Hi Ioi, The revised webrev looks good. The following is removed from systemDictionaryShared.cpp: 831 log_info(cds, verification)("Recorded verification constraints for %d classes = %d bytes (avg = %.2f bytes) ", count, bytes, avg); Please make sure no test is expecting the above output. Line 790 in the same file is a bit too long, consider break it into 2 lines. thanks, Calvin On 11/1/18, 9:47 PM, Ioi Lam wrote: > Hi, > > I've revised the webrev to get to the the root of the problem -- we > shouldn't allocate metaspace objects from within the VM Thread! > > http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ > > > The original bug that caused the "!(DumpSharedSpaces && > THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was > > https://bugs.openjdk.java.net/browse/JDK-8196626 > UseAppCDS.java crashes with "VM thread using lock Heap_lock (not > allowed to block on)" > > The fix in JDK-8196626 is just a band-aid solution -- avoiding the GC. > The proper fix is not to do any allocation inside the VM thread. So > instead of calling finalize_verification_constraints, which allocates > MetaspaceObjs, very late in the CDS dumping process, now we call be as > soon as a class has been verified. > > Thanks > - Ioi > > > > > > On 11/1/18 11:07 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8213250 >> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >> >> >> When Metaspace::allocate() fails, usually we first try to GC to see if >> some MetaspaceObjs can be freed. This is an optimization to avoid >> prematurely >> expanding the metaspace. >> >> However, GC cannot run in the last phase of CDS which runs inside the >> VM thread. >> The old code just aborts the CDS dump. The fix is to unconditionally >> expand the >> metaspace without doing a GC. >> >> (During GC dump time, no classes are unloaded, so even if we force a >> GC, it would >> not free up any metaspace anyway). >> >> Testing: >> >> Before the fix, the test case (jruby) would fail to dump once a >> couple of times. Now >> it has dumped hundreds of times without failure. >> >> I am also running hs-tier 1/2 >> >> Thanks >> - Ioi > From thomas.stuefe at gmail.com Fri Nov 2 19:36:39 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 2 Nov 2018 20:36:39 +0100 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> Message-ID: Hi Ioi, this not a Review, since I do not know the code well enough. But while looking at your change, I came across Metaspace::freeze() and saw that we do "assert_not_frozen()" on allocation paths: ClassMetaSpace::allocate and ::expand_and_allocate (redundant?) and the atypical deallocation path (::deallocate()). But not on the normal deallocation path when the CMS just dies because its class is unloaded (~ClassLoaderMetaspace()). Is this by design or an oversight? Best Regards, Thomas On Fri, Nov 2, 2018 at 5:47 AM Ioi Lam wrote: > > Hi, > > I've revised the webrev to get to the the root of the problem -- we > shouldn't allocate metaspace objects from within the VM Thread! > > http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ > > The original bug that caused the "!(DumpSharedSpaces && > THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was > > https://bugs.openjdk.java.net/browse/JDK-8196626 > UseAppCDS.java crashes with "VM thread using lock Heap_lock (not > allowed to block on)" > > The fix in JDK-8196626 is just a band-aid solution -- avoiding the GC. > The proper fix is not to do any allocation inside the VM thread. So > instead of calling finalize_verification_constraints, which allocates > MetaspaceObjs, very late in the CDS dumping process, now we call be as > soon as a class has been verified. > > Thanks > - Ioi > > > > > > On 11/1/18 11:07 AM, Ioi Lam wrote: > > https://bugs.openjdk.java.net/browse/JDK-8213250 > > http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ > > > > > > When Metaspace::allocate() fails, usually we first try to GC to see if > > some MetaspaceObjs can be freed. This is an optimization to avoid > > prematurely > > expanding the metaspace. > > > > However, GC cannot run in the last phase of CDS which runs inside the > > VM thread. > > The old code just aborts the CDS dump. The fix is to unconditionally > > expand the > > metaspace without doing a GC. > > > > (During GC dump time, no classes are unloaded, so even if we force a > > GC, it would > > not free up any metaspace anyway). > > > > Testing: > > > > Before the fix, the test case (jruby) would fail to dump once a couple > > of times. Now > > it has dumped hundreds of times without failure. > > > > I am also running hs-tier 1/2 > > > > Thanks > > - Ioi > From jiangli.zhou at oracle.com Fri Nov 2 20:24:31 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 2 Nov 2018 13:24:31 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> Message-ID: <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> Hi Ioi, - src/hotspot/share/classfile/verifier.cpp You moved SystemDictionaryShared::finalize_verification_constraints(_klass) to ClassVerifier::verify_class(). We should be very careful in this area. Have you tested with cases where classes fail verification? I haven't ran your code yet, we should avoid copying the constraints for classes that fails verification at dump time. I also need to think through any potential impact on dynamic archiving (possibly none). - src/hotspot/share/memory/metaspace.cpp -??? if (is_init_completed() && !(DumpSharedSpaces && THREAD->is_VM_thread())) { +??? if (is_init_completed()) { Please also add an assert to make sure we don't allocate metaspace objects from the VM_thread at dump time now. Thanks, Jiangli On 11/1/18 9:47 PM, Ioi Lam wrote: > Hi, > > I've revised the webrev to get to the the root of the problem -- we > shouldn't allocate metaspace objects from within the VM Thread! > > http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ > > > The original bug that caused the "!(DumpSharedSpaces && > THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was > > ??? https://bugs.openjdk.java.net/browse/JDK-8196626 > ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock (not > allowed to block on)" > > The fix in JDK-8196626 is just a band-aid solution -- avoiding the GC. > The proper fix is not to do any allocation inside the VM thread. So > instead of calling finalize_verification_constraints, which allocates > MetaspaceObjs, very late in the CDS dumping process, now we call be as > soon as a class has been verified. > > Thanks > - Ioi > > > > > > On 11/1/18 11:07 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8213250 >> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >> >> >> When Metaspace::allocate() fails, usually we first try to GC to see if >> some MetaspaceObjs can be freed. This is an optimization to avoid >> prematurely >> expanding the metaspace. >> >> However, GC cannot run in the last phase of CDS which runs inside the >> VM thread. >> The old code just aborts the CDS dump. The fix is to unconditionally >> expand the >> metaspace without doing a GC. >> >> (During GC dump time, no classes are unloaded, so even if we force a >> GC, it would >> not free up any metaspace anyway). >> >> Testing: >> >> Before the fix, the test case (jruby) would fail to dump once a >> couple of times. Now >> it has dumped hundreds of times without failure. >> >> I am also running hs-tier 1/2 >> >> Thanks >> - Ioi > From igor.ignatyev at oracle.com Fri Nov 2 20:28:22 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 2 Nov 2018 13:28:22 -0700 Subject: RFR(XS) : 8177708 : Convert TestReserveMemorySpecial_test to Gtest In-Reply-To: References: <9BD612CF-150E-4D4F-A15B-93A4154A921A@oracle.com> <102A0AFA-26D2-4D2E-A6DA-4A54A79B6FF9@oracle.com> <31A74048-B1BB-4C5B-ADBE-C9C3724BF948@oracle.com> <095D5F1E-2CB6-4EAB-9839-16C72E4C4CC3@oracle.com> Message-ID: Hi Gerard, thanks for your review! All, can I get a second review from a Reviewer? -- Igor > On Nov 2, 2018, at 11:50 AM, Gerard Ziemski wrote: > > hi Igor, > > Thanks for the explanation, looks OK. > > > cheers > >> On Nov 1, 2018, at 2:10 PM, Igor Ignatyev wrote: >> >> Gerard, >> >> #1. this assert is kinda contract of the function. test_reserve_memory_special_shm is an aux test function which works iif UseSHM is true. invoking it when UseSHM is false is a test error, and I'd prefer not to hide such errors and get a test error. 'if (!$X) { return }' construction is used only in actual tests and is current workaround to express things similar to @requiers in jtreg. >> >> #2. I did the testing appropriated for the made changes: all gtest tests (including new ones) on all "regular" platforms w/ precompiled header enabled and disabled. I've also run mach5 tier1 job just in case, running any other tiers won't provide us w/ any extra coverage. >> >> I have to say though, as these tests depend on host configuration (thru vm flags values), getting passed status doesn't mean much as all the test tasks could have ended up on hosts which don't support large pages. >> >> Thanks, >> -- Igor >> >>> On Nov 1, 2018, at 10:58 AM, Gerard Ziemski wrote: >>> >>> hi Igor, >>> >>> Nice cleanup. >>> >>> I tried to follow the original code and the changes/cleanups you did, and it seems OK to me, even if the new code seems a bit less verbal. I do have 2 questions though: >>> >>> #1 In test/hotspot/gtest/runtime/test_os_linux.cpp we have: >>> >>> 83 static void test_reserve_memory_special_shm(size_t size, size_t alignment) { >>> 84 ASSERT_TRUE(UseSHM) << "must be used only when UseSHM is true"; >>> >>> But in src/hotspot/os/linux/os_linux.cpp we have: >>> >>> 6110 static void test_reserve_memory_special_shm(size_t size, size_t alignment) { >>> 6111 if (!UseSHM) { >>> 6112 return; >>> 6113 } >>> >>> Why do we bother with ?ASSERT_TRUE? here, when the original code didn?t, and when other checks for "UseHugeTLBFS" simply silently return? I?d prefer we either use ASSERT_TRUE for all the flags here (if it doesn?t hurt the test), or leave it as in the original code. >>> >>> #2 Did you run any Mach5 tiers to test this change? >>> >>> >>> cheers >>> >>>> On Nov 1, 2018, at 12:11 PM, Igor Ignatyev wrote: >>>> >>>> Hi Gerard, >>>> >>>> #1. As I mentioned earlier and you can see from your grep results, this test is still called WhiteBox::runMemoryUnitTests which is used in runtime/memory/RunUnitTestsConcurrently to run this and a few other tests multiple times in several concurrent threads. runtime/memory/RunUnitTestsConcurrently will be converted to gtest separately, and this conversion will also include removing of all these tests. >>>> >>>> #2. only linux and windows version of TestReserveMemorySpecial_test aren't trivial, the rest are just empty. >>>> >>>> #3. semantically linux gtest is doing the same as TestReserveMemorySpecial in open/src/hotspot/os/linux/os_linux.cpp, and windows gtest -- open/src/hotspot/os/linux/os_windows.cpp. yes, there are some differences caused by difference in "frameworks" and/or aimed to improve the tests, e.g. changes in how tests are split, error/trace message are printed, etc. >>>> >>>> Thanks, >>>> -- Igor >>>> >>>>> On Nov 1, 2018, at 8:44 AM, Gerard Ziemski wrote: >>>>> >>>>> hi Igor, >>>>> >>>>> I took a look at the webrev, but I think I?m missing the big picture. If I grep for TestReserveMemorySpecial I get: >>>>> >>>>> # grep -rn TestReserveMemorySpecial >>>>> open/src/hotspot/os/bsd/os_bsd.cpp:3778:void TestReserveMemorySpecial_test() { >>>>> open/src/hotspot/os/linux/os_linux.cpp:5967:class TestReserveMemorySpecial : AllStatic { >>>>> open/src/hotspot/os/linux/os_linux.cpp:6146:void TestReserveMemorySpecial_test() { >>>>> open/src/hotspot/os/linux/os_linux.cpp:6147: TestReserveMemorySpecial::test(); >>>>> open/src/hotspot/os/linux/os_linux.hpp:36: friend class TestReserveMemorySpecial; >>>>> open/src/hotspot/os/windows/os_windows.cpp:5561:void TestReserveMemorySpecial_test() { >>>>> open/src/hotspot/os/solaris/os_solaris.cpp:5405:void TestReserveMemorySpecial_test() { >>>>> open/src/hotspot/os/aix/os_aix.cpp:4321:void TestReserveMemorySpecial_test() { >>>>> open/src/hotspot/share/prims/whitebox.cpp:231:void TestReserveMemorySpecial_test(); >>>>> open/src/hotspot/share/prims/whitebox.cpp:239: TestReserveMemorySpecial_test(); >>>>> open/src/hotspot/share/utilities/internalVMTests.cpp:45: run_unit_test(TestReserveMemorySpecial_test); >>>>> >>>>> #1 Wouldn?t you want to delete TestReserveMemorySpecial class from the platform files (ex. open/src/hotspot/os/linux/os_linux.cpp, etc?) now that we have dedicated gtest files? >>>>> >>>>> #2 Also, I only see linux and windows gtests, what about the ones for Mac and Solaris, now that you removed the test from internalVMTests.cpp? >>>>> >>>>> #3 Are the linux, windows gtest files what TestReserveMemorySpecial class is/was in open/src/hotspot/os/linux/os_linux.cpp, open/src/hotspot/os/windows/os_windows.cpp? On cursory look I see some differences. >>>>> >>>>> >>>>> cheers >>>>> >>>>>> On Oct 22, 2018, at 1:36 PM, Igor Ignatyev wrote: >>>>>> >>>>>> http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>>>>> 331 lines changed: 330 ins; 1 del; 0 mod; >>>>>> Hi all, >>>>>> >>>>>> could you please review this small and trivial patch which converts TestReserveMemorySpecial_test tests to Gtest? the old tests[1,2] haven't been removed as they are used by WhiteBox::runMemoryUnitTests they will be removed when the rest tests are converted. >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177708 >>>>>> [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/linux/os_linux.cpp#l5969 >>>>>> [2] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/windows/os_windows.cpp#l5564 >>>>>> >>>>>> Thanks, >>>>>> -- Igor >>>>> >>>> >>> >> > From mandy.chung at oracle.com Fri Nov 2 20:55:29 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 2 Nov 2018 13:55:29 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> Message-ID: Hi Dean, I reviewed webrev.4 version.? It looks good.? Happy to see moving the doPrivileged support to Java and the performance improvement. On 10/31/18 3:23 PM, dean.long at oracle.com wrote: > https://bugs.openjdk.java.net/browse/JDK-8212605 > http://cr.openjdk.java.net/~dlong/8212605/webrev.1 > > This change implements AccessController.doPrivileged in Java. This > gives a performance improvement while also being useful to Project > Loom by removing the Java --> native --> Java transition.? One reason > doPrivileged has historically been in native is because of the need to > guarantee the cleanup of the privileged context when doPrivileged > returns.? To do that in Java, the information that > AccessController.getContext needs is pushed onto the Java stack as > arguments to a method that getContext will recognize during its stack > walk.? This allows us to remove the native privileged stack while > guaranteeing that the privileged context goes away when the method > returns. > Tested with tier1-tier3 hotspot and jdk tests and JCK > api/java_security tests.? For the first few rounds of testing, I kept > the old native privileged stack and compared the results of the old > and new implementations for each getContext call, which did catch some > early bugs.? The changes were also examined by internal security > experts and run through additional internal security tests. > > The improvement on this [1] doPrivileged microbenchmark is approximate > 50x. > > There is no attempt to optimize getContext() or security permission > checks in this change, however, this is intended to be a first step > towards other possible improvements, for example those proposed here [2]. > FYI.? Sean and I also did some experiment to replace JVM_GetStackAccessControlContext with StackWalker some time ago. Another potential area to move the code from VM to Java for the future as David explored and probably involves? performance work in the stack walker. Mandy From dean.long at oracle.com Fri Nov 2 21:09:45 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 2 Nov 2018 14:09:45 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> Message-ID: Thanks Mandy.? I also appreciate you noticing (off-list) that I can remove the extra space in "Class " in several places.? I have updated webrev.4 in place. dl On 11/2/18 1:55 PM, Mandy Chung wrote: > Hi Dean, > > I reviewed webrev.4 version.? It looks good.? Happy to see moving the > doPrivileged support to Java and the performance improvement. > > On 10/31/18 3:23 PM, dean.long at oracle.com wrote: >> https://bugs.openjdk.java.net/browse/JDK-8212605 >> http://cr.openjdk.java.net/~dlong/8212605/webrev.1 >> >> This change implements AccessController.doPrivileged in Java. This >> gives a performance improvement while also being useful to Project >> Loom by removing the Java --> native --> Java transition.? One reason >> doPrivileged has historically been in native is because of the need >> to guarantee the cleanup of the privileged context when doPrivileged >> returns.? To do that in Java, the information that >> AccessController.getContext needs is pushed onto the Java stack as >> arguments to a method that getContext will recognize during its stack >> walk.? This allows us to remove the native privileged stack while >> guaranteeing that the privileged context goes away when the method >> returns. >> Tested with tier1-tier3 hotspot and jdk tests and JCK >> api/java_security tests.? For the first few rounds of testing, I kept >> the old native privileged stack and compared the results of the old >> and new implementations for each getContext call, which did catch >> some early bugs.? The changes were also examined by internal security >> experts and run through additional internal security tests. >> >> The improvement on this [1] doPrivileged microbenchmark is >> approximate 50x. >> >> There is no attempt to optimize getContext() or security permission >> checks in this change, however, this is intended to be a first step >> towards other possible improvements, for example those proposed here >> [2]. >> > > FYI.? Sean and I also did some experiment to replace > JVM_GetStackAccessControlContext with StackWalker some time ago. > Another potential area to move the code from VM to Java for the future > as David explored and probably involves? performance work in the stack > walker. > > Mandy From kim.barrett at oracle.com Fri Nov 2 21:26:13 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 2 Nov 2018 17:26:13 -0400 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> Message-ID: <37009003-5EDF-455F-ADEF-EB2E3C9B5666@oracle.com> > On Nov 2, 2018, at 7:51 AM, Erik ?sterlund wrote: > So that takes me back to my original intuition: isn't this the same as checking at the very top: if (offset == BrooksPointer::byte_offset()) return false; > > If it really is as simple as just that, and I did not miss something not obvious, then perhaps we would be better off abstracting a cell header check here, as opposed to wrapping the whole thing in a virtual member function. Especially since there are now a number of these occurrences where some basic knowledge about cell header size that is 0 for all GCs except Shenandoah where it is 8, leaves us wrapping a whole bunch of stuff behind a GC interface, only to handle that one difference. > > While I don't know exactly what this cell abstraction would reasonably look like in its full shape, perhaps we could start with something simple like a virtual member function size_t cell_header_size() on CollectedHeap with a suitable comment explaining that a cell header is a GC thing we sometimes put above the object header. And then see if there are more related functions that lead us to a helper class for cells or something like that. > > I'm open to suggestions of course. Thought I'd check with you if this sounds sane to you. Of course if my assumption is wrong that this check could be written much simpler than it looks like, then I suppose I need to give up on that idea. It all boils down to that really. I think Erik makes some good points here. At least for now, I?m withdrawing my approval of the changes offered so far. From kim.barrett at oracle.com Fri Nov 2 21:54:15 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 2 Nov 2018 17:54:15 -0400 Subject: RFR: 6735527: Bitmap - speed up searches In-Reply-To: <1f7893b7-3362-e25d-23fd-e708718c24d8@redhat.com> References: <7B080943-5D16-476C-83EE-C7A0DE194F03@oracle.com> <1f7893b7-3362-e25d-23fd-e708718c24d8@redhat.com> Message-ID: > On Nov 2, 2018, at 5:17 AM, Aleksey Shipilev wrote: > > On 10/27/2018 02:18 AM, Kim Barrett wrote: >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/6735527/open.00/ > > I like this patch. Shenandoah concurrent phase times have improved a little bit with it, especially > when the heap is very sparse. > > -Aleksey Thanks! From vladimir.x.ivanov at oracle.com Fri Nov 2 22:16:28 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 2 Nov 2018 15:16:28 -0700 Subject: RFR(XS) : 8177708 : Convert TestReserveMemorySpecial_test to Gtest In-Reply-To: References: <9BD612CF-150E-4D4F-A15B-93A4154A921A@oracle.com> <102A0AFA-26D2-4D2E-A6DA-4A54A79B6FF9@oracle.com> <31A74048-B1BB-4C5B-ADBE-C9C3724BF948@oracle.com> <095D5F1E-2CB6-4EAB-9839-16C72E4C4CC3@oracle.com> Message-ID: Looks good. Best regards, Vladimir Ivanov On 02/11/2018 13:28, Igor Ignatyev wrote: > Hi Gerard, > > thanks for your review! > > All, > > can I get a second review from a Reviewer? > > -- Igor > >> On Nov 2, 2018, at 11:50 AM, Gerard Ziemski wrote: >> >> hi Igor, >> >> Thanks for the explanation, looks OK. >> >> >> cheers >> >>> On Nov 1, 2018, at 2:10 PM, Igor Ignatyev wrote: >>> >>> Gerard, >>> >>> #1. this assert is kinda contract of the function. test_reserve_memory_special_shm is an aux test function which works iif UseSHM is true. invoking it when UseSHM is false is a test error, and I'd prefer not to hide such errors and get a test error. 'if (!$X) { return }' construction is used only in actual tests and is current workaround to express things similar to @requiers in jtreg. >>> >>> #2. I did the testing appropriated for the made changes: all gtest tests (including new ones) on all "regular" platforms w/ precompiled header enabled and disabled. I've also run mach5 tier1 job just in case, running any other tiers won't provide us w/ any extra coverage. >>> >>> I have to say though, as these tests depend on host configuration (thru vm flags values), getting passed status doesn't mean much as all the test tasks could have ended up on hosts which don't support large pages. >>> >>> Thanks, >>> -- Igor >>> >>>> On Nov 1, 2018, at 10:58 AM, Gerard Ziemski wrote: >>>> >>>> hi Igor, >>>> >>>> Nice cleanup. >>>> >>>> I tried to follow the original code and the changes/cleanups you did, and it seems OK to me, even if the new code seems a bit less verbal. I do have 2 questions though: >>>> >>>> #1 In test/hotspot/gtest/runtime/test_os_linux.cpp we have: >>>> >>>> 83 static void test_reserve_memory_special_shm(size_t size, size_t alignment) { >>>> 84 ASSERT_TRUE(UseSHM) << "must be used only when UseSHM is true"; >>>> >>>> But in src/hotspot/os/linux/os_linux.cpp we have: >>>> >>>> 6110 static void test_reserve_memory_special_shm(size_t size, size_t alignment) { >>>> 6111 if (!UseSHM) { >>>> 6112 return; >>>> 6113 } >>>> >>>> Why do we bother with ?ASSERT_TRUE? here, when the original code didn?t, and when other checks for "UseHugeTLBFS" simply silently return? I?d prefer we either use ASSERT_TRUE for all the flags here (if it doesn?t hurt the test), or leave it as in the original code. >>>> >>>> #2 Did you run any Mach5 tiers to test this change? >>>> >>>> >>>> cheers >>>> >>>>> On Nov 1, 2018, at 12:11 PM, Igor Ignatyev wrote: >>>>> >>>>> Hi Gerard, >>>>> >>>>> #1. As I mentioned earlier and you can see from your grep results, this test is still called WhiteBox::runMemoryUnitTests which is used in runtime/memory/RunUnitTestsConcurrently to run this and a few other tests multiple times in several concurrent threads. runtime/memory/RunUnitTestsConcurrently will be converted to gtest separately, and this conversion will also include removing of all these tests. >>>>> >>>>> #2. only linux and windows version of TestReserveMemorySpecial_test aren't trivial, the rest are just empty. >>>>> >>>>> #3. semantically linux gtest is doing the same as TestReserveMemorySpecial in open/src/hotspot/os/linux/os_linux.cpp, and windows gtest -- open/src/hotspot/os/linux/os_windows.cpp. yes, there are some differences caused by difference in "frameworks" and/or aimed to improve the tests, e.g. changes in how tests are split, error/trace message are printed, etc. >>>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>>>>> On Nov 1, 2018, at 8:44 AM, Gerard Ziemski wrote: >>>>>> >>>>>> hi Igor, >>>>>> >>>>>> I took a look at the webrev, but I think I?m missing the big picture. If I grep for TestReserveMemorySpecial I get: >>>>>> >>>>>> # grep -rn TestReserveMemorySpecial >>>>>> open/src/hotspot/os/bsd/os_bsd.cpp:3778:void TestReserveMemorySpecial_test() { >>>>>> open/src/hotspot/os/linux/os_linux.cpp:5967:class TestReserveMemorySpecial : AllStatic { >>>>>> open/src/hotspot/os/linux/os_linux.cpp:6146:void TestReserveMemorySpecial_test() { >>>>>> open/src/hotspot/os/linux/os_linux.cpp:6147: TestReserveMemorySpecial::test(); >>>>>> open/src/hotspot/os/linux/os_linux.hpp:36: friend class TestReserveMemorySpecial; >>>>>> open/src/hotspot/os/windows/os_windows.cpp:5561:void TestReserveMemorySpecial_test() { >>>>>> open/src/hotspot/os/solaris/os_solaris.cpp:5405:void TestReserveMemorySpecial_test() { >>>>>> open/src/hotspot/os/aix/os_aix.cpp:4321:void TestReserveMemorySpecial_test() { >>>>>> open/src/hotspot/share/prims/whitebox.cpp:231:void TestReserveMemorySpecial_test(); >>>>>> open/src/hotspot/share/prims/whitebox.cpp:239: TestReserveMemorySpecial_test(); >>>>>> open/src/hotspot/share/utilities/internalVMTests.cpp:45: run_unit_test(TestReserveMemorySpecial_test); >>>>>> >>>>>> #1 Wouldn?t you want to delete TestReserveMemorySpecial class from the platform files (ex. open/src/hotspot/os/linux/os_linux.cpp, etc?) now that we have dedicated gtest files? >>>>>> >>>>>> #2 Also, I only see linux and windows gtests, what about the ones for Mac and Solaris, now that you removed the test from internalVMTests.cpp? >>>>>> >>>>>> #3 Are the linux, windows gtest files what TestReserveMemorySpecial class is/was in open/src/hotspot/os/linux/os_linux.cpp, open/src/hotspot/os/windows/os_windows.cpp? On cursory look I see some differences. >>>>>> >>>>>> >>>>>> cheers >>>>>> >>>>>>> On Oct 22, 2018, at 1:36 PM, Igor Ignatyev wrote: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>>>>>> 331 lines changed: 330 ins; 1 del; 0 mod; >>>>>>> Hi all, >>>>>>> >>>>>>> could you please review this small and trivial patch which converts TestReserveMemorySpecial_test tests to Gtest? the old tests[1,2] haven't been removed as they are used by WhiteBox::runMemoryUnitTests they will be removed when the rest tests are converted. >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8177708/webrev.00/index.html >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8177708 >>>>>>> [1] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/linux/os_linux.cpp#l5969 >>>>>>> [2] http://hg.openjdk.java.net/jdk/jdk/file/tip/src/hotspot/os/windows/os_windows.cpp#l5564 >>>>>>> >>>>>>> Thanks, >>>>>>> -- Igor >>>>>> >>>>> >>>> >>> >> > From rkennke at redhat.com Fri Nov 2 22:31:02 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 2 Nov 2018 23:31:02 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> Message-ID: <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> Hi Erik, I had a discussion with Andrew Dinn (who actually wrote that code back then) and will copy his explanation here, I think it covers the situation: "" Let's just recap without considering Shenandoah: MacroAssembler::needs_explicit_null_check is called from several places, not just the SEGV handler. In those other cases offset is a putative field offset for an object. With uncompressed pointers the method has to decide whether this offset will lie within the read-protected page at address 0 i.e. 0 <= offset < os::vm_page_size(). If that is true then a null check can be omitted for a read or write operation. If not an explicit check is needed. With compressed pointers when Universe::narrow_oop_base() != NULL the test also checks whether this offset may lie within the read-protected page at the heap base only -- oh -- that's the same test! Of course, we'll see that this is different when the SEGV handler makes the call. n.b. I think this case is included because there is the possibility that a null pointer can be represented by the narrow oop base address. However, I think in reality special case checks in all encode decode operations ensure that a null oop is always represented as zero (for AArch64 at least). I'm not absolutely sure of that though. Now, when called from the SEGV handler offset is actually the offending read/write address that caused the SEGV. In that case a null pointer will only have been dereferenced if the address lies in the protected zero page or in the protected heap base page. The first if clause handles the compressed pointer case and reduces it to the uncompressed pointer case by subtracting base when offset >= base. So, either way the offending address is potentially a null pointer dereference under exactly the same conditions as for those other calls. Next, how does Shenandoah modify this? Well, it looks like that depends on how this method is called. If it can be invoked from any of those other places to check whether a Brooks pointer read or write is ok to validate with an implicit null check then the offset may be passed as -8 i.e. 0xfffffffffffffff8. However, that's actually not enough. When invoked from the handler because of an access at of (0 + offset) then offset may legitimately be 0x00fffffffffffff8 or lie between 0 and os::vm_page_size(). When invoked form the handler because of a dereference of (narrow_oop_base + offset) then the subtract i the if clause means that offset may legitimately be 0xfffffffffffffff8 or lie between 0 and os::vm_page_size(). So, the masking in the Shenandoah case is to reduce those two special cases into one. "" Roman > Hi Roman, > > On 2018-11-01 17:58, Roman Kennke wrote: >> Hi Erik, >> >>> Would you mind explaining how you need to override this and why? I'm >>> afraid I did not quite get it from your description in the RFC (which is >>> also why I didn't come up with a solution either). >>> >>> Am I correct that you need to return false if the passed in offset is >>> the cell header offset -8, or is there anything else at all required to >>> that logic? >> >> No, it's really just that. Plus take care of it in the case of combined >> narrow-oops-offset plus -8. > >>> You mentioned something about the high order byte being >>> masked on AArch64, but didn't quite connect the dot to how that is >>> related to this code. Is it? >> >> Yes, we also need to mask the high order byte in AArch64 because of the >> way addressing works on that platform, and because -8 flips on those >> negative bits. > > Okay. Looking at > https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html > that you posted before, assuming this is how it will be extended. > > So in the case with a heap base, the resulting address will never be > negative, and you know it will trap (presuming there is an mprotected > page at offset -8 from the heap base pointer. So it seems in this case, > the check could have just been offset == -8. > > So let's say we access null -8 without a heap base then. The address > becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the > signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address > masking of the high order byte. I presume that is what the AArch64 code > is dealing with. But I don't quite understand why. Addresses starting > with 0x00FF are not valid in user space so we never need to worry about > any memory being mapped in there accidentally. So I'm not sure what this > code protects against. > > So that takes me back to my original intuition: isn't this the same as > checking at the very top: if (offset == BrooksPointer::byte_offset()) > return false; > > If it really is as simple as just that, and I did not miss something not > obvious, then perhaps we would be better off abstracting a cell header > check here, as opposed to wrapping the whole thing in a virtual member > function. Especially since there are now a number of these occurrences > where some basic knowledge about cell header size that is 0 for all GCs > except Shenandoah where it is 8, leaves us wrapping a whole bunch of > stuff behind a GC interface, only to handle that one difference. > > While I don't know exactly what this cell abstraction would reasonably > look like in its full shape, perhaps we could start with something > simple like a virtual member function size_t cell_header_size() on > CollectedHeap with a suitable comment explaining that a cell header is a > GC thing we sometimes put above the object header. And then see if there > are more related functions that lead us to a helper class for cells or > something like that. > > I'm open to suggestions of course. Thought I'd check with you if this > sounds sane to you. Of course if my assumption is wrong that this check > could be written much simpler than it looks like, then I suppose I need > to give up on that idea. It all boils down to that really. > > Thanks, > /Erik > >> Also, the mach5 job came back with FAILED (see below). Can somebody with >> access check and see what's up? >> >> Thanks, >> Roman >> >> >> Build Details: 2018-11-01-1146402.roman.source >> 0 Failed Tests >> Mach5 Tasks Results Summary >> >> ???? EXECUTED_WITH_FAILURE: 6 >> ???? KILLED: 0 >> ???? UNABLE_TO_RUN: 21 >> ???? PASSED: 48 >> ???? FAILED: 0 >> ???? NA: 0 >> ???? Build >> >> ???????? 6 Executed with failure >> ???????????? solaris-sparcv9-solaris-sparcv9-build-8 error while >> building, return value: 2 >> ???????????? solaris-sparcv9-debug-solaris-sparcv9-build-9 error while >> building, return value: 2 >> ???????????? windows-x64-windows-x64-build-10 error while building, >> return value: 2 >> ???????????? windows-x64-debug-windows-x64-build-11 error while building, >> return value: 2 >> ???????????? windows-x64-open-windows-x64-build-12 error while building, >> return value: 2 >> ???????????? windows-x64-open-debug-windows-x64-build-13 error while >> building, return value: 2 >> ???????? 2 Not run >> ???????????? solaris-sparcv9-install-solaris-sparcv9-build-16 Dependency >> task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >> ???????????? windows-x64-install-windows-x64-build-17 Dependency task >> failed: YJftjiBUYc >> >> ???? Test >> >> ???????? 19 Not run >> >> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 >> >> Dependency task failed: >> mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >> >> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 >> >> Dependency task failed: >> mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 >> >> tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 >> Dependency task failed: YJftjiBUYc >> >> tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 >> Dependency task failed: YJftjiBVYc >> >> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 >> Dependency >> task failed: YJftjiBVYc >> >> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 >> Dependency >> task failed: YJftjiBVYc >> >> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 >> Dependency >> task failed: YJftjiBVYc >> >> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 >> >> Dependency task failed: YJftjiBVYc >> >> tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 >> Dependency task failed: YJftjiBVYc >> >> tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 >> Dependency task failed: YJftjiBVYc >> ???????????? See all 19... >> >> >> >>> Thanks, >>> /Erik >>> >>> On 2018-11-01 12:20, Roman Kennke wrote: >>>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>>> >>>> Erik: ok for you too? >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >>>>>> >>>>>> Hi Erik, >>>>>> >>>>>> right. Fixed this, and what what Kim mentioned plus a missing >>>>>> include: >>>>>> >>>>>> Incremental: >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>>> Full: >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>>> >>>>>> Ok now? >>>>> Looks good. >>>>> >>> >> From vladimir.kozlov at oracle.com Fri Nov 2 22:49:34 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 2 Nov 2018 15:49:34 -0700 Subject: RFR: 8212989: Allow CompiledMethod ExceptionCache have unloaded klasses In-Reply-To: <604b581e-00f2-24df-b06f-738bdefc593a@oracle.com> References: <5BDAF881.3030905@oracle.com> <604b581e-00f2-24df-b06f-738bdefc593a@oracle.com> Message-ID: <6f6332b1-2a61-6f86-9dd4-7d6d3e9622c2@oracle.com> On 11/1/18 3:14 PM, Erik ?sterlund wrote: > Hi Vladimir, > > Thanks for the quick reply. > > On 2018-11-01 21:14, Vladimir Kozlov wrote: >> Hi Erik, >> >> It seems these? changed are not based on latest sources. I don't see call to CodeCache::do_unloading in >> CodeCache::do_unloading() which is present in latest code: >> >> http://hg.openjdk.java.net/jdk/jdk/file/5ea020bcaa0d/src/hotspot/share/code/codeCache.cpp#l683 > > Yeah this code is taken from the context of the future. Here is the exact same patch rebased against jdk-jdk: > http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01/ > > And new webrev with your suggestions: > http://cr.openjdk.java.net/~eosterlund/8212989/webrev.02/ Looks good. > > And incremental of that to the jdk-jdk rebase: > http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01_02/ > >> >> May be make some methods private, like CompiledMethod::exception_cache_acquire(). > > Sure. Made the accessors protected, as some are used in nmethod too and it seems fine for subclasses to access. > >> It seems you remove the only case when release_set_exception_cache() is called. > > That is true. Removed. > >> I don't see where purge_exception_caches() is used. > > It's only called by a future version of ZGC from its new concurrent class unloading feature that is currently baking in > the ZGC repository. The release_exception_cache member function deletes entries synchronously if cleanup is triggered in > a safepoint (all GCs except ZGC), and otherwise defers deletion to the purge_exception_caches() method, that is expected > to happen one global handshake operation after the exception caches have been unlinked by cleanup. > >> Next sentence seems incomplete: >> >> +? // unlinked. That is also when the CodeCache::exception_cache_free_list() > > Fixed. > >> Why you use exception_cache_acquire() in clean_exception_cache() and simple exception_cache() in >> add_exception_cache_entry()? > > The reason is that in the context of insertion, you are holding the lock protecting against concurrent insertions. > Therefore, entries you load are either entries that were protected by the same lock (other insertions), or entries that > are as old or older than entries that have died a whole safepoint operation earlier (due to prepending inserts), which > makes its memory state stable. Therefore, no acquire is required when reading the head in the insertion path. Got it. Thanks, Vladimir > > Thanks, > /Erik > >> Thanks, >> Vladimir >> >> On 11/1/18 5:58 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> The ExceptionCaches of compiled methods is traditionally pruned in safepoints during class unloading. This allows the >>> exception cache to have lock-free reads, while performing inserts under a lock, and pruning under a safepoint. With >>> concurrent class unloading, the exception caches need to be cleaned concurrently. >>> >>> To retain the lock-free reads that seem good to keep lock-free, a lock-free cleaning mechanism was introduced. Only >>> one thread per compiled method cleans each exception cache, but that thread races with both single writers (under the >>> ExceptionCache_lock) and any number of readers. >>> >>> The head of the exception cache list is concurrently cleaned by both inserting threads and cleaning threads. This >>> allows having an invariant that no newly prepended entries ever produce next pointers to dead exception caches, that >>> are concurrently removed from the list. As for the internal next pointers, they are only pruned by the one concurrent >>> cleaning thread. This happens concurrent to reads that simply skip over dead entries as they have different Klass* >>> pointers to the one being searched for. >>> >>> The single concurrent cleanup thread does not delete things removed from the list straight away. Instead, they are >>> placed on a purge list that is freed up after a subsequent global handshaking operation. That allows ABA-free CAS >>> instructions in the lock-free paths, and allows safe concurrent reading of entries in the exception cache list. >>> >>> Note that we already incorrectly "relied on memory_order_consume" for the head pointer. But it used a volatile load. >>> Assuming volatile loads retain memory_order_consume semantics is not allowed in HotSpot runtime code, so I changed >>> those loads to use acquire accordingly. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212989 >>> >>> Thanks, >>> /Erik From ioi.lam at oracle.com Sat Nov 3 00:27:40 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 2 Nov 2018 17:27:40 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> Message-ID: <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> Hi Jiangli, Thanks for the review On 11/2/18 1:24 PM, Jiangli Zhou wrote: > Hi Ioi, > > - src/hotspot/share/classfile/verifier.cpp > > You moved > SystemDictionaryShared::finalize_verification_constraints(_klass) to > ClassVerifier::verify_class(). We should be very careful in this area. > Have you tested with cases where classes fail verification? void ClassVerifier::verify_class(TRAPS) { ? log_info(verification)("Verifying class %s with new format", _klass->external_name()); ? Array* methods = _klass->methods(); ? int num_methods = methods->length(); ? for (int index = 0; index < num_methods; index++) { ??? // Check for recursive re-verification before each method. ??? if (was_recursively_verified())? return; ??? Method* m = methods->at(index); ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { ????? // If m is native or abstract, skip it.? It is checked in class file ????? // parser that methods do not override a final method. Overpass methods ????? // are trusted since the VM generates them. ????? continue; ??? } ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); ? } ? if (DumpSharedSpaces) { SystemDictionaryShared::finalize_verification_constraints(_klass); ? } If verification failed, the CHECK_VERIFY macro will force the function to return. So we will call finalize_verification_constraints only when verification has succeeded. > I haven't ran your code yet, we should avoid copying the constraints > for classes that fails verification at dump time. If a class has failed verification, the class will not be archived. So whether we have called finalize_verification_constraints on it really doesn't matter. > I also need to think through any potential impact on dynamic archiving > (possibly none). > > - src/hotspot/share/memory/metaspace.cpp > > -??? if (is_init_completed() && !(DumpSharedSpaces && > THREAD->is_VM_thread())) { > +??? if (is_init_completed()) { > > Please also add an assert to make sure we don't allocate metaspace > objects from the VM_thread at dump time now. > I think it's better to add a blanket assert(!THREAD->is_VM_thread()), because no one should be allocating metaspace objects inside VM thread. Adding this has a wider impact, so I will do it in a separate RFE (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we need to back it out. Thanks - Ioi > Thanks, > > Jiangli > > > On 11/1/18 9:47 PM, Ioi Lam wrote: >> Hi, >> >> I've revised the webrev to get to the the root of the problem -- we >> shouldn't allocate metaspace objects from within the VM Thread! >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >> >> >> The original bug that caused the "!(DumpSharedSpaces && >> THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was >> >> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock (not >> allowed to block on)" >> >> The fix in JDK-8196626 is just a band-aid solution -- avoiding the >> GC. The proper fix is not to do any allocation inside the VM thread. >> So instead of calling finalize_verification_constraints, which >> allocates MetaspaceObjs, very late in the CDS dumping process, now we >> call be as soon as a class has been verified. >> >> Thanks >> - Ioi >> >> >> >> >> >> On 11/1/18 11:07 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>> >>> >>> When Metaspace::allocate() fails, usually we first try to GC to see if >>> some MetaspaceObjs can be freed. This is an optimization to avoid >>> prematurely >>> expanding the metaspace. >>> >>> However, GC cannot run in the last phase of CDS which runs inside >>> the VM thread. >>> The old code just aborts the CDS dump. The fix is to unconditionally >>> expand the >>> metaspace without doing a GC. >>> >>> (During GC dump time, no classes are unloaded, so even if we force a >>> GC, it would >>> not free up any metaspace anyway). >>> >>> Testing: >>> >>> Before the fix, the test case (jruby) would fail to dump once a >>> couple of times. Now >>> it has dumped hundreds of times without failure. >>> >>> I am also running hs-tier 1/2 >>> >>> Thanks >>> - Ioi >> > From ioi.lam at oracle.com Sat Nov 3 03:54:04 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 2 Nov 2018 20:54:04 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <5BDCA5DA.5000800@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <5BDCA5DA.5000800@oracle.com> Message-ID: <9b7c8aa3-affe-419f-d158-1bd76db7fd8e@oracle.com> Hi Calvin, thanks for the review. On 11/2/18 12:30 PM, Calvin Cheung wrote: > Hi Ioi, > > The revised webrev looks good. > > The following is removed from systemDictionaryShared.cpp: > 831???? log_info(cds, verification)("Recorded verification constraints > for %d classes = %d bytes (avg = %.2f bytes) ", count, bytes, avg); > > Please make sure no test is expecting the above output. I checked all CDS tests and there's no tests for that expects the above output. > > Line 790 in the same file is a bit too long, consider break it into 2 > lines. > Done. I also broke the same line in SystemDictionaryShared::add_verification_constraint into two lines. Thanks - Ioi > thanks, > Calvin > > > On 11/1/18, 9:47 PM, Ioi Lam wrote: >> Hi, >> >> I've revised the webrev to get to the the root of the problem -- we >> shouldn't allocate metaspace objects from within the VM Thread! >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >> >> >> The original bug that caused the "!(DumpSharedSpaces && >> THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was >> >> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock (not >> allowed to block on)" >> >> The fix in JDK-8196626 is just a band-aid solution -- avoiding the >> GC. The proper fix is not to do any allocation inside the VM thread. >> So instead of calling finalize_verification_constraints, which >> allocates MetaspaceObjs, very late in the CDS dumping process, now we >> call be as soon as a class has been verified. >> >> Thanks >> - Ioi >> >> >> >> >> >> On 11/1/18 11:07 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>> >>> >>> When Metaspace::allocate() fails, usually we first try to GC to see if >>> some MetaspaceObjs can be freed. This is an optimization to avoid >>> prematurely >>> expanding the metaspace. >>> >>> However, GC cannot run in the last phase of CDS which runs inside >>> the VM thread. >>> The old code just aborts the CDS dump. The fix is to unconditionally >>> expand the >>> metaspace without doing a GC. >>> >>> (During GC dump time, no classes are unloaded, so even if we force a >>> GC, it would >>> not free up any metaspace anyway). >>> >>> Testing: >>> >>> Before the fix, the test case (jruby) would fail to dump once a >>> couple of times. Now >>> it has dumped hundreds of times without failure. >>> >>> I am also running hs-tier 1/2 >>> >>> Thanks >>> - Ioi >> From jiangli.zhou at oracle.com Sat Nov 3 05:42:48 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 2 Nov 2018 22:42:48 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> Message-ID: <47520602-3945-9691-9c3b-217153a362a6@oracle.com> Hi Ioi, On 11/2/18 5:27 PM, Ioi Lam wrote: > Hi Jiangli, > > Thanks for the review > > > On 11/2/18 1:24 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> - src/hotspot/share/classfile/verifier.cpp >> >> You moved >> SystemDictionaryShared::finalize_verification_constraints(_klass) to >> ClassVerifier::verify_class(). We should be very careful in this >> area. Have you tested with cases where classes fail verification? > > void ClassVerifier::verify_class(TRAPS) { > ? log_info(verification)("Verifying class %s with new format", > _klass->external_name()); > > ? Array* methods = _klass->methods(); > ? int num_methods = methods->length(); > > ? for (int index = 0; index < num_methods; index++) { > ??? // Check for recursive re-verification before each method. > ??? if (was_recursively_verified())? return; > > ??? Method* m = methods->at(index); > ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { > ????? // If m is native or abstract, skip it.? It is checked in class > file > ????? // parser that methods do not override a final method. Overpass > methods > ????? // are trusted since the VM generates them. > ????? continue; > ??? } > ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); > ? } > > ? if (DumpSharedSpaces) { > SystemDictionaryShared::finalize_verification_constraints(_klass); > ? } > > If verification failed, the CHECK_VERIFY macro will force the function > to return. So we will call finalize_verification_constraints only when > verification has succeeded. Ok, thanks for checking. An alternative is to allocate from the RO space directly after relocation. That way you can avoid adding another CDS specific in the verifier code. There is also a FIXME for using the symbol offset in SharedDictionaryEntry::finalize_verification_constraints, which you could handle together. > > >> I haven't ran your code yet, we should avoid copying the constraints >> for classes that fails verification at dump time. > > If a class has failed verification, the class will not be archived. So > whether we have called finalize_verification_constraints on it really > doesn't matter. Since it doesn't do the copying when verification fails, it is okay in this case. With the work for dynamic archiving, we want to avoid unnecessary work at dump time, since performance/footprint will become higher priority with archiving at runtime. > >> I also need to think through any potential impact on dynamic >> archiving (possibly none). >> >> - src/hotspot/share/memory/metaspace.cpp >> >> -??? if (is_init_completed() && !(DumpSharedSpaces && >> THREAD->is_VM_thread())) { >> +??? if (is_init_completed()) { >> >> Please also add an assert to make sure we don't allocate metaspace >> objects from the VM_thread at dump time now. >> > > I think it's better to add a blanket assert(!THREAD->is_VM_thread()), > because no one should be allocating metaspace objects inside VM thread. > > Adding this has a wider impact, so I will do it in a separate RFE > (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we need to > back it out. I'd suggest including the assert for dump time with the fix so we are sure it is safe. Then you can follow up with the broader assert that you are planning for JDK-8213331. Thanks, Jiangli > > Thanks > - Ioi >> Thanks, >> >> Jiangli >> >> >> On 11/1/18 9:47 PM, Ioi Lam wrote: >>> Hi, >>> >>> I've revised the webrev to get to the the root of the problem -- we >>> shouldn't allocate metaspace objects from within the VM Thread! >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>> >>> >>> The original bug that caused the "!(DumpSharedSpaces && >>> THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was >>> >>> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >>> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock (not >>> allowed to block on)" >>> >>> The fix in JDK-8196626 is just a band-aid solution -- avoiding the >>> GC. The proper fix is not to do any allocation inside the VM thread. >>> So instead of calling finalize_verification_constraints, which >>> allocates MetaspaceObjs, very late in the CDS dumping process, now >>> we call be as soon as a class has been verified. >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> >>> >>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>> >>>> >>>> When Metaspace::allocate() fails, usually we first try to GC to see if >>>> some MetaspaceObjs can be freed. This is an optimization to avoid >>>> prematurely >>>> expanding the metaspace. >>>> >>>> However, GC cannot run in the last phase of CDS which runs inside >>>> the VM thread. >>>> The old code just aborts the CDS dump. The fix is to >>>> unconditionally expand the >>>> metaspace without doing a GC. >>>> >>>> (During GC dump time, no classes are unloaded, so even if we force >>>> a GC, it would >>>> not free up any metaspace anyway). >>>> >>>> Testing: >>>> >>>> Before the fix, the test case (jruby) would fail to dump once a >>>> couple of times. Now >>>> it has dumped hundreds of times without failure. >>>> >>>> I am also running hs-tier 1/2 >>>> >>>> Thanks >>>> - Ioi >>> >> > From ioi.lam at oracle.com Sat Nov 3 05:59:19 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 2 Nov 2018 22:59:19 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> Message-ID: <7285bac9-48dc-9112-46db-d65e0c036c52@oracle.com> Hi Thomas, Yes, it was an oversight. I've added the following line: ClassLoaderMetaspace::~ClassLoaderMetaspace() { + Metaspace::assert_not_frozen(); DEBUG_ONLY(Atomic::inc(&g_internal_statistics.num_metaspace_deaths)); ? ... Thanks - Ioi On 11/2/18 12:36 PM, Thomas St?fe wrote: > Hi Ioi, > > this not a Review, since I do not know the code well enough. > > But while looking at your change, I came across Metaspace::freeze() > and saw that we do "assert_not_frozen()" on allocation paths: > ClassMetaSpace::allocate and ::expand_and_allocate (redundant?) and > the atypical deallocation path (::deallocate()). But not on the normal > deallocation path when the CMS just dies because its class is unloaded > (~ClassLoaderMetaspace()). Is this by design or an oversight? > > Best Regards, Thomas > > > On Fri, Nov 2, 2018 at 5:47 AM Ioi Lam wrote: >> Hi, >> >> I've revised the webrev to get to the the root of the problem -- we >> shouldn't allocate metaspace objects from within the VM Thread! >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >> >> The original bug that caused the "!(DumpSharedSpaces && >> THREAD->is_VM_thread())" check to be added in Metaspace::allocate() was >> >> https://bugs.openjdk.java.net/browse/JDK-8196626 >> UseAppCDS.java crashes with "VM thread using lock Heap_lock (not >> allowed to block on)" >> >> The fix in JDK-8196626 is just a band-aid solution -- avoiding the GC. >> The proper fix is not to do any allocation inside the VM thread. So >> instead of calling finalize_verification_constraints, which allocates >> MetaspaceObjs, very late in the CDS dumping process, now we call be as >> soon as a class has been verified. >> >> Thanks >> - Ioi >> >> >> >> >> >> On 11/1/18 11:07 AM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>> >>> >>> When Metaspace::allocate() fails, usually we first try to GC to see if >>> some MetaspaceObjs can be freed. This is an optimization to avoid >>> prematurely >>> expanding the metaspace. >>> >>> However, GC cannot run in the last phase of CDS which runs inside the >>> VM thread. >>> The old code just aborts the CDS dump. The fix is to unconditionally >>> expand the >>> metaspace without doing a GC. >>> >>> (During GC dump time, no classes are unloaded, so even if we force a >>> GC, it would >>> not free up any metaspace anyway). >>> >>> Testing: >>> >>> Before the fix, the test case (jruby) would fail to dump once a couple >>> of times. Now >>> it has dumped hundreds of times without failure. >>> >>> I am also running hs-tier 1/2 >>> >>> Thanks >>> - Ioi From igor.ignatyev at oracle.com Sat Nov 3 06:35:49 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 2 Nov 2018 23:35:49 -0700 Subject: RFR(XS/trivial) : 8213337 : windows-x64-slowdebug build is broken by 8177708 Message-ID: <75FB6703-69DB-424F-B09A-D102C6E2BBE4@oracle.com> http://cr.openjdk.java.net/~iignatyev//8213337/webrev.00/index.html > 1 line changed: 1 ins; 0 del; 0 mod; Hi all, could you please review this small and trivial fix? test/hotspot/gtest/runtime/test_os_windows.cpp (created by 8177708) uses FLAG_SET_CMDLINE defined in runtime/globals_extension.hpp, but doesn't include this file neither explicitly nor implicitly. this affects only on build w/ disabled precompiled headers, and windows-x64-slowdebug is the only regularly windows build which has precompiled headers disabled. testing: windows-x64-slowdebug, tier1 webrev: http://cr.openjdk.java.net/~iignatyev//8213337/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8213337 Thanks, -- Igor From david.holmes at oracle.com Sat Nov 3 06:43:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 3 Nov 2018 16:43:07 +1000 Subject: RFR(XS/trivial) : 8213337 : windows-x64-slowdebug build is broken by 8177708 In-Reply-To: <75FB6703-69DB-424F-B09A-D102C6E2BBE4@oracle.com> References: <75FB6703-69DB-424F-B09A-D102C6E2BBE4@oracle.com> Message-ID: <9aecc03a-9c1e-33f0-bc8d-fe0630194766@oracle.com> Reviewed. Thanks, David On 3/11/2018 4:35 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8213337/webrev.00/index.html >> 1 line changed: 1 ins; 0 del; 0 mod; > > Hi all, > > could you please review this small and trivial fix? test/hotspot/gtest/runtime/test_os_windows.cpp (created by 8177708) uses FLAG_SET_CMDLINE defined in runtime/globals_extension.hpp, but doesn't include this file neither explicitly nor implicitly. this affects only on build w/ disabled precompiled headers, and windows-x64-slowdebug is the only regularly windows build which has precompiled headers disabled. > > testing: windows-x64-slowdebug, tier1 > webrev: http://cr.openjdk.java.net/~iignatyev//8213337/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8213337 > > Thanks, > -- Igor > From thomas.stuefe at gmail.com Sat Nov 3 06:59:51 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 3 Nov 2018 07:59:51 +0100 Subject: RFR(XS/trivial) : 8213337 : windows-x64-slowdebug build is broken by 8177708 In-Reply-To: <75FB6703-69DB-424F-B09A-D102C6E2BBE4@oracle.com> References: <75FB6703-69DB-424F-B09A-D102C6E2BBE4@oracle.com> Message-ID: Looks good. ..Thomas On Sat, Nov 3, 2018 at 7:36 AM Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8213337/webrev.00/index.html > > 1 line changed: 1 ins; 0 del; 0 mod; > > Hi all, > > could you please review this small and trivial fix? test/hotspot/gtest/runtime/test_os_windows.cpp (created by 8177708) uses FLAG_SET_CMDLINE defined in runtime/globals_extension.hpp, but doesn't include this file neither explicitly nor implicitly. this affects only on build w/ disabled precompiled headers, and windows-x64-slowdebug is the only regularly windows build which has precompiled headers disabled. > > testing: windows-x64-slowdebug, tier1 > webrev: http://cr.openjdk.java.net/~iignatyev//8213337/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8213337 > > Thanks, > -- Igor From magnus.ihse.bursie at oracle.com Sat Nov 3 09:06:19 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Sat, 3 Nov 2018 10:06:19 +0100 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency Message-ID: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> The reasons for the current set of files included in precompiled.hpp is somewhat lost in the mists of history. However, it is clear that it is not optimal. This patch replaces the current set with a new set, based on how often a header file is included in a C++ file. This selection contains all header files that are included by at least 130 C++ files. Testing has shown that this is around the optimal value -- include many more, and too many "innocent" files get hurt by unneeded work, leave out many more, and we miss out on optimization possibilities. The same set turned out to work well for both clang and gcc. However, files named "*.inline.hpp" did hurt rather than help performance, so those have been left out. For visual studio, the same set was also optimal, as long as the inline files were included. Presumably, visual studio is better than gcc/clang on handling precompiled headers containing inlined code. Here are some rough comparisons from our internal CI system, for building the target "hotspot" from scratch. macosx-x64: old: 00:05:00 new: 00:03:47 linux-x64: old: 00:05:43 new: 00:04:51 windows-x64: old: 00:05:18 new: 00:04:33 linux-aarch64: old: 00:07:57 new: 00:03:48 Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 /Magnus From erik.osterlund at oracle.com Sat Nov 3 09:13:05 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sat, 3 Nov 2018 10:13:05 +0100 Subject: RFR: 8212989: Allow CompiledMethod ExceptionCache have unloaded klasses In-Reply-To: <6f6332b1-2a61-6f86-9dd4-7d6d3e9622c2@oracle.com> References: <5BDAF881.3030905@oracle.com> <604b581e-00f2-24df-b06f-738bdefc593a@oracle.com> <6f6332b1-2a61-6f86-9dd4-7d6d3e9622c2@oracle.com> Message-ID: <62F789E9-05DE-4363-91EE-F6407A0DB570@oracle.com> Hi Vladimir, Thanks for the review. /Erik > On 2 Nov 2018, at 23:49, Vladimir Kozlov wrote: > >> On 11/1/18 3:14 PM, Erik ?sterlund wrote: >> Hi Vladimir, >> Thanks for the quick reply. >>> On 2018-11-01 21:14, Vladimir Kozlov wrote: >>> Hi Erik, >>> >>> It seems these changed are not based on latest sources. I don't see call to CodeCache::do_unloading in CodeCache::do_unloading() which is present in latest code: >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/5ea020bcaa0d/src/hotspot/share/code/codeCache.cpp#l683 >> Yeah this code is taken from the context of the future. Here is the exact same patch rebased against jdk-jdk: >> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01/ >> And new webrev with your suggestions: >> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.02/ > > Looks good. > >> And incremental of that to the jdk-jdk rebase: >> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01_02/ >>> >>> May be make some methods private, like CompiledMethod::exception_cache_acquire(). >> Sure. Made the accessors protected, as some are used in nmethod too and it seems fine for subclasses to access. >>> It seems you remove the only case when release_set_exception_cache() is called. >> That is true. Removed. >>> I don't see where purge_exception_caches() is used. >> It's only called by a future version of ZGC from its new concurrent class unloading feature that is currently baking in the ZGC repository. The release_exception_cache member function deletes entries synchronously if cleanup is triggered in a safepoint (all GCs except ZGC), and otherwise defers deletion to the purge_exception_caches() method, that is expected to happen one global handshake operation after the exception caches have been unlinked by cleanup. >>> Next sentence seems incomplete: >>> >>> + // unlinked. That is also when the CodeCache::exception_cache_free_list() >> Fixed. >>> Why you use exception_cache_acquire() in clean_exception_cache() and simple exception_cache() in add_exception_cache_entry()? >> The reason is that in the context of insertion, you are holding the lock protecting against concurrent insertions. Therefore, entries you load are either entries that were protected by the same lock (other insertions), or entries that are as old or older than entries that have died a whole safepoint operation earlier (due to prepending inserts), which makes its memory state stable. Therefore, no acquire is required when reading the head in the insertion path. > > Got it. > > Thanks, > Vladimir > >> Thanks, >> /Erik >>> Thanks, >>> Vladimir >>> >>>> On 11/1/18 5:58 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> The ExceptionCaches of compiled methods is traditionally pruned in safepoints during class unloading. This allows the exception cache to have lock-free reads, while performing inserts under a lock, and pruning under a safepoint. With concurrent class unloading, the exception caches need to be cleaned concurrently. >>>> >>>> To retain the lock-free reads that seem good to keep lock-free, a lock-free cleaning mechanism was introduced. Only one thread per compiled method cleans each exception cache, but that thread races with both single writers (under the ExceptionCache_lock) and any number of readers. >>>> >>>> The head of the exception cache list is concurrently cleaned by both inserting threads and cleaning threads. This allows having an invariant that no newly prepended entries ever produce next pointers to dead exception caches, that are concurrently removed from the list. As for the internal next pointers, they are only pruned by the one concurrent cleaning thread. This happens concurrent to reads that simply skip over dead entries as they have different Klass* pointers to the one being searched for. >>>> >>>> The single concurrent cleanup thread does not delete things removed from the list straight away. Instead, they are placed on a purge list that is freed up after a subsequent global handshaking operation. That allows ABA-free CAS instructions in the lock-free paths, and allows safe concurrent reading of entries in the exception cache list. >>>> >>>> Note that we already incorrectly "relied on memory_order_consume" for the head pointer. But it used a volatile load. Assuming volatile loads retain memory_order_consume semantics is not allowed in HotSpot runtime code, so I changed those loads to use acquire accordingly. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8212989 >>>> >>>> Thanks, >>>> /Erik From shade at redhat.com Sat Nov 3 09:57:21 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 3 Nov 2018 10:57:21 +0100 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> Message-ID: <49e315ef-041a-e12f-b24d-a4201b1dedc7@redhat.com> On 11/03/2018 10:06 AM, Magnus Ihse Bursie wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 > WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 This looks good to me. Thanks, -Aleksey From david.holmes at oracle.com Sat Nov 3 12:09:28 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 3 Nov 2018 22:09:28 +1000 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> Message-ID: <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> Looks okay - thanks for doing all the experiments! (Though I'm still curious what happens if you recompile individual header files :) ). 25 // Precompiled headers are turned off for Sun Studio, May as well change to Solaris Studio if you're going to fix the typo :) Thanks. David On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: > The reasons for the current set of files included in precompiled.hpp is > somewhat lost in the mists of history. However, it is clear that it is > not optimal. > > This patch replaces the current set with a new set, based on how often a > header file is included in a C++ file. This selection contains all > header files that are included by at least 130 C++ files. Testing has > shown that this is around the optimal value -- include many more, and > too many "innocent" files get hurt by unneeded work, leave out many > more, and we miss out on optimization possibilities. > > The same set turned out to work well for both clang and gcc. However, > files named "*.inline.hpp" did hurt rather than help performance, so > those have been left out. For visual studio, the same set was also > optimal, as long as the inline files were included. Presumably, visual > studio is better than gcc/clang on handling precompiled headers > containing inlined code. > > Here are some rough comparisons from our internal CI system, for > building the target "hotspot" from scratch. > > macosx-x64: > old: 00:05:00 > new: 00:03:47 > > linux-x64: > old: 00:05:43 > new: 00:04:51 > > windows-x64: > old: 00:05:18 > new: 00:04:33 > > linux-aarch64: > old: 00:07:57 > new: 00:03:48 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 > WebRev: > http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 > > > /Magnus From dean.long at oracle.com Sat Nov 3 20:00:30 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Sat, 3 Nov 2018 13:00:30 -0700 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> Message-ID: <46bff210-18fb-474e-42f3-0a277d955b84@oracle.com> I made a pass at improving the comments based on feedback I've received.? I updated webrev.4 in place, along with an incremental diff: http://cr.openjdk.java.net/~dlong/8212605/webrev.4.update/ dl On 10/31/18 9:39 PM, Bernd Eckenfels wrote: > I find the tail call optimization comment in wrapException adds only > confusion to an otherwise clear helper. But maybe it?s just me who > does not understand it. From ioi.lam at oracle.com Sat Nov 3 22:07:05 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 3 Nov 2018 15:07:05 -0700 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <47520602-3945-9691-9c3b-217153a362a6@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> <47520602-3945-9691-9c3b-217153a362a6@oracle.com> Message-ID: <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> On 11/2/18 10:42 PM, Jiangli Zhou wrote: > Hi Ioi, > > > On 11/2/18 5:27 PM, Ioi Lam wrote: >> Hi Jiangli, >> >> Thanks for the review >> >> >> On 11/2/18 1:24 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> - src/hotspot/share/classfile/verifier.cpp >>> >>> You moved >>> SystemDictionaryShared::finalize_verification_constraints(_klass) to >>> ClassVerifier::verify_class(). We should be very careful in this >>> area. Have you tested with cases where classes fail verification? >> >> void ClassVerifier::verify_class(TRAPS) { >> ? log_info(verification)("Verifying class %s with new format", >> _klass->external_name()); >> >> ? Array* methods = _klass->methods(); >> ? int num_methods = methods->length(); >> >> ? for (int index = 0; index < num_methods; index++) { >> ??? // Check for recursive re-verification before each method. >> ??? if (was_recursively_verified())? return; >> >> ??? Method* m = methods->at(index); >> ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { >> ????? // If m is native or abstract, skip it.? It is checked in class >> file >> ????? // parser that methods do not override a final method. Overpass >> methods >> ????? // are trusted since the VM generates them. >> ????? continue; >> ??? } >> ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); >> ? } >> >> ? if (DumpSharedSpaces) { >> SystemDictionaryShared::finalize_verification_constraints(_klass); >> ? } >> >> If verification failed, the CHECK_VERIFY macro will force the >> function to return. So we will call finalize_verification_constraints >> only when verification has succeeded. > > Ok, thanks for checking. > > An alternative is to allocate from the RO space directly after > relocation. That way you can avoid adding another CDS specific in the > verifier code. There is also a FIXME for using the symbol offset in > SharedDictionaryEntry::finalize_verification_constraints, which you > could handle together. I'll leave this patch simple so it can easily be backported to JDK 11. I'll do what you suggested and deal with the FIXME in a separate RFE (https://bugs.openjdk.java.net/browse/JDK-8213346). >> >> >>> I haven't ran your code yet, we should avoid copying the constraints >>> for classes that fails verification at dump time. >> >> If a class has failed verification, the class will not be archived. >> So whether we have called finalize_verification_constraints on it >> really doesn't matter. > > Since it doesn't do the copying when verification fails, it is okay in > this case. With the work for dynamic archiving, we want to avoid > unnecessary work at dump time, since performance/footprint will become > higher priority with archiving at runtime. >> >>> I also need to think through any potential impact on dynamic >>> archiving (possibly none). >>> >>> - src/hotspot/share/memory/metaspace.cpp >>> >>> -??? if (is_init_completed() && !(DumpSharedSpaces && >>> THREAD->is_VM_thread())) { >>> +??? if (is_init_completed()) { >>> >>> Please also add an assert to make sure we don't allocate metaspace >>> objects from the VM_thread at dump time now. >>> >> >> I think it's better to add a blanket assert(!THREAD->is_VM_thread()), >> because no one should be allocating metaspace objects inside VM thread. >> >> Adding this has a wider impact, so I will do it in a separate RFE >> (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we need >> to back it out. > > I'd suggest including the assert for dump time with the fix so we are > sure it is safe. Then you can follow up with the broader assert that > you are planning for JDK-8213331. > I see. I've added MetaWord* Metaspace::allocate(ClassLoaderData* loader_data, size_t word_size, ????????????????????????????? MetaspaceObj::Type type, TRAPS) { ? assert(!_frozen, "sanity"); + assert(!(DumpSharedSpaces && THREAD->is_VM_thread()), "sanity"); I'll just remove the (DumpSharedSpaces &&) in JDK-8213331. Thanks - Ioi > Thanks, > Jiangli >> >> Thanks >> - Ioi >>> Thanks, >>> >>> Jiangli >>> >>> >>> On 11/1/18 9:47 PM, Ioi Lam wrote: >>>> Hi, >>>> >>>> I've revised the webrev to get to the the root of the problem -- we >>>> shouldn't allocate metaspace objects from within the VM Thread! >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>>> >>>> >>>> The original bug that caused the "!(DumpSharedSpaces && >>>> THREAD->is_VM_thread())" check to be added in Metaspace::allocate() >>>> was >>>> >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >>>> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock >>>> (not allowed to block on)" >>>> >>>> The fix in JDK-8196626 is just a band-aid solution -- avoiding the >>>> GC. The proper fix is not to do any allocation inside the VM >>>> thread. So instead of calling finalize_verification_constraints, >>>> which allocates MetaspaceObjs, very late in the CDS dumping >>>> process, now we call be as soon as a class has been verified. >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> >>>> >>>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>>> >>>>> >>>>> When Metaspace::allocate() fails, usually we first try to GC to >>>>> see if >>>>> some MetaspaceObjs can be freed. This is an optimization to avoid >>>>> prematurely >>>>> expanding the metaspace. >>>>> >>>>> However, GC cannot run in the last phase of CDS which runs inside >>>>> the VM thread. >>>>> The old code just aborts the CDS dump. The fix is to >>>>> unconditionally expand the >>>>> metaspace without doing a GC. >>>>> >>>>> (During GC dump time, no classes are unloaded, so even if we force >>>>> a GC, it would >>>>> not free up any metaspace anyway). >>>>> >>>>> Testing: >>>>> >>>>> Before the fix, the test case (jruby) would fail to dump once a >>>>> couple of times. Now >>>>> it has dumped hundreds of times without failure. >>>>> >>>>> I am also running hs-tier 1/2 >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> >> > From david.holmes at oracle.com Sat Nov 3 22:24:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 4 Nov 2018 08:24:15 +1000 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> Message-ID: On 3/11/2018 10:09 PM, David Holmes wrote: > Looks okay - thanks for doing all the experiments! (Though I'm still > curious what happens if you recompile individual header files :) ). s/recompile/precompile/ :) David > ?25 // Precompiled headers are turned off for Sun Studio, > > May as well change to Solaris Studio if you're going to fix the typo :) > > Thanks. > David > > On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: >> The reasons for the current set of files included in precompiled.hpp >> is somewhat lost in the mists of history. However, it is clear that it >> is not optimal. >> >> This patch replaces the current set with a new set, based on how often >> a header file is included in a C++ file. This selection contains all >> header files that are included by at least 130 C++ files. Testing has >> shown that this is around the optimal value -- include many more, and >> too many "innocent" files get hurt by unneeded work, leave out many >> more, and we miss out on optimization possibilities. >> >> The same set turned out to work well for both clang and gcc. However, >> files named "*.inline.hpp" did hurt rather than help performance, so >> those have been left out. For visual studio, the same set was also >> optimal, as long as the inline files were included. Presumably, visual >> studio is better than gcc/clang on handling precompiled headers >> containing inlined code. >> >> Here are some rough comparisons from our internal CI system, for >> building the target "hotspot" from scratch. >> >> macosx-x64: >> old: 00:05:00 >> new: 00:03:47 >> >> linux-x64: >> old: 00:05:43 >> new: 00:04:51 >> >> windows-x64: >> old: 00:05:18 >> new: 00:04:33 >> >> linux-aarch64: >> old: 00:07:57 >> new: 00:03:48 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 >> WebRev: >> http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 >> >> >> /Magnus From magnus.ihse.bursie at oracle.com Sun Nov 4 07:27:55 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Sun, 4 Nov 2018 08:27:55 +0100 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> Message-ID: <42E17EFC-8E6C-4032-A136-529A96E72E93@oracle.com> > 3 nov. 2018 kl. 23:24 skrev David Holmes : > >> On 3/11/2018 10:09 PM, David Holmes wrote: >> Looks okay - thanks for doing all the experiments! (Though I'm still curious what happens if you recompile individual header files :) ). > > s/recompile/precompile/ :) What do you mean? To have more than one set of PCH? As far as I know, all compilers we use only support a single PCH per source file to compile. So while you could have a PCH tailored per source file, that would only make sense in a scenario were you recompile all source files (but no header files) often. /Magnus > > David > >> 25 // Precompiled headers are turned off for Sun Studio, >> May as well change to Solaris Studio if you're going to fix the typo :) >> Thanks. >> David >>> On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: >>> The reasons for the current set of files included in precompiled.hpp is somewhat lost in the mists of history. However, it is clear that it is not optimal. >>> >>> This patch replaces the current set with a new set, based on how often a header file is included in a C++ file. This selection contains all header files that are included by at least 130 C++ files. Testing has shown that this is around the optimal value -- include many more, and too many "innocent" files get hurt by unneeded work, leave out many more, and we miss out on optimization possibilities. >>> >>> The same set turned out to work well for both clang and gcc. However, files named "*.inline.hpp" did hurt rather than help performance, so those have been left out. For visual studio, the same set was also optimal, as long as the inline files were included. Presumably, visual studio is better than gcc/clang on handling precompiled headers containing inlined code. >>> >>> Here are some rough comparisons from our internal CI system, for building the target "hotspot" from scratch. >>> >>> macosx-x64: >>> old: 00:05:00 >>> new: 00:03:47 >>> >>> linux-x64: >>> old: 00:05:43 >>> new: 00:04:51 >>> >>> windows-x64: >>> old: 00:05:18 >>> new: 00:04:33 >>> >>> linux-aarch64: >>> old: 00:07:57 >>> new: 00:03:48 >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 >>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 >>> >>> /Magnus From Alan.Bateman at oracle.com Sun Nov 4 09:03:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 4 Nov 2018 09:03:39 +0000 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <46bff210-18fb-474e-42f3-0a277d955b84@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <46bff210-18fb-474e-42f3-0a277d955b84@oracle.com> Message-ID: <3f15e776-4aae-b3a7-302a-0c451d73a0c1@oracle.com> On 03/11/2018 20:00, dean.long at oracle.com wrote: > I made a pass at improving the comments based on feedback I've > received.? I updated webrev.4 in place, along with an incremental diff: I looked through the updated webrev.4, mostly studying the changes in AccessController.java and jvm.cpp and the changes look good to me. -Alan. From jiangli.zhou at oracle.com Sun Nov 4 20:12:31 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Sun, 4 Nov 2018 12:12:31 -0800 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> <47520602-3945-9691-9c3b-217153a362a6@oracle.com> <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> Message-ID: On 11/3/18 3:07 PM, Ioi Lam wrote: > > > On 11/2/18 10:42 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> >> On 11/2/18 5:27 PM, Ioi Lam wrote: >>> Hi Jiangli, >>> >>> Thanks for the review >>> >>> >>> On 11/2/18 1:24 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> - src/hotspot/share/classfile/verifier.cpp >>>> >>>> You moved >>>> SystemDictionaryShared::finalize_verification_constraints(_klass) >>>> to ClassVerifier::verify_class(). We should be very careful in this >>>> area. Have you tested with cases where classes fail verification? >>> >>> void ClassVerifier::verify_class(TRAPS) { >>> ? log_info(verification)("Verifying class %s with new format", >>> _klass->external_name()); >>> >>> ? Array* methods = _klass->methods(); >>> ? int num_methods = methods->length(); >>> >>> ? for (int index = 0; index < num_methods; index++) { >>> ??? // Check for recursive re-verification before each method. >>> ??? if (was_recursively_verified())? return; >>> >>> ??? Method* m = methods->at(index); >>> ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { >>> ????? // If m is native or abstract, skip it.? It is checked in >>> class file >>> ????? // parser that methods do not override a final method. >>> Overpass methods >>> ????? // are trusted since the VM generates them. >>> ????? continue; >>> ??? } >>> ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); >>> ? } >>> >>> ? if (DumpSharedSpaces) { >>> SystemDictionaryShared::finalize_verification_constraints(_klass); >>> ? } >>> >>> If verification failed, the CHECK_VERIFY macro will force the >>> function to return. So we will call >>> finalize_verification_constraints only when verification has succeeded. >> >> Ok, thanks for checking. >> >> An alternative is to allocate from the RO space directly after >> relocation. That way you can avoid adding another CDS specific in the >> verifier code. There is also a FIXME for using the symbol offset in >> SharedDictionaryEntry::finalize_verification_constraints, which you >> could handle together. > > I'll leave this patch simple so it can easily be backported to JDK 11. > > I'll do what you suggested and deal with the FIXME in a separate RFE > (https://bugs.openjdk.java.net/browse/JDK-8213346). Follow up with a separate RFE sounds good to me. For backporting to JDK 11, the risk of the fix seems to be higher than the benefit (considering the nature of the issue). Maybe we should have the proper fix backported instead (if backporting is required)? > >>> >>> >>>> I haven't ran your code yet, we should avoid copying the >>>> constraints for classes that fails verification at dump time. >>> >>> If a class has failed verification, the class will not be archived. >>> So whether we have called finalize_verification_constraints on it >>> really doesn't matter. >> >> Since it doesn't do the copying when verification fails, it is okay >> in this case. With the work for dynamic archiving, we want to avoid >> unnecessary work at dump time, since performance/footprint will >> become higher priority with archiving at runtime. >>> >>>> I also need to think through any potential impact on dynamic >>>> archiving (possibly none). >>>> >>>> - src/hotspot/share/memory/metaspace.cpp >>>> >>>> -??? if (is_init_completed() && !(DumpSharedSpaces && >>>> THREAD->is_VM_thread())) { >>>> +??? if (is_init_completed()) { >>>> >>>> Please also add an assert to make sure we don't allocate metaspace >>>> objects from the VM_thread at dump time now. >>>> >>> >>> I think it's better to add a blanket >>> assert(!THREAD->is_VM_thread()), because no one should be allocating >>> metaspace objects inside VM thread. >>> >>> Adding this has a wider impact, so I will do it in a separate RFE >>> (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we need >>> to back it out. >> >> I'd suggest including the assert for dump time with the fix so we are >> sure it is safe. Then you can follow up with the broader assert that >> you are planning for JDK-8213331. >> > > I see. I've added > > MetaWord* Metaspace::allocate(ClassLoaderData* loader_data, size_t > word_size, > ????????????????????????????? MetaspaceObj::Type type, TRAPS) { > ? assert(!_frozen, "sanity"); > + assert(!(DumpSharedSpaces && THREAD->is_VM_thread()), "sanity"); > > I'll just remove the (DumpSharedSpaces &&) in JDK-8213331. Thumbs up. Thanks, Jiangli > > Thanks > - Ioi > >> Thanks, >> Jiangli >>> >>> Thanks >>> - Ioi >>>> Thanks, >>>> >>>> Jiangli >>>> >>>> >>>> On 11/1/18 9:47 PM, Ioi Lam wrote: >>>>> Hi, >>>>> >>>>> I've revised the webrev to get to the the root of the problem -- >>>>> we shouldn't allocate metaspace objects from within the VM Thread! >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>>>> >>>>> >>>>> The original bug that caused the "!(DumpSharedSpaces && >>>>> THREAD->is_VM_thread())" check to be added in >>>>> Metaspace::allocate() was >>>>> >>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >>>>> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock >>>>> (not allowed to block on)" >>>>> >>>>> The fix in JDK-8196626 is just a band-aid solution -- avoiding the >>>>> GC. The proper fix is not to do any allocation inside the VM >>>>> thread. So instead of calling finalize_verification_constraints, >>>>> which allocates MetaspaceObjs, very late in the CDS dumping >>>>> process, now we call be as soon as a class has been verified. >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>>>> >>>>>> >>>>>> When Metaspace::allocate() fails, usually we first try to GC to >>>>>> see if >>>>>> some MetaspaceObjs can be freed. This is an optimization to avoid >>>>>> prematurely >>>>>> expanding the metaspace. >>>>>> >>>>>> However, GC cannot run in the last phase of CDS which runs inside >>>>>> the VM thread. >>>>>> The old code just aborts the CDS dump. The fix is to >>>>>> unconditionally expand the >>>>>> metaspace without doing a GC. >>>>>> >>>>>> (During GC dump time, no classes are unloaded, so even if we >>>>>> force a GC, it would >>>>>> not free up any metaspace anyway). >>>>>> >>>>>> Testing: >>>>>> >>>>>> Before the fix, the test case (jruby) would fail to dump once a >>>>>> couple of times. Now >>>>>> it has dumped hundreds of times without failure. >>>>>> >>>>>> I am also running hs-tier 1/2 >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>> >>>> >>> >> > From david.holmes at oracle.com Sun Nov 4 21:38:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 5 Nov 2018 07:38:10 +1000 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BC9E8BA.50407@oracle.com> References: <5BC9E8BA.50407@oracle.com> Message-ID: Hi Erik, Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. Thanks, David On 20/10/2018 12:22 AM, Erik ?sterlund wrote: > Hi, > > Today the nmethods are unloaded either serially or in parallel due to GC > (and e.g. class unloading). With ZGC concurrent class unloading, a > concurrent fashion is required. Rather than inventing yet a third way of > unloading nmethods due to class unloading, it would be ideal if there > could be one unified way of doing it that makes everyone happy. > > To solve this problem in a more general way, a new member function on > CompiledMethod is added: is_unloading(). It returns whether a > CompiledMethod has broken oops. In order to not have to iterate over all > the oops every time this question is asked, it caches the result, so it > needs to be computed only once per "epoch". Every time a CodeCache > unloading is triggered by the GC, a new epoch is started, which forces > re-computation of whether the CompiledMethod is_unloading() the first > time it is called. > > So by having is_unloading() be lazily evaluated, it is now possible to > build a do_unloading() method on nmethod that simply makes the nmethod > unloaded if it is_unloading(), and otherwise cleans the various caches. > Cleaning e.g. the inline caches of said nmethod, uses the same > is_unloading() method on its targets to figure out if the IC cache > should be cleaned or not. Again, the epoched caching mechanism makes > sure we don't recompute this value. > > The new do_unloading() method may be called in both serial, parallel and > concurrent contexts. So I removed the parallel variation of this that we > had before, that unnecessarily postponed the unloading due to not having > computed whether the nmethod should be unloaded or not. Since that is > now computed on-demand lazily, there is no longer a need for postponing > this, nor to have two phases for parallel nmethod unloading. > > While there is more work involved in making concurrent nmethod unloading > work, this is a good cleanup towards that goal. > > Bug ID: > https://bugs.openjdk.java.net/browse/JDK-8209189 > > Webrev: > cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ > > Thanks, > /Erik From david.holmes at oracle.com Sun Nov 4 22:07:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 5 Nov 2018 08:07:05 +1000 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: <42E17EFC-8E6C-4032-A136-529A96E72E93@oracle.com> References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> <42E17EFC-8E6C-4032-A136-529A96E72E93@oracle.com> Message-ID: <507b7651-f9fc-65f1-9ee4-9f6fe391246f@oracle.com> On 4/11/2018 5:27 PM, Magnus Ihse Bursie wrote: > >> 3 nov. 2018 kl. 23:24 skrev David Holmes : >> >>> On 3/11/2018 10:09 PM, David Holmes wrote: >>> Looks okay - thanks for doing all the experiments! (Though I'm still curious what happens if you recompile individual header files :) ). >> >> s/recompile/precompile/ :) > > What do you mean? To have more than one set of PCH? As far as I know, all compilers we use only support a single PCH per source file to compile. So while you could have a PCH tailored per source file, that would only make sense in a scenario were you recompile all source files (but no header files) often. The GCC description for PCH describes it as a per-file header file optimization. You can precompile each .hpp file and if it is located in the same directory as the .hpp (or is on an include directory ahead of the .hpp) then the PCH file will be used instead. So rather than precompiling precompiled.hpp and having source files include it, you would precompile each individual (chosen) header file and have that in the special PCH include directory, and they would then be included where a source file asked for the corresponding .hpp file. That would also remove the problem of things not compiling with PCH disabled as a single set of includes would be used for both PCH and non-PCH building. I don't know if the other compilers support PCH in a similar manner to gcc. Cheers, David > /Magnus > >> >> David >> >>> 25 // Precompiled headers are turned off for Sun Studio, >>> May as well change to Solaris Studio if you're going to fix the typo :) >>> Thanks. >>> David >>>> On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: >>>> The reasons for the current set of files included in precompiled.hpp is somewhat lost in the mists of history. However, it is clear that it is not optimal. >>>> >>>> This patch replaces the current set with a new set, based on how often a header file is included in a C++ file. This selection contains all header files that are included by at least 130 C++ files. Testing has shown that this is around the optimal value -- include many more, and too many "innocent" files get hurt by unneeded work, leave out many more, and we miss out on optimization possibilities. >>>> >>>> The same set turned out to work well for both clang and gcc. However, files named "*.inline.hpp" did hurt rather than help performance, so those have been left out. For visual studio, the same set was also optimal, as long as the inline files were included. Presumably, visual studio is better than gcc/clang on handling precompiled headers containing inlined code. >>>> >>>> Here are some rough comparisons from our internal CI system, for building the target "hotspot" from scratch. >>>> >>>> macosx-x64: >>>> old: 00:05:00 >>>> new: 00:03:47 >>>> >>>> linux-x64: >>>> old: 00:05:43 >>>> new: 00:04:51 >>>> >>>> windows-x64: >>>> old: 00:05:18 >>>> new: 00:04:33 >>>> >>>> linux-aarch64: >>>> old: 00:07:57 >>>> new: 00:03:48 >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 >>>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 >>>> >>>> /Magnus > From ioi.lam at oracle.com Sun Nov 4 22:29:11 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 4 Nov 2018 14:29:11 -0800 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> <47520602-3945-9691-9c3b-217153a362a6@oracle.com> <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> Message-ID: <11b40c83-f15b-d1f0-2268-d6d05275f89e@oracle.com> On 11/4/18 12:12 PM, Jiangli Zhou wrote: > On 11/3/18 3:07 PM, Ioi Lam wrote: > >> >> >> On 11/2/18 10:42 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> >>> On 11/2/18 5:27 PM, Ioi Lam wrote: >>>> Hi Jiangli, >>>> >>>> Thanks for the review >>>> >>>> >>>> On 11/2/18 1:24 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> - src/hotspot/share/classfile/verifier.cpp >>>>> >>>>> You moved >>>>> SystemDictionaryShared::finalize_verification_constraints(_klass) >>>>> to ClassVerifier::verify_class(). We should be very careful in >>>>> this area. Have you tested with cases where classes fail >>>>> verification? >>>> >>>> void ClassVerifier::verify_class(TRAPS) { >>>> ? log_info(verification)("Verifying class %s with new format", >>>> _klass->external_name()); >>>> >>>> ? Array* methods = _klass->methods(); >>>> ? int num_methods = methods->length(); >>>> >>>> ? for (int index = 0; index < num_methods; index++) { >>>> ??? // Check for recursive re-verification before each method. >>>> ??? if (was_recursively_verified())? return; >>>> >>>> ??? Method* m = methods->at(index); >>>> ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { >>>> ????? // If m is native or abstract, skip it.? It is checked in >>>> class file >>>> ????? // parser that methods do not override a final method. >>>> Overpass methods >>>> ????? // are trusted since the VM generates them. >>>> ????? continue; >>>> ??? } >>>> ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); >>>> ? } >>>> >>>> ? if (DumpSharedSpaces) { >>>> SystemDictionaryShared::finalize_verification_constraints(_klass); >>>> ? } >>>> >>>> If verification failed, the CHECK_VERIFY macro will force the >>>> function to return. So we will call >>>> finalize_verification_constraints only when verification has >>>> succeeded. >>> >>> Ok, thanks for checking. >>> >>> An alternative is to allocate from the RO space directly after >>> relocation. That way you can avoid adding another CDS specific in >>> the verifier code. There is also a FIXME for using the symbol offset >>> in SharedDictionaryEntry::finalize_verification_constraints, which >>> you could handle together. >> >> I'll leave this patch simple so it can easily be backported to JDK 11. >> >> I'll do what you suggested and deal with the FIXME in a separate RFE >> (https://bugs.openjdk.java.net/browse/JDK-8213346). > > Follow up with a separate RFE sounds good to me. For backporting to > JDK 11, the risk of the fix seems to be higher than the benefit > (considering the nature of the issue). Maybe we should have the proper > fix backported instead (if backporting is required)? > With JDK 11, if you have more than a few thousand classes, -Xshare:dump will randomly fail. The workaround is to explicitly set -XX:MetaspaceSize=128m. I think we should fix it in JDK 11. The fix in the current RFR is pretty harmless. It just moves the allocation of the _verifier_constraints earlier. Thanks - Ioi >> >>>> >>>> >>>>> I haven't ran your code yet, we should avoid copying the >>>>> constraints for classes that fails verification at dump time. >>>> >>>> If a class has failed verification, the class will not be archived. >>>> So whether we have called finalize_verification_constraints on it >>>> really doesn't matter. >>> >>> Since it doesn't do the copying when verification fails, it is okay >>> in this case. With the work for dynamic archiving, we want to avoid >>> unnecessary work at dump time, since performance/footprint will >>> become higher priority with archiving at runtime. >>>> >>>>> I also need to think through any potential impact on dynamic >>>>> archiving (possibly none). >>>>> >>>>> - src/hotspot/share/memory/metaspace.cpp >>>>> >>>>> -??? if (is_init_completed() && !(DumpSharedSpaces && >>>>> THREAD->is_VM_thread())) { >>>>> +??? if (is_init_completed()) { >>>>> >>>>> Please also add an assert to make sure we don't allocate metaspace >>>>> objects from the VM_thread at dump time now. >>>>> >>>> >>>> I think it's better to add a blanket >>>> assert(!THREAD->is_VM_thread()), because no one should be >>>> allocating metaspace objects inside VM thread. >>>> >>>> Adding this has a wider impact, so I will do it in a separate RFE >>>> (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we need >>>> to back it out. >>> >>> I'd suggest including the assert for dump time with the fix so we >>> are sure it is safe. Then you can follow up with the broader assert >>> that you are planning for JDK-8213331. >>> >> >> I see. I've added >> >> MetaWord* Metaspace::allocate(ClassLoaderData* loader_data, size_t >> word_size, >> ????????????????????????????? MetaspaceObj::Type type, TRAPS) { >> ? assert(!_frozen, "sanity"); >> + assert(!(DumpSharedSpaces && THREAD->is_VM_thread()), "sanity"); >> >> I'll just remove the (DumpSharedSpaces &&) in JDK-8213331. > > Thumbs up. > > Thanks, > Jiangli > >> >> Thanks >> - Ioi >> >>> Thanks, >>> Jiangli >>>> >>>> Thanks >>>> - Ioi >>>>> Thanks, >>>>> >>>>> Jiangli >>>>> >>>>> >>>>> On 11/1/18 9:47 PM, Ioi Lam wrote: >>>>>> Hi, >>>>>> >>>>>> I've revised the webrev to get to the the root of the problem -- >>>>>> we shouldn't allocate metaspace objects from within the VM Thread! >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>>>>> >>>>>> >>>>>> The original bug that caused the "!(DumpSharedSpaces && >>>>>> THREAD->is_VM_thread())" check to be added in >>>>>> Metaspace::allocate() was >>>>>> >>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >>>>>> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock >>>>>> (not allowed to block on)" >>>>>> >>>>>> The fix in JDK-8196626 is just a band-aid solution -- avoiding >>>>>> the GC. The proper fix is not to do any allocation inside the VM >>>>>> thread. So instead of calling finalize_verification_constraints, >>>>>> which allocates MetaspaceObjs, very late in the CDS dumping >>>>>> process, now we call be as soon as a class has been verified. >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>>>>> >>>>>>> >>>>>>> When Metaspace::allocate() fails, usually we first try to GC to >>>>>>> see if >>>>>>> some MetaspaceObjs can be freed. This is an optimization to >>>>>>> avoid prematurely >>>>>>> expanding the metaspace. >>>>>>> >>>>>>> However, GC cannot run in the last phase of CDS which runs >>>>>>> inside the VM thread. >>>>>>> The old code just aborts the CDS dump. The fix is to >>>>>>> unconditionally expand the >>>>>>> metaspace without doing a GC. >>>>>>> >>>>>>> (During GC dump time, no classes are unloaded, so even if we >>>>>>> force a GC, it would >>>>>>> not free up any metaspace anyway). >>>>>>> >>>>>>> Testing: >>>>>>> >>>>>>> Before the fix, the test case (jruby) would fail to dump once a >>>>>>> couple of times. Now >>>>>>> it has dumped hundreds of times without failure. >>>>>>> >>>>>>> I am also running hs-tier 1/2 >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Mon Nov 5 01:43:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 5 Nov 2018 11:43:32 +1000 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances Message-ID: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> This impacts GC, compiler, runtime and serviceability. Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ To facilitate changes to the Mutex/Monitor code we need to ensure there are no statically initialized Monitor/Mutex instances in the VM - as the constructors may rely on VM initialization that has not happened when C++ initializers execute. There were 6 locally defined "lock" members of classes that were statically initialized, and these are across all areas of the VM: - Decoder::_shared_decoder_lock - DCmdFactory::_dcmdFactory_lock - JfrThreadSampler::_transition_block_lock - NMethodSweeper::_stat_lock - ThreadsSMRSupport::_delete_lock - CodeCacheUnloadingTask::_lock The last is actually now unused and so is just deleted. The rest join all other Monitor/Mutex instances defined in the global list in MutexLocker and initialized in mutex_init(). I've verified that none of the monitor/mutex instances can be used prior to mutex_init, by code inspection - more details in the bug report. I investigated asserting that a Monitor/Mutex is only constructed during mutex_init() but assertions don't fail cleanly during C++ static initialization - the result is that during the build jmod goes into an infinite loop printing out "[Too many errors, abort]". Tested using tier1-3 Thanks, David From erik.osterlund at oracle.com Mon Nov 5 06:19:58 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 5 Nov 2018 07:19:58 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: References: <5BC9E8BA.50407@oracle.com> Message-ID: <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> Hi David, Yes that lock is now unused. Sorry I forgot to delete it! /Erik > On 4 Nov 2018, at 22:38, David Holmes wrote: > > Hi Erik, > > Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) > > I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. > > Thanks, > David > >> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >> Hi, >> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >> Bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8209189 >> Webrev: >> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >> Thanks, >> /Erik From thomas.stuefe at gmail.com Mon Nov 5 07:17:24 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 5 Nov 2018 08:17:24 +0100 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: <507b7651-f9fc-65f1-9ee4-9f6fe391246f@oracle.com> References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> <42E17EFC-8E6C-4032-A136-529A96E72E93@oracle.com> <507b7651-f9fc-65f1-9ee4-9f6fe391246f@oracle.com> Message-ID: Hi David, On Sun, Nov 4, 2018 at 11:09 PM David Holmes wrote: > > On 4/11/2018 5:27 PM, Magnus Ihse Bursie wrote: > > > >> 3 nov. 2018 kl. 23:24 skrev David Holmes : > >> > >>> On 3/11/2018 10:09 PM, David Holmes wrote: > >>> Looks okay - thanks for doing all the experiments! (Though I'm still curious what happens if you recompile individual header files :) ). > >> > >> s/recompile/precompile/ :) > > > > What do you mean? To have more than one set of PCH? As far as I know, all compilers we use only support a single PCH per source file to compile. So while you could have a PCH tailored per source file, that would only make sense in a scenario were you recompile all source files (but no header files) often. > > The GCC description for PCH describes it as a per-file header file > optimization. You can precompile each .hpp file and if it is located in > the same directory as the .hpp (or is on an include directory ahead of > the .hpp) then the PCH file will be used instead. So rather than > precompiling precompiled.hpp and having source files include it, you > would precompile each individual (chosen) header file and have that in > the special PCH include directory, and they would then be included where > a source file asked for the corresponding .hpp file. > I assume precompiling a header means precompiling it itself and any header it includes, no? In that case, would precompiling each header individually not cause us to implicitly precompile included headers many times (e.g. globalDefinitions.hpp)? Thanks, Thomas > That would also remove the problem of things not compiling with PCH > disabled as a single set of includes would be used for both PCH and > non-PCH building. > > I don't know if the other compilers support PCH in a similar manner to gcc. > > Cheers, > David > > > /Magnus > > > >> > >> David > >> > >>> 25 // Precompiled headers are turned off for Sun Studio, > >>> May as well change to Solaris Studio if you're going to fix the typo :) > >>> Thanks. > >>> David > >>>> On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: > >>>> The reasons for the current set of files included in precompiled.hpp is somewhat lost in the mists of history. However, it is clear that it is not optimal. > >>>> > >>>> This patch replaces the current set with a new set, based on how often a header file is included in a C++ file. This selection contains all header files that are included by at least 130 C++ files. Testing has shown that this is around the optimal value -- include many more, and too many "innocent" files get hurt by unneeded work, leave out many more, and we miss out on optimization possibilities. > >>>> > >>>> The same set turned out to work well for both clang and gcc. However, files named "*.inline.hpp" did hurt rather than help performance, so those have been left out. For visual studio, the same set was also optimal, as long as the inline files were included. Presumably, visual studio is better than gcc/clang on handling precompiled headers containing inlined code. > >>>> > >>>> Here are some rough comparisons from our internal CI system, for building the target "hotspot" from scratch. > >>>> > >>>> macosx-x64: > >>>> old: 00:05:00 > >>>> new: 00:03:47 > >>>> > >>>> linux-x64: > >>>> old: 00:05:43 > >>>> new: 00:04:51 > >>>> > >>>> windows-x64: > >>>> old: 00:05:18 > >>>> new: 00:04:33 > >>>> > >>>> linux-aarch64: > >>>> old: 00:07:57 > >>>> new: 00:03:48 > >>>> > >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 > >>>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 > >>>> > >>>> /Magnus > > From david.holmes at oracle.com Mon Nov 5 07:28:48 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 5 Nov 2018 17:28:48 +1000 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> <42E17EFC-8E6C-4032-A136-529A96E72E93@oracle.com> <507b7651-f9fc-65f1-9ee4-9f6fe391246f@oracle.com> Message-ID: <10a9ab67-81c6-3cf2-9f42-817574b4f4dc@oracle.com> Hi Thomas, On 5/11/2018 5:17 PM, Thomas St?fe wrote: > Hi David, > > On Sun, Nov 4, 2018 at 11:09 PM David Holmes wrote: >> >> On 4/11/2018 5:27 PM, Magnus Ihse Bursie wrote: >>> >>>> 3 nov. 2018 kl. 23:24 skrev David Holmes : >>>> >>>>> On 3/11/2018 10:09 PM, David Holmes wrote: >>>>> Looks okay - thanks for doing all the experiments! (Though I'm still curious what happens if you recompile individual header files :) ). >>>> >>>> s/recompile/precompile/ :) >>> >>> What do you mean? To have more than one set of PCH? As far as I know, all compilers we use only support a single PCH per source file to compile. So while you could have a PCH tailored per source file, that would only make sense in a scenario were you recompile all source files (but no header files) often. >> >> The GCC description for PCH describes it as a per-file header file >> optimization. You can precompile each .hpp file and if it is located in >> the same directory as the .hpp (or is on an include directory ahead of >> the .hpp) then the PCH file will be used instead. So rather than >> precompiling precompiled.hpp and having source files include it, you >> would precompile each individual (chosen) header file and have that in >> the special PCH include directory, and they would then be included where >> a source file asked for the corresponding .hpp file. >> > > I assume precompiling a header means precompiling it itself and any > header it includes, no? In that case, would precompiling each header > individually not cause us to implicitly precompile included headers > many times (e.g. globalDefinitions.hpp)? I presume you'd have to figure out the order in which to do the precompilation to avoid that. The makefile aspects of this seems non-trivial compared to precompiled.hpp. Cheers, David > Thanks, Thomas > >> That would also remove the problem of things not compiling with PCH >> disabled as a single set of includes would be used for both PCH and >> non-PCH building. >> >> I don't know if the other compilers support PCH in a similar manner to gcc. >> >> Cheers, >> David >> >>> /Magnus >>> >>>> >>>> David >>>> >>>>> 25 // Precompiled headers are turned off for Sun Studio, >>>>> May as well change to Solaris Studio if you're going to fix the typo :) >>>>> Thanks. >>>>> David >>>>>> On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: >>>>>> The reasons for the current set of files included in precompiled.hpp is somewhat lost in the mists of history. However, it is clear that it is not optimal. >>>>>> >>>>>> This patch replaces the current set with a new set, based on how often a header file is included in a C++ file. This selection contains all header files that are included by at least 130 C++ files. Testing has shown that this is around the optimal value -- include many more, and too many "innocent" files get hurt by unneeded work, leave out many more, and we miss out on optimization possibilities. >>>>>> >>>>>> The same set turned out to work well for both clang and gcc. However, files named "*.inline.hpp" did hurt rather than help performance, so those have been left out. For visual studio, the same set was also optimal, as long as the inline files were included. Presumably, visual studio is better than gcc/clang on handling precompiled headers containing inlined code. >>>>>> >>>>>> Here are some rough comparisons from our internal CI system, for building the target "hotspot" from scratch. >>>>>> >>>>>> macosx-x64: >>>>>> old: 00:05:00 >>>>>> new: 00:03:47 >>>>>> >>>>>> linux-x64: >>>>>> old: 00:05:43 >>>>>> new: 00:04:51 >>>>>> >>>>>> windows-x64: >>>>>> old: 00:05:18 >>>>>> new: 00:04:33 >>>>>> >>>>>> linux-aarch64: >>>>>> old: 00:07:57 >>>>>> new: 00:03:48 >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 >>>>>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 >>>>>> >>>>>> /Magnus >>> From per.liden at oracle.com Mon Nov 5 07:34:26 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 5 Nov 2018 08:34:26 +0100 Subject: RFR: 8212989: Allow CompiledMethod ExceptionCache have unloaded klasses In-Reply-To: <604b581e-00f2-24df-b06f-738bdefc593a@oracle.com> References: <5BDAF881.3030905@oracle.com> <604b581e-00f2-24df-b06f-738bdefc593a@oracle.com> Message-ID: <70b582cf-41ed-5f21-00a5-c982db2058f9@oracle.com> Hi Erik, On 11/1/18 11:14 PM, Erik ?sterlund wrote: > Hi Vladimir, > > Thanks for the quick reply. > > On 2018-11-01 21:14, Vladimir Kozlov wrote: >> Hi Erik, >> >> It seems these? changed are not based on latest sources. I don't see >> call to CodeCache::do_unloading in CodeCache::do_unloading() which is >> present in latest code: >> >> http://hg.openjdk.java.net/jdk/jdk/file/5ea020bcaa0d/src/hotspot/share/code/codeCache.cpp#l683 >> > > Yeah this code is taken from the context of the future. Here is the > exact same patch rebased against jdk-jdk: > http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01/ > > And new webrev with your suggestions: > http://cr.openjdk.java.net/~eosterlund/8212989/webrev.02/ Looks good to me. Just one minor nit: src/hotspot/share/code/codeCache.cpp ------------------------------------ 912 ExceptionCache* free_list_head = Atomic::load(&_exception_cache_purge_list); May I suggest that we rename free_list_head to purge_list_head. cheers, Per > > And incremental of that to the jdk-jdk rebase: > http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01_02/ > >> >> May be make some methods private, like >> CompiledMethod::exception_cache_acquire(). > > Sure. Made the accessors protected, as some are used in nmethod too and > it seems fine for subclasses to access. > >> It seems you remove the only case when release_set_exception_cache() >> is called. > > That is true. Removed. > >> I don't see where purge_exception_caches() is used. > > It's only called by a future version of ZGC from its new concurrent > class unloading feature that is currently baking in the ZGC repository. > The release_exception_cache member function deletes entries > synchronously if cleanup is triggered in a safepoint (all GCs except > ZGC), and otherwise defers deletion to the purge_exception_caches() > method, that is expected to happen one global handshake operation after > the exception caches have been unlinked by cleanup. > >> Next sentence seems incomplete: >> >> +? // unlinked. That is also when the >> CodeCache::exception_cache_free_list() > > Fixed. > >> Why you use exception_cache_acquire() in clean_exception_cache() and >> simple exception_cache() in add_exception_cache_entry()? > > The reason is that in the context of insertion, you are holding the lock > protecting against concurrent insertions. Therefore, entries you load > are either entries that were protected by the same lock (other > insertions), or entries that are as old or older than entries that have > died a whole safepoint operation earlier (due to prepending inserts), > which makes its memory state stable. Therefore, no acquire is required > when reading the head in the insertion path. > > Thanks, > /Erik > >> Thanks, >> Vladimir >> >> On 11/1/18 5:58 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> The ExceptionCaches of compiled methods is traditionally pruned in >>> safepoints during class unloading. This allows the exception cache to >>> have lock-free reads, while performing inserts under a lock, and >>> pruning under a safepoint. With concurrent class unloading, the >>> exception caches need to be cleaned concurrently. >>> >>> To retain the lock-free reads that seem good to keep lock-free, a >>> lock-free cleaning mechanism was introduced. Only one thread per >>> compiled method cleans each exception cache, but that thread races >>> with both single writers (under the ExceptionCache_lock) and any >>> number of readers. >>> >>> The head of the exception cache list is concurrently cleaned by both >>> inserting threads and cleaning threads. This allows having an >>> invariant that no newly prepended entries ever produce next pointers >>> to dead exception caches, that are concurrently removed from the >>> list. As for the internal next pointers, they are only pruned by the >>> one concurrent cleaning thread. This happens concurrent to reads that >>> simply skip over dead entries as they have different Klass* pointers >>> to the one being searched for. >>> >>> The single concurrent cleanup thread does not delete things removed >>> from the list straight away. Instead, they are placed on a purge list >>> that is freed up after a subsequent global handshaking operation. >>> That allows ABA-free CAS instructions in the lock-free paths, and >>> allows safe concurrent reading of entries in the exception cache list. >>> >>> Note that we already incorrectly "relied on memory_order_consume" for >>> the head pointer. But it used a volatile load. Assuming volatile >>> loads retain memory_order_consume semantics is not allowed in HotSpot >>> runtime code, so I changed those loads to use acquire accordingly. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212989 >>> >>> Thanks, >>> /Erik From erik.osterlund at oracle.com Mon Nov 5 07:44:07 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 5 Nov 2018 08:44:07 +0100 Subject: RFR: 8212989: Allow CompiledMethod ExceptionCache have unloaded klasses In-Reply-To: <70b582cf-41ed-5f21-00a5-c982db2058f9@oracle.com> References: <5BDAF881.3030905@oracle.com> <604b581e-00f2-24df-b06f-738bdefc593a@oracle.com> <70b582cf-41ed-5f21-00a5-c982db2058f9@oracle.com> Message-ID: Hi Per, Thanks for the review. On 2018-11-05 08:34, Per Liden wrote: > Hi Erik, > > On 11/1/18 11:14 PM, Erik ?sterlund wrote: >> Hi Vladimir, >> >> Thanks for the quick reply. >> >> On 2018-11-01 21:14, Vladimir Kozlov wrote: >>> Hi Erik, >>> >>> It seems these? changed are not based on latest sources. I don't see >>> call to CodeCache::do_unloading in CodeCache::do_unloading() which is >>> present in latest code: >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/5ea020bcaa0d/src/hotspot/share/code/codeCache.cpp#l683 >>> >> >> Yeah this code is taken from the context of the future. Here is the >> exact same patch rebased against jdk-jdk: >> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01/ >> >> And new webrev with your suggestions: >> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.02/ > > Looks good to me. Just one minor nit: > > src/hotspot/share/code/codeCache.cpp > ------------------------------------ > > 912?????? ExceptionCache* free_list_head = > Atomic::load(&_exception_cache_purge_list); > > May I suggest that we rename free_list_head to purge_list_head. Ahh yes. Will do before pushing. Thanks, /Erik > cheers, > Per > > >> >> And incremental of that to the jdk-jdk rebase: >> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.01_02/ >> >>> >>> May be make some methods private, like >>> CompiledMethod::exception_cache_acquire(). >> >> Sure. Made the accessors protected, as some are used in nmethod too >> and it seems fine for subclasses to access. >> >>> It seems you remove the only case when release_set_exception_cache() >>> is called. >> >> That is true. Removed. >> >>> I don't see where purge_exception_caches() is used. >> >> It's only called by a future version of ZGC from its new concurrent >> class unloading feature that is currently baking in the ZGC >> repository. The release_exception_cache member function deletes >> entries synchronously if cleanup is triggered in a safepoint (all GCs >> except ZGC), and otherwise defers deletion to the >> purge_exception_caches() method, that is expected to happen one global >> handshake operation after the exception caches have been unlinked by >> cleanup. >> >>> Next sentence seems incomplete: >>> >>> +? // unlinked. That is also when the >>> CodeCache::exception_cache_free_list() >> >> Fixed. >> >>> Why you use exception_cache_acquire() in clean_exception_cache() and >>> simple exception_cache() in add_exception_cache_entry()? >> >> The reason is that in the context of insertion, you are holding the >> lock protecting against concurrent insertions. Therefore, entries you >> load are either entries that were protected by the same lock (other >> insertions), or entries that are as old or older than entries that >> have died a whole safepoint operation earlier (due to prepending >> inserts), which makes its memory state stable. Therefore, no acquire >> is required when reading the head in the insertion path. >> >> Thanks, >> /Erik >> >>> Thanks, >>> Vladimir >>> >>> On 11/1/18 5:58 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> The ExceptionCaches of compiled methods is traditionally pruned in >>>> safepoints during class unloading. This allows the exception cache >>>> to have lock-free reads, while performing inserts under a lock, and >>>> pruning under a safepoint. With concurrent class unloading, the >>>> exception caches need to be cleaned concurrently. >>>> >>>> To retain the lock-free reads that seem good to keep lock-free, a >>>> lock-free cleaning mechanism was introduced. Only one thread per >>>> compiled method cleans each exception cache, but that thread races >>>> with both single writers (under the ExceptionCache_lock) and any >>>> number of readers. >>>> >>>> The head of the exception cache list is concurrently cleaned by both >>>> inserting threads and cleaning threads. This allows having an >>>> invariant that no newly prepended entries ever produce next pointers >>>> to dead exception caches, that are concurrently removed from the >>>> list. As for the internal next pointers, they are only pruned by the >>>> one concurrent cleaning thread. This happens concurrent to reads >>>> that simply skip over dead entries as they have different Klass* >>>> pointers to the one being searched for. >>>> >>>> The single concurrent cleanup thread does not delete things removed >>>> from the list straight away. Instead, they are placed on a purge >>>> list that is freed up after a subsequent global handshaking >>>> operation. That allows ABA-free CAS instructions in the lock-free >>>> paths, and allows safe concurrent reading of entries in the >>>> exception cache list. >>>> >>>> Note that we already incorrectly "relied on memory_order_consume" >>>> for the head pointer. But it used a volatile load. Assuming volatile >>>> loads retain memory_order_consume semantics is not allowed in >>>> HotSpot runtime code, so I changed those loads to use acquire >>>> accordingly. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8212989/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8212989 >>>> >>>> Thanks, >>>> /Erik From magnus.ihse.bursie at oracle.com Mon Nov 5 09:41:39 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 5 Nov 2018 10:41:39 +0100 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: <10a9ab67-81c6-3cf2-9f42-817574b4f4dc@oracle.com> References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> <42E17EFC-8E6C-4032-A136-529A96E72E93@oracle.com> <507b7651-f9fc-65f1-9ee4-9f6fe391246f@oracle.com> <10a9ab67-81c6-3cf2-9f42-817574b4f4dc@oracle.com> Message-ID: On 2018-11-05 08:28, David Holmes wrote: > Hi Thomas, > > On 5/11/2018 5:17 PM, Thomas St?fe wrote: >> Hi David, >> >> On Sun, Nov 4, 2018 at 11:09 PM David Holmes >> wrote: >>> >>> On 4/11/2018 5:27 PM, Magnus Ihse Bursie wrote: >>>> >>>>> 3 nov. 2018 kl. 23:24 skrev David Holmes : >>>>> >>>>>> On 3/11/2018 10:09 PM, David Holmes wrote: >>>>>> Looks okay - thanks for doing all the experiments! (Though I'm >>>>>> still curious what happens if you recompile individual header >>>>>> files :) ). >>>>> >>>>> s/recompile/precompile/? :) >>>> >>>> What do you mean? To have more than one set of PCH? As far as I >>>> know, all compilers we use only support a single PCH per source >>>> file to compile. So while you could have a PCH tailored per source >>>> file, that would only make sense in a scenario were you recompile >>>> all source files (but no header files) often. >>> >>> The GCC description for PCH describes it as a per-file header file >>> optimization. You can precompile each .hpp file and if it is located in >>> the same directory as the .hpp (or is on an include directory ahead of >>> the .hpp) then the PCH file will be used instead. So rather than >>> precompiling precompiled.hpp and having source files include it, you >>> would precompile each individual (chosen) header file and have that in >>> the special PCH include directory, and they would then be included >>> where >>> a source file asked for the corresponding .hpp file. >>> >> >> I assume precompiling a header means precompiling it itself and any >> header it includes, no? In that case, would precompiling each header >> individually not cause us to implicitly precompile included headers >> many times (e.g. globalDefinitions.hpp)? > > I presume you'd have to figure out the order in which to do the > precompilation to avoid that. The makefile aspects of this seems > non-trivial compared to precompiled.hpp. I see. I think this is something we might have in mind for perhaps some kind of future improvement. The gcc behavior you describe is not shared by any other compiler, as far as I know, which also makes this a bit of a special solution. Since additional speed gains on gcc seems not like our most pressing issue, I can't say I'm interested right now in making a special solution. If I were to spend more time on PCH, I'd rather spend it on either Windows (which is the slowest build platform in our CI), or Solaris, which does not have precompiled headers at all. (I actually looked into solstudio PCH as part of this fix to see if the new set of headers would allow us to enable it. Unfortunately, it seems like they need a single PCH per directory of source code files (e.g. all files in share/logging can share PCH with each other, but they cannot share PCH with the files in share/utilities. As I understood the problem, at least; the error messages were unclear and undocumented.) /Magnus > > Cheers, > David > >> Thanks, Thomas >> >>> That would also remove the problem of things not compiling with PCH >>> disabled as a single set of includes would be used for both PCH and >>> non-PCH building. >>> >>> I don't know if the other compilers support PCH in a similar manner >>> to gcc. >>> >>> Cheers, >>> David >>> >>>> /Magnus >>>> >>>>> >>>>> David >>>>> >>>>>> ?? 25 // Precompiled headers are turned off for Sun Studio, >>>>>> May as well change to Solaris Studio if you're going to fix the >>>>>> typo :) >>>>>> Thanks. >>>>>> David >>>>>>> On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: >>>>>>> The reasons for the current set of files included in >>>>>>> precompiled.hpp is somewhat lost in the mists of history. >>>>>>> However, it is clear that it is not optimal. >>>>>>> >>>>>>> This patch replaces the current set with a new set, based on how >>>>>>> often a header file is included in a C++ file. This selection >>>>>>> contains all header files that are included by at least 130 C++ >>>>>>> files. Testing has shown that this is around the optimal value >>>>>>> -- include many more, and too many "innocent" files get hurt by >>>>>>> unneeded work, leave out many more, and we miss out on >>>>>>> optimization possibilities. >>>>>>> >>>>>>> The same set turned out to work well for both clang and gcc. >>>>>>> However, files named "*.inline.hpp" did hurt rather than help >>>>>>> performance, so those have been left out. For visual studio, the >>>>>>> same set was also optimal, as long as the inline files were >>>>>>> included. Presumably, visual studio is better than gcc/clang on >>>>>>> handling precompiled headers containing inlined code. >>>>>>> >>>>>>> Here are some rough comparisons from our internal CI system, for >>>>>>> building the target "hotspot" from scratch. >>>>>>> >>>>>>> macosx-x64: >>>>>>> old: 00:05:00 >>>>>>> new: 00:03:47 >>>>>>> >>>>>>> linux-x64: >>>>>>> old: 00:05:43 >>>>>>> new: 00:04:51 >>>>>>> >>>>>>> windows-x64: >>>>>>> old: 00:05:18 >>>>>>> new: 00:04:33 >>>>>>> >>>>>>> linux-aarch64: >>>>>>> old: 00:07:57 >>>>>>> new: 00:03:48 >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 >>>>>>> WebRev: >>>>>>> http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 >>>>>>> >>>>>>> /Magnus >>>> From per.liden at oracle.com Mon Nov 5 09:51:17 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 5 Nov 2018 10:51:17 +0100 Subject: RFR: 8212585: Clean up CompiledMethod::oops_reloc_begin() In-Reply-To: References: <5BC756EE.6040900@oracle.com> Message-ID: <1e1788aa-b484-3656-9515-00b9c28c1484@oracle.com> Hi Erik, On 10/18/18 11:07 PM, Erik ?sterlund wrote: > Hi, > > I decided to make this less risky for platforms that do not yet need to > be scanned concurrently, and hence don't really need to be "fixed". > > In this new webrev, I decided to check for the frame completed offset if > available and further away than verified entry + native jump size. Then > it is safe for me to go ahead and scan this stuff concurrently using > nmethod entry barriers. Otherwise, the same good old behaviour we used > to rely on applies so that we won't get any surprises where we do not > want them. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212585/webrev.01/ Looks good! /Per > > Thanks, > /Erik > > On 2018-10-17 17:36, Erik ?sterlund wrote: >> Hi, >> >> In CompiledMethod::oops_reloc_begin() there is code to prevent looking >> for oops at the verified entry of an nmethod that is not entrant, as a >> native jump instruction has been overwritten there. Except there would >> *never* be any immediate oops there for any compiler, even before >> becoming not entrant, with or without OSR compilation, so this special >> handling of not entrant vs entrant nmethods is seemingly completely >> unnecessary. >> >> This gets increasingly awkward when oops_do is called concurrently >> with concurrent class unloading, where an nmethod is racing to become >> not entrant. >> >> To clean this up, I propose to change the boundary for immediate oop >> scanning and start looking for oops only after the frame building is >> completed, as there is absolutely no point in looking for oops before >> that in the first place. This removes unnecessary workarounds that >> exist today, and removes unnecessary races going forward. >> >> It seems like Graal as JIT also inlines oops into the code stream, but >> never sets the CodeOffsets::Frame_Complete in the nmethod. So I >> recognize such nmethods and unconditionally start scanning at the >> verified entry + native jump instruction size. I spoke to the Graal >> folks, and they said they do not put oops in there either. So I think >> everyone involved should be happy with this solution. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212585 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212585/webrev.00/ >> >> Thanks, >> /Erik > From robbin.ehn at oracle.com Mon Nov 5 09:56:54 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 5 Nov 2018 10:56:54 +0100 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> Message-ID: <7e997976-e91a-faaf-8673-ee32a43038fa@oracle.com> Hi Kim, Seems reasonable, I must admit I hate the MutexUnlocker, but I see way you choose to that pattern here. It's not obvious to me why you needed to change the: 644 OrderAccess::storeload(); to 668 OrderAccess::fence(); Thanks, Robbin On 10/19/18 10:47 PM, Kim Barrett wrote: > Please review this change to the Service thread to delete empty > OopStorage blocks when needed. > > As part of this, eliminated delete_empty_blocks_safepoint, and renamed > delete_empty_blocks_concurrent to remove the now redundant _concurrent > suffix. Also added a predicate for the Service thread's use, to test > whether there is cleanup work to be done. > > The previously unused empty block deletion has been revised for this > new usage. This includes making loops safepoint polite. > > Various parts of OopStorage are now aware of the Service thread > cleanup and notify that thread when appropriate. (allocate's > obtaining the block to allocate from was refactored to make it easier > to insert that notification.) > > As part of this, OopStorage::Block::release_entries now takes the > owning storage object as an argument (rather than a pointer to its > _deferred_updates list). This allows release_entries to perform > additional operations on the owner (so long as C++ DR45 has been > fixed, and we've added more AIX-specific workarounds for that). > > While touching the Service thread, fixed Service_lock locking to use > use MonitorLockerEx rather than MutexLockerEx. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8210986 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8210986/open.00/ > > Testing: > Mach5 tier1-5. > Ran some performance tests to verify no regressions due to additional > load on the Service thread. > From per.liden at oracle.com Mon Nov 5 10:02:23 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 5 Nov 2018 11:02:23 +0100 Subject: RFR: 8212996: Use AS_NO_KEEPALIVE when accessing dead java.lang.invoke.CallSites during nmethod unloading In-Reply-To: References: Message-ID: <7c93d61b-4540-0daa-4a8d-6e56d0c897ba@oracle.com> Hi Erik, On 10/25/18 9:29 PM, Erik ?sterlund wrote: > Hi, > > When nmethods are unloaded, they may have dead > java.lang.invoke.CallSites loaded, and then we follow their dead > java.lang.invoke.MethodHandleNatives.CallSiteContext, so that its > dependency context can be cleaned. These dead oops should be loaded with > AS_NO_KEEPALIVE accessors, otherwise GC barriers can become very unhappy. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8212996/webrev.00/ Looks good to me! /Per > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8212996 > > Thanks, > /Erik From erik.osterlund at oracle.com Mon Nov 5 10:14:06 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 5 Nov 2018 11:14:06 +0100 Subject: RFR: 8212585: Clean up CompiledMethod::oops_reloc_begin() In-Reply-To: <1e1788aa-b484-3656-9515-00b9c28c1484@oracle.com> References: <5BC756EE.6040900@oracle.com> <1e1788aa-b484-3656-9515-00b9c28c1484@oracle.com> Message-ID: <5BE017EE.2000202@oracle.com> Hi Per, Thanks for the review. /Erik On 2018-11-05 10:51, Per Liden wrote: > Hi Erik, > > On 10/18/18 11:07 PM, Erik ?sterlund wrote: >> Hi, >> >> I decided to make this less risky for platforms that do not yet need >> to be scanned concurrently, and hence don't really need to be "fixed". >> >> In this new webrev, I decided to check for the frame completed offset >> if available and further away than verified entry + native jump size. >> Then it is safe for me to go ahead and scan this stuff concurrently >> using nmethod entry barriers. Otherwise, the same good old behaviour >> we used to rely on applies so that we won't get any surprises where >> we do not want them. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212585/webrev.01/ > > Looks good! > > /Per > >> >> Thanks, >> /Erik >> >> On 2018-10-17 17:36, Erik ?sterlund wrote: >>> Hi, >>> >>> In CompiledMethod::oops_reloc_begin() there is code to prevent >>> looking for oops at the verified entry of an nmethod that is not >>> entrant, as a native jump instruction has been overwritten there. >>> Except there would *never* be any immediate oops there for any >>> compiler, even before becoming not entrant, with or without OSR >>> compilation, so this special handling of not entrant vs entrant >>> nmethods is seemingly completely unnecessary. >>> >>> This gets increasingly awkward when oops_do is called concurrently >>> with concurrent class unloading, where an nmethod is racing to >>> become not entrant. >>> >>> To clean this up, I propose to change the boundary for immediate oop >>> scanning and start looking for oops only after the frame building is >>> completed, as there is absolutely no point in looking for oops >>> before that in the first place. This removes unnecessary workarounds >>> that exist today, and removes unnecessary races going forward. >>> >>> It seems like Graal as JIT also inlines oops into the code stream, >>> but never sets the CodeOffsets::Frame_Complete in the nmethod. So I >>> recognize such nmethods and unconditionally start scanning at the >>> verified entry + native jump instruction size. I spoke to the Graal >>> folks, and they said they do not put oops in there either. So I >>> think everyone involved should be happy with this solution. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212585 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212585/webrev.00/ >>> >>> Thanks, >>> /Erik >> From erik.osterlund at oracle.com Mon Nov 5 10:14:21 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 5 Nov 2018 11:14:21 +0100 Subject: RFR: 8212996: Use AS_NO_KEEPALIVE when accessing dead java.lang.invoke.CallSites during nmethod unloading In-Reply-To: <7c93d61b-4540-0daa-4a8d-6e56d0c897ba@oracle.com> References: <7c93d61b-4540-0daa-4a8d-6e56d0c897ba@oracle.com> Message-ID: <5BE017FD.4000802@oracle.com> Hi Per, Thanks for the review. /Erik On 2018-11-05 11:02, Per Liden wrote: > Hi Erik, > > On 10/25/18 9:29 PM, Erik ?sterlund wrote: >> Hi, >> >> When nmethods are unloaded, they may have dead >> java.lang.invoke.CallSites loaded, and then we follow their dead >> java.lang.invoke.MethodHandleNatives.CallSiteContext, so that its >> dependency context can be cleaned. These dead oops should be loaded >> with AS_NO_KEEPALIVE accessors, otherwise GC barriers can become very >> unhappy. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212996/webrev.00/ > > Looks good to me! > > /Per > >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8212996 >> >> Thanks, >> /Erik From adinn at redhat.com Mon Nov 5 10:59:07 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 5 Nov 2018 10:59:07 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> Message-ID: On 02/11/18 22:31, Roman Kennke wrote: > I had a discussion with Andrew Dinn (who actually wrote that code back > then) and will copy his explanation here, I think it covers the situation: Err, ... just to be clear ... I don't believe I did write that code (the relevant check-in was by shade). I might, perhaps, have been consulted when it was written and I think my explanation (below) of what it does is correct. Perhaps Aleksey could also comment? > "" > Let's just recap without considering Shenandoah: > > MacroAssembler::needs_explicit_null_check is called from several places, > not just the SEGV handler. > > In those other cases offset is a putative field offset for an object. > > With uncompressed pointers the method has to decide whether this offset > will lie within the read-protected page at address 0 i.e. 0 <= offset < > os::vm_page_size(). If that is true then a null check can be omitted for > a read or write operation. If not an explicit check is needed. > > With compressed pointers when Universe::narrow_oop_base() != NULL the > test also checks whether this offset may lie within the read-protected > page at the heap base only -- oh -- that's the same test! Of course, > we'll see that this is different when the SEGV handler makes the call. > > n.b. I think this case is included because there is the possibility that > a null pointer can be represented by the narrow oop base address. > However, I think in reality special case checks in all encode decode > operations ensure that a null oop is always represented as zero (for > AArch64 at least). I'm not absolutely sure of that though. > > Now, when called from the SEGV handler offset is actually the offending > read/write address that caused the SEGV. In that case a null pointer > will only have been dereferenced if the address lies in the protected > zero page or in the protected heap base page. The first if clause > handles the compressed pointer case and reduces it to the uncompressed > pointer case by subtracting base when offset >= base. > > So, either way the offending address is potentially a null pointer > dereference under exactly the same conditions as for those other calls. > > Next, how does Shenandoah modify this? Well, it looks like that depends > on how this method is called. If it can be invoked from any of those > other places to check whether a Brooks pointer read or write is ok to > validate with an implicit null check then the offset may be passed as -8 > i.e. 0xfffffffffffffff8. However, that's actually not enough. > > When invoked from the handler because of an access at of (0 + offset) > then offset may legitimately be 0x00fffffffffffff8 or lie between 0 and > os::vm_page_size(). > > When invoked form the handler because of a dereference of > (narrow_oop_base + offset) then the subtract i the if clause means that > offset may legitimately be 0xfffffffffffffff8 or lie between 0 and > os::vm_page_size(). > > So, the masking in the Shenandoah case is to reduce those two special > cases into one. > "" > > Roman > > > >> Hi Roman, >> >> On 2018-11-01 17:58, Roman Kennke wrote: >>> Hi Erik, >>> >>>> Would you mind explaining how you need to override this and why? I'm >>>> afraid I did not quite get it from your description in the RFC (which is >>>> also why I didn't come up with a solution either). >>>> >>>> Am I correct that you need to return false if the passed in offset is >>>> the cell header offset -8, or is there anything else at all required to >>>> that logic? >>> >>> No, it's really just that. Plus take care of it in the case of combined >>> narrow-oops-offset plus -8. >> >>>> You mentioned something about the high order byte being >>>> masked on AArch64, but didn't quite connect the dot to how that is >>>> related to this code. Is it? >>> >>> Yes, we also need to mask the high order byte in AArch64 because of the >>> way addressing works on that platform, and because -8 flips on those >>> negative bits. >> >> Okay. Looking at >> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >> that you posted before, assuming this is how it will be extended. >> >> So in the case with a heap base, the resulting address will never be >> negative, and you know it will trap (presuming there is an mprotected >> page at offset -8 from the heap base pointer. So it seems in this case, >> the check could have just been offset == -8. >> >> So let's say we access null -8 without a heap base then. The address >> becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the >> signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address >> masking of the high order byte. I presume that is what the AArch64 code >> is dealing with. But I don't quite understand why. Addresses starting >> with 0x00FF are not valid in user space so we never need to worry about >> any memory being mapped in there accidentally. So I'm not sure what this >> code protects against. >> >> So that takes me back to my original intuition: isn't this the same as >> checking at the very top: if (offset == BrooksPointer::byte_offset()) >> return false; >> >> If it really is as simple as just that, and I did not miss something not >> obvious, then perhaps we would be better off abstracting a cell header >> check here, as opposed to wrapping the whole thing in a virtual member >> function. Especially since there are now a number of these occurrences >> where some basic knowledge about cell header size that is 0 for all GCs >> except Shenandoah where it is 8, leaves us wrapping a whole bunch of >> stuff behind a GC interface, only to handle that one difference. >> >> While I don't know exactly what this cell abstraction would reasonably >> look like in its full shape, perhaps we could start with something >> simple like a virtual member function size_t cell_header_size() on >> CollectedHeap with a suitable comment explaining that a cell header is a >> GC thing we sometimes put above the object header. And then see if there >> are more related functions that lead us to a helper class for cells or >> something like that. >> >> I'm open to suggestions of course. Thought I'd check with you if this >> sounds sane to you. Of course if my assumption is wrong that this check >> could be written much simpler than it looks like, then I suppose I need >> to give up on that idea. It all boils down to that really. >> >> Thanks, >> /Erik >> >>> Also, the mach5 job came back with FAILED (see below). Can somebody with >>> access check and see what's up? >>> >>> Thanks, >>> Roman >>> >>> >>> Build Details: 2018-11-01-1146402.roman.source >>> 0 Failed Tests >>> Mach5 Tasks Results Summary >>> >>> ???? EXECUTED_WITH_FAILURE: 6 >>> ???? KILLED: 0 >>> ???? UNABLE_TO_RUN: 21 >>> ???? PASSED: 48 >>> ???? FAILED: 0 >>> ???? NA: 0 >>> ???? Build >>> >>> ???????? 6 Executed with failure >>> ???????????? solaris-sparcv9-solaris-sparcv9-build-8 error while >>> building, return value: 2 >>> ???????????? solaris-sparcv9-debug-solaris-sparcv9-build-9 error while >>> building, return value: 2 >>> ???????????? windows-x64-windows-x64-build-10 error while building, >>> return value: 2 >>> ???????????? windows-x64-debug-windows-x64-build-11 error while building, >>> return value: 2 >>> ???????????? windows-x64-open-windows-x64-build-12 error while building, >>> return value: 2 >>> ???????????? windows-x64-open-debug-windows-x64-build-13 error while >>> building, return value: 2 >>> ???????? 2 Not run >>> ???????????? solaris-sparcv9-install-solaris-sparcv9-build-16 Dependency >>> task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>> ???????????? windows-x64-install-windows-x64-build-17 Dependency task >>> failed: YJftjiBUYc >>> >>> ???? Test >>> >>> ???????? 19 Not run >>> >>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 >>> >>> Dependency task failed: >>> mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>> >>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 >>> >>> Dependency task failed: >>> mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 >>> >>> tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 >>> Dependency task failed: YJftjiBUYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 >>> Dependency task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 >>> Dependency >>> task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 >>> Dependency >>> task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 >>> Dependency >>> task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 >>> >>> Dependency task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 >>> Dependency task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 >>> Dependency task failed: YJftjiBVYc >>> ???????????? See all 19... >>> >>> >>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-11-01 12:20, Roman Kennke wrote: >>>>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>>>> >>>>> Erik: ok for you too? >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >>>>>>> >>>>>>> Hi Erik, >>>>>>> >>>>>>> right. Fixed this, and what what Kim mentioned plus a missing >>>>>>> include: >>>>>>> >>>>>>> Incremental: >>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>>>> Full: >>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>>>> >>>>>>> Ok now? >>>>>> Looks good. >>>>>> >>>> >>> > -- regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From rkennke at redhat.com Mon Nov 5 11:09:44 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 5 Nov 2018 12:09:44 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> Message-ID: >> I had a discussion with Andrew Dinn (who actually wrote that code back >> then) and will copy his explanation here, I think it covers the situation: > > Err, ... just to be clear ... I don't believe I did write that code (the > relevant check-in was by shade). I might, perhaps, have been consulted > when it was written and I think my explanation (below) of what it does > is correct. Perhaps Aleksey could also comment? Oh sorry. I probably mis-remember. The relevant code seems to be done by Aleksey, Roland and me, but I seem to remember that you and me discussed some of it before (at the very least, I am pretty sure, I did consult you about the aarch64 masking part ;-) ). Thank you for the long explanation anyway! Roman >> "" >> Let's just recap without considering Shenandoah: >> >> MacroAssembler::needs_explicit_null_check is called from several places, >> not just the SEGV handler. >> >> In those other cases offset is a putative field offset for an object. >> >> With uncompressed pointers the method has to decide whether this offset >> will lie within the read-protected page at address 0 i.e. 0 <= offset < >> os::vm_page_size(). If that is true then a null check can be omitted for >> a read or write operation. If not an explicit check is needed. >> >> With compressed pointers when Universe::narrow_oop_base() != NULL the >> test also checks whether this offset may lie within the read-protected >> page at the heap base only -- oh -- that's the same test! Of course, >> we'll see that this is different when the SEGV handler makes the call. >> >> n.b. I think this case is included because there is the possibility that >> a null pointer can be represented by the narrow oop base address. >> However, I think in reality special case checks in all encode decode >> operations ensure that a null oop is always represented as zero (for >> AArch64 at least). I'm not absolutely sure of that though. >> >> Now, when called from the SEGV handler offset is actually the offending >> read/write address that caused the SEGV. In that case a null pointer >> will only have been dereferenced if the address lies in the protected >> zero page or in the protected heap base page. The first if clause >> handles the compressed pointer case and reduces it to the uncompressed >> pointer case by subtracting base when offset >= base. >> >> So, either way the offending address is potentially a null pointer >> dereference under exactly the same conditions as for those other calls. >> >> Next, how does Shenandoah modify this? Well, it looks like that depends >> on how this method is called. If it can be invoked from any of those >> other places to check whether a Brooks pointer read or write is ok to >> validate with an implicit null check then the offset may be passed as -8 >> i.e. 0xfffffffffffffff8. However, that's actually not enough. >> >> When invoked from the handler because of an access at of (0 + offset) >> then offset may legitimately be 0x00fffffffffffff8 or lie between 0 and >> os::vm_page_size(). >> >> When invoked form the handler because of a dereference of >> (narrow_oop_base + offset) then the subtract i the if clause means that >> offset may legitimately be 0xfffffffffffffff8 or lie between 0 and >> os::vm_page_size(). >> >> So, the masking in the Shenandoah case is to reduce those two special >> cases into one. >> "" >> >> Roman >> >> >> >>> Hi Roman, >>> >>> On 2018-11-01 17:58, Roman Kennke wrote: >>>> Hi Erik, >>>> >>>>> Would you mind explaining how you need to override this and why? I'm >>>>> afraid I did not quite get it from your description in the RFC (which is >>>>> also why I didn't come up with a solution either). >>>>> >>>>> Am I correct that you need to return false if the passed in offset is >>>>> the cell header offset -8, or is there anything else at all required to >>>>> that logic? >>>> >>>> No, it's really just that. Plus take care of it in the case of combined >>>> narrow-oops-offset plus -8. >>> >>>>> You mentioned something about the high order byte being >>>>> masked on AArch64, but didn't quite connect the dot to how that is >>>>> related to this code. Is it? >>>> >>>> Yes, we also need to mask the high order byte in AArch64 because of the >>>> way addressing works on that platform, and because -8 flips on those >>>> negative bits. >>> >>> Okay. Looking at >>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >>> that you posted before, assuming this is how it will be extended. >>> >>> So in the case with a heap base, the resulting address will never be >>> negative, and you know it will trap (presuming there is an mprotected >>> page at offset -8 from the heap base pointer. So it seems in this case, >>> the check could have just been offset == -8. >>> >>> So let's say we access null -8 without a heap base then. The address >>> becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the >>> signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address >>> masking of the high order byte. I presume that is what the AArch64 code >>> is dealing with. But I don't quite understand why. Addresses starting >>> with 0x00FF are not valid in user space so we never need to worry about >>> any memory being mapped in there accidentally. So I'm not sure what this >>> code protects against. >>> >>> So that takes me back to my original intuition: isn't this the same as >>> checking at the very top: if (offset == BrooksPointer::byte_offset()) >>> return false; >>> >>> If it really is as simple as just that, and I did not miss something not >>> obvious, then perhaps we would be better off abstracting a cell header >>> check here, as opposed to wrapping the whole thing in a virtual member >>> function. Especially since there are now a number of these occurrences >>> where some basic knowledge about cell header size that is 0 for all GCs >>> except Shenandoah where it is 8, leaves us wrapping a whole bunch of >>> stuff behind a GC interface, only to handle that one difference. >>> >>> While I don't know exactly what this cell abstraction would reasonably >>> look like in its full shape, perhaps we could start with something >>> simple like a virtual member function size_t cell_header_size() on >>> CollectedHeap with a suitable comment explaining that a cell header is a >>> GC thing we sometimes put above the object header. And then see if there >>> are more related functions that lead us to a helper class for cells or >>> something like that. >>> >>> I'm open to suggestions of course. Thought I'd check with you if this >>> sounds sane to you. Of course if my assumption is wrong that this check >>> could be written much simpler than it looks like, then I suppose I need >>> to give up on that idea. It all boils down to that really. >>> >>> Thanks, >>> /Erik >>> >>>> Also, the mach5 job came back with FAILED (see below). Can somebody with >>>> access check and see what's up? >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>> Build Details: 2018-11-01-1146402.roman.source >>>> 0 Failed Tests >>>> Mach5 Tasks Results Summary >>>> >>>> ???? EXECUTED_WITH_FAILURE: 6 >>>> ???? KILLED: 0 >>>> ???? UNABLE_TO_RUN: 21 >>>> ???? PASSED: 48 >>>> ???? FAILED: 0 >>>> ???? NA: 0 >>>> ???? Build >>>> >>>> ???????? 6 Executed with failure >>>> ???????????? solaris-sparcv9-solaris-sparcv9-build-8 error while >>>> building, return value: 2 >>>> ???????????? solaris-sparcv9-debug-solaris-sparcv9-build-9 error while >>>> building, return value: 2 >>>> ???????????? windows-x64-windows-x64-build-10 error while building, >>>> return value: 2 >>>> ???????????? windows-x64-debug-windows-x64-build-11 error while building, >>>> return value: 2 >>>> ???????????? windows-x64-open-windows-x64-build-12 error while building, >>>> return value: 2 >>>> ???????????? windows-x64-open-debug-windows-x64-build-13 error while >>>> building, return value: 2 >>>> ???????? 2 Not run >>>> ???????????? solaris-sparcv9-install-solaris-sparcv9-build-16 Dependency >>>> task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>> ???????????? windows-x64-install-windows-x64-build-17 Dependency task >>>> failed: YJftjiBUYc >>>> >>>> ???? Test >>>> >>>> ???????? 19 Not run >>>> >>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 >>>> >>>> Dependency task failed: >>>> mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>> >>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 >>>> >>>> Dependency task failed: >>>> mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 >>>> >>>> tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 >>>> Dependency task failed: YJftjiBUYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 >>>> >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 >>>> Dependency task failed: YJftjiBVYc >>>> ???????????? See all 19... >>>> >>>> >>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-11-01 12:20, Roman Kennke wrote: >>>>>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>>>>> >>>>>> Erik: ok for you too? >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >>>>>>>> >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> right. Fixed this, and what what Kim mentioned plus a missing >>>>>>>> include: >>>>>>>> >>>>>>>> Incremental: >>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>>>>> Full: >>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>>>>> >>>>>>>> Ok now? >>>>>>> Looks good. >>>>>>> >>>>> >>>> >> > From shade at redhat.com Mon Nov 5 12:15:39 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 5 Nov 2018 13:15:39 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8211926 Fix: http://cr.openjdk.java.net/~shade/8211926/webrev.01/ Found this while porting some Shenandoah addons to jdk/jdk. I took the liberty of homogenizing the internals of those methods to always call "small" path when range does not allow us to enter the "large" path. This protects us from accidentally breaking VM when asserts are disabled, and makes "large" methods usable as drop-ins for "small" ones. Testing: hotspot-tier1, new gtest, jdk-submit (running) Thanks, -Aleksey From thomas.schatzl at oracle.com Mon Nov 5 13:05:23 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 05 Nov 2018 14:05:23 +0100 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> Message-ID: Hi, On Mon, 2018-11-05 at 11:43 +1000, David Holmes wrote: > This impacts GC, compiler, runtime and serviceability. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 > webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ > > To facilitate changes to the Mutex/Monitor code we need to ensure > there are no statically initialized Monitor/Mutex instances in the VM > - as the constructors may rely on VM initialization that has not > happened when C++ initializers execute. > > There were 6 locally defined "lock" members of classes that were > statically initialized, and these are across all areas of the VM: > > - Decoder::_shared_decoder_lock > - DCmdFactory::_dcmdFactory_lock > - JfrThreadSampler::_transition_block_lock > - NMethodSweeper::_stat_lock > - ThreadsSMRSupport::_delete_lock > - CodeCacheUnloadingTask::_lock > > The last is actually now unused and so is just deleted. The rest Removal of the CodeCacheUnloadingTask lock looks good. I also looked through the other changes which seems simple enough to me too, but should obviously be looked at from people from the respective areas too. Some nits: Maybe the spacing in mutexLocker.cpp:325 for the JfrThreadSampler_lock could be fixed though. Also the assert in decoder.cpp:94 seems to be as superfluous to me as in the other lines where it has been removed. Thanks, Thomas From erik.osterlund at oracle.com Mon Nov 5 13:08:54 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 5 Nov 2018 14:08:54 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> Message-ID: <5BE040E6.8080103@oracle.com> Hi Roman, So I get what you are saying right, we are using the bool needs_explicit_null_check(intptr_t offset) function both with offsets (what the parameter seems to suggest should be passed in), but also with raw addresses from the signal handler, posing as offsets. And we happened to get away with that because the check was written in such a clever way that it would return the right answer regardless of whether it was an offset or an address, saving us from writing those ~3 lines extra code for a new function expecting an address. Except now with negative offsets, that also additionally get masked by the AArch64 linux signal handler, this suddenly falls apart and we can no longer seamlessly pass in offsets and addresses to get consistent answers. The same function suddenly handles different combinations of masked/unmasked positive/negative addresses, with/without heap base, as well as positive/negative offsets. The first obvious suggestion that comes to mind is to sign extend the high order byte inside of the signal handler on AArch64, to fix the address before it gets passed in to needs_explicit_null_check, as opposed to letting the shared needs_explicit_null_check function take that responsibility, handling not just offsets and virtual addresses, but additionally also masked virtual addresses from the signal handler. So with the AArch64 stuff gone from this function, I still can't help but think that the conditions in this code would be possible to write in a trivially understandable way that doesn't require me to scratch my head, if there was one function expecting addresses and one function expecting offsets. And in that case, I think it would look trivial to have a cell header check in those functions, which would fit in well. I think. Thoughts? Thanks, /Erik On 2018-11-02 23:31, Roman Kennke wrote: > Hi Erik, > > I had a discussion with Andrew Dinn (who actually wrote that code back > then) and will copy his explanation here, I think it covers the situation: > > "" > Let's just recap without considering Shenandoah: > > MacroAssembler::needs_explicit_null_check is called from several places, > not just the SEGV handler. > > In those other cases offset is a putative field offset for an object. > > With uncompressed pointers the method has to decide whether this offset > will lie within the read-protected page at address 0 i.e. 0 <= offset < > os::vm_page_size(). If that is true then a null check can be omitted for > a read or write operation. If not an explicit check is needed. > > With compressed pointers when Universe::narrow_oop_base() != NULL the > test also checks whether this offset may lie within the read-protected > page at the heap base only -- oh -- that's the same test! Of course, > we'll see that this is different when the SEGV handler makes the call. > > n.b. I think this case is included because there is the possibility that > a null pointer can be represented by the narrow oop base address. > However, I think in reality special case checks in all encode decode > operations ensure that a null oop is always represented as zero (for > AArch64 at least). I'm not absolutely sure of that though. > > Now, when called from the SEGV handler offset is actually the offending > read/write address that caused the SEGV. In that case a null pointer > will only have been dereferenced if the address lies in the protected > zero page or in the protected heap base page. The first if clause > handles the compressed pointer case and reduces it to the uncompressed > pointer case by subtracting base when offset >= base. > > So, either way the offending address is potentially a null pointer > dereference under exactly the same conditions as for those other calls. > > Next, how does Shenandoah modify this? Well, it looks like that depends > on how this method is called. If it can be invoked from any of those > other places to check whether a Brooks pointer read or write is ok to > validate with an implicit null check then the offset may be passed as -8 > i.e. 0xfffffffffffffff8. However, that's actually not enough. > > When invoked from the handler because of an access at of (0 + offset) > then offset may legitimately be 0x00fffffffffffff8 or lie between 0 and > os::vm_page_size(). > > When invoked form the handler because of a dereference of > (narrow_oop_base + offset) then the subtract i the if clause means that > offset may legitimately be 0xfffffffffffffff8 or lie between 0 and > os::vm_page_size(). > > So, the masking in the Shenandoah case is to reduce those two special > cases into one. > "" > > Roman > > > >> Hi Roman, >> >> On 2018-11-01 17:58, Roman Kennke wrote: >>> Hi Erik, >>> >>>> Would you mind explaining how you need to override this and why? I'm >>>> afraid I did not quite get it from your description in the RFC (which is >>>> also why I didn't come up with a solution either). >>>> >>>> Am I correct that you need to return false if the passed in offset is >>>> the cell header offset -8, or is there anything else at all required to >>>> that logic? >>> No, it's really just that. Plus take care of it in the case of combined >>> narrow-oops-offset plus -8. >>>> You mentioned something about the high order byte being >>>> masked on AArch64, but didn't quite connect the dot to how that is >>>> related to this code. Is it? >>> Yes, we also need to mask the high order byte in AArch64 because of the >>> way addressing works on that platform, and because -8 flips on those >>> negative bits. >> Okay. Looking at >> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >> that you posted before, assuming this is how it will be extended. >> >> So in the case with a heap base, the resulting address will never be >> negative, and you know it will trap (presuming there is an mprotected >> page at offset -8 from the heap base pointer. So it seems in this case, >> the check could have just been offset == -8. >> >> So let's say we access null -8 without a heap base then. The address >> becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the >> signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address >> masking of the high order byte. I presume that is what the AArch64 code >> is dealing with. But I don't quite understand why. Addresses starting >> with 0x00FF are not valid in user space so we never need to worry about >> any memory being mapped in there accidentally. So I'm not sure what this >> code protects against. >> >> So that takes me back to my original intuition: isn't this the same as >> checking at the very top: if (offset == BrooksPointer::byte_offset()) >> return false; >> >> If it really is as simple as just that, and I did not miss something not >> obvious, then perhaps we would be better off abstracting a cell header >> check here, as opposed to wrapping the whole thing in a virtual member >> function. Especially since there are now a number of these occurrences >> where some basic knowledge about cell header size that is 0 for all GCs >> except Shenandoah where it is 8, leaves us wrapping a whole bunch of >> stuff behind a GC interface, only to handle that one difference. >> >> While I don't know exactly what this cell abstraction would reasonably >> look like in its full shape, perhaps we could start with something >> simple like a virtual member function size_t cell_header_size() on >> CollectedHeap with a suitable comment explaining that a cell header is a >> GC thing we sometimes put above the object header. And then see if there >> are more related functions that lead us to a helper class for cells or >> something like that. >> >> I'm open to suggestions of course. Thought I'd check with you if this >> sounds sane to you. Of course if my assumption is wrong that this check >> could be written much simpler than it looks like, then I suppose I need >> to give up on that idea. It all boils down to that really. >> >> Thanks, >> /Erik >> >>> Also, the mach5 job came back with FAILED (see below). Can somebody with >>> access check and see what's up? >>> >>> Thanks, >>> Roman >>> >>> >>> Build Details: 2018-11-01-1146402.roman.source >>> 0 Failed Tests >>> Mach5 Tasks Results Summary >>> >>> EXECUTED_WITH_FAILURE: 6 >>> KILLED: 0 >>> UNABLE_TO_RUN: 21 >>> PASSED: 48 >>> FAILED: 0 >>> NA: 0 >>> Build >>> >>> 6 Executed with failure >>> solaris-sparcv9-solaris-sparcv9-build-8 error while >>> building, return value: 2 >>> solaris-sparcv9-debug-solaris-sparcv9-build-9 error while >>> building, return value: 2 >>> windows-x64-windows-x64-build-10 error while building, >>> return value: 2 >>> windows-x64-debug-windows-x64-build-11 error while building, >>> return value: 2 >>> windows-x64-open-windows-x64-build-12 error while building, >>> return value: 2 >>> windows-x64-open-debug-windows-x64-build-13 error while >>> building, return value: 2 >>> 2 Not run >>> solaris-sparcv9-install-solaris-sparcv9-build-16 Dependency >>> task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>> windows-x64-install-windows-x64-build-17 Dependency task >>> failed: YJftjiBUYc >>> >>> Test >>> >>> 19 Not run >>> >>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 >>> >>> Dependency task failed: >>> mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>> >>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 >>> >>> Dependency task failed: >>> mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 >>> >>> tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 >>> Dependency task failed: YJftjiBUYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 >>> Dependency task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 >>> Dependency >>> task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 >>> Dependency >>> task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 >>> Dependency >>> task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 >>> >>> Dependency task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 >>> Dependency task failed: YJftjiBVYc >>> >>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 >>> Dependency task failed: YJftjiBVYc >>> See all 19... >>> >>> >>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-11-01 12:20, Roman Kennke wrote: >>>>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>>>> >>>>> Erik: ok for you too? >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke wrote: >>>>>>> >>>>>>> Hi Erik, >>>>>>> >>>>>>> right. Fixed this, and what what Kim mentioned plus a missing >>>>>>> include: >>>>>>> >>>>>>> Incremental: >>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>>>> Full: >>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>>>> >>>>>>> Ok now? >>>>>> Looks good. >>>>>> From rkennke at redhat.com Mon Nov 5 13:59:41 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 5 Nov 2018 14:59:41 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE040E6.8080103@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> Message-ID: <419ff00b-75a4-9e4a-ce03-081e5b32fe59@redhat.com> Hi Erik, > So I get what you are saying right, we are using the bool > needs_explicit_null_check(intptr_t offset) function both with offsets > (what the parameter seems to suggest should be passed in), but also with > raw addresses from the signal handler, posing as offsets. And we > happened to get away with that because the check was written in such a > clever way that it would return the right answer regardless of whether > it was an offset or an address, saving us from writing those ~3 lines > extra code for a new function expecting an address. > > Except now with negative offsets, that also additionally get masked by > the AArch64 linux signal handler, this suddenly falls apart and we can > no longer seamlessly pass in offsets and addresses to get consistent > answers. The same function suddenly handles different combinations of > masked/unmasked positive/negative addresses, with/without heap base, as > well as positive/negative offsets. > > The first obvious suggestion that comes to mind is to sign extend the > high order byte inside of the signal handler on AArch64, to fix the > address before it gets passed in to needs_explicit_null_check, as > opposed to letting the shared needs_explicit_null_check function take > that responsibility, handling not just offsets and virtual addresses, > but additionally also masked virtual addresses from the signal handler. > > So with the AArch64 stuff gone from this function, I still can't help > but think that the conditions in this code would be possible to write in > a trivially understandable way that doesn't require me to scratch my > head, if there was one function expecting addresses and one function > expecting offsets. And in that case, I think it would look trivial to > have a cell header check in those functions, which would fit in well. I > think. > > Thoughts? By splitting it up we'd have a simple case (offset known), and a complex case (we may get a lea'd address). The simple case seems to be a special case of the complex one, and the complex one would look exactly like what we have today. We'd add no value by extracting the simple case. Right? It might help to rename the "offset" argument to something that fits better though. I agree about sign-extending the address in aarch64 signal handler. This would simplify the method. Roman > Thanks, > /Erik > > On 2018-11-02 23:31, Roman Kennke wrote: >> Hi Erik, >> >> I had a discussion with Andrew Dinn (who actually wrote that code back >> then) and will copy his explanation here, I think it covers the >> situation: >> >> "" >> Let's just recap without considering Shenandoah: >> >> MacroAssembler::needs_explicit_null_check is called from several places, >> not just the SEGV handler. >> >> In those other cases offset is a putative field offset for an object. >> >> With uncompressed pointers the method has to decide whether this offset >> will lie within the read-protected page at address 0 i.e. 0 <= offset < >> os::vm_page_size(). If that is true then a null check can be omitted for >> a read or write operation. If not an explicit check is needed. >> >> With compressed pointers when Universe::narrow_oop_base() != NULL the >> test also checks whether this offset may lie within the read-protected >> page at the heap base only -- oh -- that's the same test! Of course, >> we'll see that this is different when the SEGV handler makes the call. >> >> n.b. I think this case is included because there is the possibility that >> a null pointer can be represented by the narrow oop base address. >> However, I think in reality special case checks in all encode decode >> operations ensure that a null oop is always represented as zero (for >> AArch64 at least). I'm not absolutely sure of that though. >> >> Now, when called from the SEGV handler offset is actually the offending >> read/write address that caused the SEGV. In that case a null pointer >> will only have been dereferenced if the address lies in the protected >> zero page or in the protected heap base page. The first if clause >> handles the compressed pointer case and reduces it to the uncompressed >> pointer case by subtracting base when offset >= base. >> >> So, either way the offending address is potentially a null pointer >> dereference under exactly the same conditions as for those other calls. >> >> Next, how does Shenandoah modify this? Well, it looks like that depends >> on how this method is called. If it can be invoked from any of those >> other places to check whether a Brooks pointer read or write is ok to >> validate with an implicit null check then the offset may be passed as -8 >> i.e. 0xfffffffffffffff8. However, that's actually not enough. >> >> When invoked from the handler because of an access at of (0 + offset) >> then offset may legitimately be 0x00fffffffffffff8 or lie between 0 and >> os::vm_page_size(). >> >> When invoked form the handler because of a dereference of >> (narrow_oop_base + offset) then the subtract i the if clause means that >> offset may legitimately be 0xfffffffffffffff8 or lie between 0 and >> os::vm_page_size(). >> >> So, the masking in the Shenandoah case is to reduce those two special >> cases into one. >> "" >> >> Roman >> >> >> >>> Hi Roman, >>> >>> On 2018-11-01 17:58, Roman Kennke wrote: >>>> Hi Erik, >>>> >>>>> Would you mind explaining how you need to override this and why? I'm >>>>> afraid I did not quite get it from your description in the RFC >>>>> (which is >>>>> also why I didn't come up with a solution either). >>>>> >>>>> Am I correct that you need to return false if the passed in offset is >>>>> the cell header offset -8, or is there anything else at all >>>>> required to >>>>> that logic? >>>> No, it's really just that. Plus take care of it in the case of combined >>>> narrow-oops-offset plus -8. >>>>> You mentioned something about the high order byte being >>>>> masked on AArch64, but didn't quite connect the dot to how that is >>>>> related to this code. Is it? >>>> Yes, we also need to mask the high order byte in AArch64 because of the >>>> way addressing works on that platform, and because -8 flips on those >>>> negative bits. >>> Okay. Looking at >>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >>> >>> that you posted before, assuming this is how it will be extended. >>> >>> So in the case with a heap base, the resulting address will never be >>> negative, and you know it will trap (presuming there is an mprotected >>> page at offset -8 from the heap base pointer. So it seems in this case, >>> the check could have just been offset == -8. >>> >>> So let's say we access null -8 without a heap base then. The address >>> becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the >>> signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address >>> masking of the high order byte. I presume that is what the AArch64 code >>> is dealing with. But I don't quite understand why. Addresses starting >>> with 0x00FF are not valid in user space so we never need to worry about >>> any memory being mapped in there accidentally. So I'm not sure what this >>> code protects against. >>> >>> So that takes me back to my original intuition: isn't this the same as >>> checking at the very top: if (offset == BrooksPointer::byte_offset()) >>> return false; >>> >>> If it really is as simple as just that, and I did not miss something not >>> obvious, then perhaps we would be better off abstracting a cell header >>> check here, as opposed to wrapping the whole thing in a virtual member >>> function. Especially since there are now a number of these occurrences >>> where some basic knowledge about cell header size that is 0 for all GCs >>> except Shenandoah where it is 8, leaves us wrapping a whole bunch of >>> stuff behind a GC interface, only to handle that one difference. >>> >>> While I don't know exactly what this cell abstraction would reasonably >>> look like in its full shape, perhaps we could start with something >>> simple like a virtual member function size_t cell_header_size() on >>> CollectedHeap with a suitable comment explaining that a cell header is a >>> GC thing we sometimes put above the object header. And then see if there >>> are more related functions that lead us to a helper class for cells or >>> something like that. >>> >>> I'm open to suggestions of course. Thought I'd check with you if this >>> sounds sane to you. Of course if my assumption is wrong that this check >>> could be written much simpler than it looks like, then I suppose I need >>> to give up on that idea. It all boils down to that really. >>> >>> Thanks, >>> /Erik >>> >>>> Also, the mach5 job came back with FAILED (see below). Can somebody >>>> with >>>> access check and see what's up? >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>> Build Details: 2018-11-01-1146402.roman.source >>>> 0 Failed Tests >>>> Mach5 Tasks Results Summary >>>> >>>> ????? EXECUTED_WITH_FAILURE: 6 >>>> ????? KILLED: 0 >>>> ????? UNABLE_TO_RUN: 21 >>>> ????? PASSED: 48 >>>> ????? FAILED: 0 >>>> ????? NA: 0 >>>> ????? Build >>>> >>>> ????????? 6 Executed with failure >>>> ????????????? solaris-sparcv9-solaris-sparcv9-build-8 error while >>>> building, return value: 2 >>>> ????????????? solaris-sparcv9-debug-solaris-sparcv9-build-9 error while >>>> building, return value: 2 >>>> ????????????? windows-x64-windows-x64-build-10 error while building, >>>> return value: 2 >>>> ????????????? windows-x64-debug-windows-x64-build-11 error while >>>> building, >>>> return value: 2 >>>> ????????????? windows-x64-open-windows-x64-build-12 error while >>>> building, >>>> return value: 2 >>>> ????????????? windows-x64-open-debug-windows-x64-build-13 error while >>>> building, return value: 2 >>>> ????????? 2 Not run >>>> ????????????? solaris-sparcv9-install-solaris-sparcv9-build-16 >>>> Dependency >>>> task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>> ????????????? windows-x64-install-windows-x64-build-17 Dependency task >>>> failed: YJftjiBUYc >>>> >>>> ????? Test >>>> >>>> ????????? 19 Not run >>>> >>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 >>>> >>>> >>>> Dependency task failed: >>>> mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>> >>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 >>>> >>>> >>>> Dependency task failed: >>>> mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 >>>> >>>> tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 >>>> Dependency task failed: YJftjiBUYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 >>>> >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 >>>> >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 >>>> >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 >>>> >>>> >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 >>>> Dependency task failed: YJftjiBVYc >>>> ????????????? See all 19... >>>> >>>> >>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-11-01 12:20, Roman Kennke wrote: >>>>>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>>>>> >>>>>> Erik: ok for you too? >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> right. Fixed this, and what what Kim mentioned plus a missing >>>>>>>> include: >>>>>>>> >>>>>>>> Incremental: >>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>>>>> Full: >>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>>>>> >>>>>>>> Ok now? >>>>>>> Looks good. >>>>>>> > From rkennke at redhat.com Mon Nov 5 15:25:06 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 5 Nov 2018 16:25:06 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE040E6.8080103@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> Message-ID: <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> Another way to look at it is this: The method checks if: 1. An offset that, would it be dereferenced on NULL (0) hits the protected 0-page (0<=offset Hi Roman, > > So I get what you are saying right, we are using the bool > needs_explicit_null_check(intptr_t offset) function both with offsets > (what the parameter seems to suggest should be passed in), but also with > raw addresses from the signal handler, posing as offsets. And we > happened to get away with that because the check was written in such a > clever way that it would return the right answer regardless of whether > it was an offset or an address, saving us from writing those ~3 lines > extra code for a new function expecting an address. > > Except now with negative offsets, that also additionally get masked by > the AArch64 linux signal handler, this suddenly falls apart and we can > no longer seamlessly pass in offsets and addresses to get consistent > answers. The same function suddenly handles different combinations of > masked/unmasked positive/negative addresses, with/without heap base, as > well as positive/negative offsets. > > The first obvious suggestion that comes to mind is to sign extend the > high order byte inside of the signal handler on AArch64, to fix the > address before it gets passed in to needs_explicit_null_check, as > opposed to letting the shared needs_explicit_null_check function take > that responsibility, handling not just offsets and virtual addresses, > but additionally also masked virtual addresses from the signal handler. > > So with the AArch64 stuff gone from this function, I still can't help > but think that the conditions in this code would be possible to write in > a trivially understandable way that doesn't require me to scratch my > head, if there was one function expecting addresses and one function > expecting offsets. And in that case, I think it would look trivial to > have a cell header check in those functions, which would fit in well. I > think. > > Thoughts? > > Thanks, > /Erik > > On 2018-11-02 23:31, Roman Kennke wrote: >> Hi Erik, >> >> I had a discussion with Andrew Dinn (who actually wrote that code back >> then) and will copy his explanation here, I think it covers the >> situation: >> >> "" >> Let's just recap without considering Shenandoah: >> >> MacroAssembler::needs_explicit_null_check is called from several places, >> not just the SEGV handler. >> >> In those other cases offset is a putative field offset for an object. >> >> With uncompressed pointers the method has to decide whether this offset >> will lie within the read-protected page at address 0 i.e. 0 <= offset < >> os::vm_page_size(). If that is true then a null check can be omitted for >> a read or write operation. If not an explicit check is needed. >> >> With compressed pointers when Universe::narrow_oop_base() != NULL the >> test also checks whether this offset may lie within the read-protected >> page at the heap base only -- oh -- that's the same test! Of course, >> we'll see that this is different when the SEGV handler makes the call. >> >> n.b. I think this case is included because there is the possibility that >> a null pointer can be represented by the narrow oop base address. >> However, I think in reality special case checks in all encode decode >> operations ensure that a null oop is always represented as zero (for >> AArch64 at least). I'm not absolutely sure of that though. >> >> Now, when called from the SEGV handler offset is actually the offending >> read/write address that caused the SEGV. In that case a null pointer >> will only have been dereferenced if the address lies in the protected >> zero page or in the protected heap base page. The first if clause >> handles the compressed pointer case and reduces it to the uncompressed >> pointer case by subtracting base when offset >= base. >> >> So, either way the offending address is potentially a null pointer >> dereference under exactly the same conditions as for those other calls. >> >> Next, how does Shenandoah modify this? Well, it looks like that depends >> on how this method is called. If it can be invoked from any of those >> other places to check whether a Brooks pointer read or write is ok to >> validate with an implicit null check then the offset may be passed as -8 >> i.e. 0xfffffffffffffff8. However, that's actually not enough. >> >> When invoked from the handler because of an access at of (0 + offset) >> then offset may legitimately be 0x00fffffffffffff8 or lie between 0 and >> os::vm_page_size(). >> >> When invoked form the handler because of a dereference of >> (narrow_oop_base + offset) then the subtract i the if clause means that >> offset may legitimately be 0xfffffffffffffff8 or lie between 0 and >> os::vm_page_size(). >> >> So, the masking in the Shenandoah case is to reduce those two special >> cases into one. >> "" >> >> Roman >> >> >> >>> Hi Roman, >>> >>> On 2018-11-01 17:58, Roman Kennke wrote: >>>> Hi Erik, >>>> >>>>> Would you mind explaining how you need to override this and why? I'm >>>>> afraid I did not quite get it from your description in the RFC >>>>> (which is >>>>> also why I didn't come up with a solution either). >>>>> >>>>> Am I correct that you need to return false if the passed in offset is >>>>> the cell header offset -8, or is there anything else at all >>>>> required to >>>>> that logic? >>>> No, it's really just that. Plus take care of it in the case of combined >>>> narrow-oops-offset plus -8. >>>>> You mentioned something about the high order byte being >>>>> masked on AArch64, but didn't quite connect the dot to how that is >>>>> related to this code. Is it? >>>> Yes, we also need to mask the high order byte in AArch64 because of the >>>> way addressing works on that platform, and because -8 flips on those >>>> negative bits. >>> Okay. Looking at >>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >>> >>> that you posted before, assuming this is how it will be extended. >>> >>> So in the case with a heap base, the resulting address will never be >>> negative, and you know it will trap (presuming there is an mprotected >>> page at offset -8 from the heap base pointer. So it seems in this case, >>> the check could have just been offset == -8. >>> >>> So let's say we access null -8 without a heap base then. The address >>> becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the >>> signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address >>> masking of the high order byte. I presume that is what the AArch64 code >>> is dealing with. But I don't quite understand why. Addresses starting >>> with 0x00FF are not valid in user space so we never need to worry about >>> any memory being mapped in there accidentally. So I'm not sure what this >>> code protects against. >>> >>> So that takes me back to my original intuition: isn't this the same as >>> checking at the very top: if (offset == BrooksPointer::byte_offset()) >>> return false; >>> >>> If it really is as simple as just that, and I did not miss something not >>> obvious, then perhaps we would be better off abstracting a cell header >>> check here, as opposed to wrapping the whole thing in a virtual member >>> function. Especially since there are now a number of these occurrences >>> where some basic knowledge about cell header size that is 0 for all GCs >>> except Shenandoah where it is 8, leaves us wrapping a whole bunch of >>> stuff behind a GC interface, only to handle that one difference. >>> >>> While I don't know exactly what this cell abstraction would reasonably >>> look like in its full shape, perhaps we could start with something >>> simple like a virtual member function size_t cell_header_size() on >>> CollectedHeap with a suitable comment explaining that a cell header is a >>> GC thing we sometimes put above the object header. And then see if there >>> are more related functions that lead us to a helper class for cells or >>> something like that. >>> >>> I'm open to suggestions of course. Thought I'd check with you if this >>> sounds sane to you. Of course if my assumption is wrong that this check >>> could be written much simpler than it looks like, then I suppose I need >>> to give up on that idea. It all boils down to that really. >>> >>> Thanks, >>> /Erik >>> >>>> Also, the mach5 job came back with FAILED (see below). Can somebody >>>> with >>>> access check and see what's up? >>>> >>>> Thanks, >>>> Roman >>>> >>>> >>>> Build Details: 2018-11-01-1146402.roman.source >>>> 0 Failed Tests >>>> Mach5 Tasks Results Summary >>>> >>>> ????? EXECUTED_WITH_FAILURE: 6 >>>> ????? KILLED: 0 >>>> ????? UNABLE_TO_RUN: 21 >>>> ????? PASSED: 48 >>>> ????? FAILED: 0 >>>> ????? NA: 0 >>>> ????? Build >>>> >>>> ????????? 6 Executed with failure >>>> ????????????? solaris-sparcv9-solaris-sparcv9-build-8 error while >>>> building, return value: 2 >>>> ????????????? solaris-sparcv9-debug-solaris-sparcv9-build-9 error while >>>> building, return value: 2 >>>> ????????????? windows-x64-windows-x64-build-10 error while building, >>>> return value: 2 >>>> ????????????? windows-x64-debug-windows-x64-build-11 error while >>>> building, >>>> return value: 2 >>>> ????????????? windows-x64-open-windows-x64-build-12 error while >>>> building, >>>> return value: 2 >>>> ????????????? windows-x64-open-debug-windows-x64-build-13 error while >>>> building, return value: 2 >>>> ????????? 2 Not run >>>> ????????????? solaris-sparcv9-install-solaris-sparcv9-build-16 >>>> Dependency >>>> task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>> ????????????? windows-x64-install-windows-x64-build-17 Dependency task >>>> failed: YJftjiBUYc >>>> >>>> ????? Test >>>> >>>> ????????? 19 Not run >>>> >>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 >>>> >>>> >>>> Dependency task failed: >>>> mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>> >>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 >>>> >>>> >>>> Dependency task failed: >>>> mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 >>>> >>>> tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 >>>> Dependency task failed: YJftjiBUYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 >>>> >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 >>>> >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 >>>> >>>> Dependency >>>> task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 >>>> >>>> >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 >>>> Dependency task failed: YJftjiBVYc >>>> >>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 >>>> Dependency task failed: YJftjiBVYc >>>> ????????????? See all 19... >>>> >>>> >>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-11-01 12:20, Roman Kennke wrote: >>>>>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>>>>> >>>>>> Erik: ok for you too? >>>>>> >>>>>> Thanks, >>>>>> Roman >>>>>> >>>>>> >>>>>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>> right. Fixed this, and what what Kim mentioned plus a missing >>>>>>>> include: >>>>>>>> >>>>>>>> Incremental: >>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>>>>> Full: >>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>>>>> >>>>>>>> Ok now? >>>>>>> Looks good. >>>>>>> > From martin.doerr at sap.com Mon Nov 5 15:41:03 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 5 Nov 2018 15:41:03 +0000 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> References: <5BC9E8BA.50407@oracle.com> <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> Message-ID: <9697fcc126d34725982e4826a035e035@sap.com> Hi Erik, we observe crashes on Big Endian PPC64 machines since this change was pushed. Taking a look at the IsUnloadingUnion which looks incorrect to me. Note that sizeof(IsUnloadingUnion) = sizeof(IsUnloadingStruct) = 4 on PPC64. On Big Endian, the 8 bit _value and the _inflated bits are disjoint. I haven't checked SPARC, yet. Can you take a look, please? Thanks and best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of Erik Osterlund Sent: Montag, 5. November 2018 07:20 To: David Holmes Cc: hotspot-dev developers Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent Hi David, Yes that lock is now unused. Sorry I forgot to delete it! /Erik > On 4 Nov 2018, at 22:38, David Holmes wrote: > > Hi Erik, > > Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) > > I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. > > Thanks, > David > >> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >> Hi, >> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >> Bug ID: >> https://bugs.openjdk.java.net/browse/JDK-8209189 >> Webrev: >> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >> Thanks, >> /Erik From erik.osterlund at oracle.com Mon Nov 5 15:52:36 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 5 Nov 2018 16:52:36 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <9697fcc126d34725982e4826a035e035@sap.com> References: <5BC9E8BA.50407@oracle.com> <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> <9697fcc126d34725982e4826a035e035@sap.com> Message-ID: <5BE06744.806@oracle.com> Hi Martin, Interesting. The issue seems to be that I assumed that this struct: struct IsUnloadingStruct { unsigned int _is_unloading:1; unsigned int _unloading_cycle:2; }; ...would be one byte large, as I explicitly asked for just 3 bits for both bit fields combined. If that really has the size of 4, then that is problematic indeed. But I am not sure how it could be 4 bytes, when I explicitly sized the number of bits I want in the integers so that I would only need 3 bits, with the closest type holding that being a byte. Is this happening with the xlc compiler only? Do I need some packing attribute to the struct to make it pack the field into a byte? Does it help if the integers are declared as uint8_t instead? I have not observed any problems on SPARC. Thanks, /Erik On 2018-11-05 16:41, Doerr, Martin wrote: > Hi Erik, > > we observe crashes on Big Endian PPC64 machines since this change was pushed. > > Taking a look at the IsUnloadingUnion which looks incorrect to me. > Note that sizeof(IsUnloadingUnion) = sizeof(IsUnloadingStruct) = 4 on PPC64. > On Big Endian, the 8 bit _value and the _inflated bits are disjoint. > > I haven't checked SPARC, yet. > > Can you take a look, please? > > Thanks and best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of Erik Osterlund > Sent: Montag, 5. November 2018 07:20 > To: David Holmes > Cc: hotspot-dev developers > Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent > > Hi David, > > Yes that lock is now unused. Sorry I forgot to delete it! > > /Erik > >> On 4 Nov 2018, at 22:38, David Holmes wrote: >> >> Hi Erik, >> >> Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) >> >> I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. >> >> Thanks, >> David >> >>> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >>> Hi, >>> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >>> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >>> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >>> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >>> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >>> Bug ID: >>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>> Webrev: >>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>> Thanks, >>> /Erik From martin.doerr at sap.com Mon Nov 5 16:11:36 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 5 Nov 2018 16:11:36 +0000 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BE06744.806@oracle.com> References: <5BC9E8BA.50407@oracle.com> <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> <9697fcc126d34725982e4826a035e035@sap.com> <5BE06744.806@oracle.com> Message-ID: <04a9290c3a0244849144e52228f10269@sap.com> Hi Erik, gdb shows sizeof(IsUnloadingUnion) = 4 on linux x86_64, PPC64 BE, PPC64 LE and s390. I havent't used a debugger on AIX, but I believe it's the same there. Unfortunately, C++ doesn't guarantee a certain size. So it's incorrect for BE platforms in general. Best regards, Martin -----Original Message----- From: Erik ?sterlund Sent: Montag, 5. November 2018 16:53 To: Doerr, Martin Cc: hotspot-dev developers Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent Hi Martin, Interesting. The issue seems to be that I assumed that this struct: struct IsUnloadingStruct { unsigned int _is_unloading:1; unsigned int _unloading_cycle:2; }; ...would be one byte large, as I explicitly asked for just 3 bits for both bit fields combined. If that really has the size of 4, then that is problematic indeed. But I am not sure how it could be 4 bytes, when I explicitly sized the number of bits I want in the integers so that I would only need 3 bits, with the closest type holding that being a byte. Is this happening with the xlc compiler only? Do I need some packing attribute to the struct to make it pack the field into a byte? Does it help if the integers are declared as uint8_t instead? I have not observed any problems on SPARC. Thanks, /Erik On 2018-11-05 16:41, Doerr, Martin wrote: > Hi Erik, > > we observe crashes on Big Endian PPC64 machines since this change was pushed. > > Taking a look at the IsUnloadingUnion which looks incorrect to me. > Note that sizeof(IsUnloadingUnion) = sizeof(IsUnloadingStruct) = 4 on PPC64. > On Big Endian, the 8 bit _value and the _inflated bits are disjoint. > > I haven't checked SPARC, yet. > > Can you take a look, please? > > Thanks and best regards, > Martin > > > -----Original Message----- > From: hotspot-dev On Behalf Of Erik Osterlund > Sent: Montag, 5. November 2018 07:20 > To: David Holmes > Cc: hotspot-dev developers > Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent > > Hi David, > > Yes that lock is now unused. Sorry I forgot to delete it! > > /Erik > >> On 4 Nov 2018, at 22:38, David Holmes wrote: >> >> Hi Erik, >> >> Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) >> >> I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. >> >> Thanks, >> David >> >>> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >>> Hi, >>> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >>> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >>> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >>> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >>> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >>> Bug ID: >>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>> Webrev: >>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>> Thanks, >>> /Erik From erik.osterlund at oracle.com Mon Nov 5 16:18:05 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 5 Nov 2018 17:18:05 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <04a9290c3a0244849144e52228f10269@sap.com> References: <5BC9E8BA.50407@oracle.com> <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> <9697fcc126d34725982e4826a035e035@sap.com> <5BE06744.806@oracle.com> <04a9290c3a0244849144e52228f10269@sap.com> Message-ID: <5BE06D3D.5070804@oracle.com> Hi Martin, Oh snap. If I change the IsUnloadingStruct to use __attribute__((packed)), it becomes one byte as I expected. Does that work for you? Thanks, /Erik On 2018-11-05 17:11, Doerr, Martin wrote: > Hi Erik, > > gdb shows sizeof(IsUnloadingUnion) = 4 on linux x86_64, PPC64 BE, PPC64 LE and s390. > I havent't used a debugger on AIX, but I believe it's the same there. > > Unfortunately, C++ doesn't guarantee a certain size. So it's incorrect for BE platforms in general. > > Best regards, > Martin > > > -----Original Message----- > From: Erik ?sterlund > Sent: Montag, 5. November 2018 16:53 > To: Doerr, Martin > Cc: hotspot-dev developers > Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent > > Hi Martin, > > Interesting. The issue seems to be that I assumed that this struct: > > struct IsUnloadingStruct { > unsigned int _is_unloading:1; > unsigned int _unloading_cycle:2; > }; > > ...would be one byte large, as I explicitly asked for just 3 bits for > both bit fields combined. If that really has the size of 4, then that is > problematic indeed. But I am not sure how it could be 4 bytes, when I > explicitly sized the number of bits I want in the integers so that I > would only need 3 bits, with the closest type holding that being a byte. > > Is this happening with the xlc compiler only? Do I need some packing > attribute to the struct to make it pack the field into a byte? Does it > help if the integers are declared as uint8_t instead? > > I have not observed any problems on SPARC. > > Thanks, > /Erik > > On 2018-11-05 16:41, Doerr, Martin wrote: >> Hi Erik, >> >> we observe crashes on Big Endian PPC64 machines since this change was pushed. >> >> Taking a look at the IsUnloadingUnion which looks incorrect to me. >> Note that sizeof(IsUnloadingUnion) = sizeof(IsUnloadingStruct) = 4 on PPC64. >> On Big Endian, the 8 bit _value and the _inflated bits are disjoint. >> >> I haven't checked SPARC, yet. >> >> Can you take a look, please? >> >> Thanks and best regards, >> Martin >> >> >> -----Original Message----- >> From: hotspot-dev On Behalf Of Erik Osterlund >> Sent: Montag, 5. November 2018 07:20 >> To: David Holmes >> Cc: hotspot-dev developers >> Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent >> >> Hi David, >> >> Yes that lock is now unused. Sorry I forgot to delete it! >> >> /Erik >> >>> On 4 Nov 2018, at 22:38, David Holmes wrote: >>> >>> Hi Erik, >>> >>> Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) >>> >>> I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. >>> >>> Thanks, >>> David >>> >>>> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >>>> Hi, >>>> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >>>> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >>>> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >>>> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >>>> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >>>> Bug ID: >>>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>>> Webrev: >>>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>>> Thanks, >>>> /Erik From martin.doerr at sap.com Mon Nov 5 16:39:56 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 5 Nov 2018 16:39:56 +0000 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <5BE06D3D.5070804@oracle.com> References: <5BC9E8BA.50407@oracle.com> <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> <9697fcc126d34725982e4826a035e035@sap.com> <5BE06744.806@oracle.com> <04a9290c3a0244849144e52228f10269@sap.com> <5BE06D3D.5070804@oracle.com> Message-ID: <2ea571725e7e4327ad679495e09691bd@sap.com> Hi Erik, this seems to be a valid workaround for GCC: I got sizeof(IsUnloadingUnion) = 1 But it's just a GCC hack. Best regards, Martin -----Original Message----- From: Erik ?sterlund Sent: Montag, 5. November 2018 17:18 To: Doerr, Martin Cc: hotspot-dev developers Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent Hi Martin, Oh snap. If I change the IsUnloadingStruct to use __attribute__((packed)), it becomes one byte as I expected. Does that work for you? Thanks, /Erik On 2018-11-05 17:11, Doerr, Martin wrote: > Hi Erik, > > gdb shows sizeof(IsUnloadingUnion) = 4 on linux x86_64, PPC64 BE, PPC64 LE and s390. > I havent't used a debugger on AIX, but I believe it's the same there. > > Unfortunately, C++ doesn't guarantee a certain size. So it's incorrect for BE platforms in general. > > Best regards, > Martin > > > -----Original Message----- > From: Erik ?sterlund > Sent: Montag, 5. November 2018 16:53 > To: Doerr, Martin > Cc: hotspot-dev developers > Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent > > Hi Martin, > > Interesting. The issue seems to be that I assumed that this struct: > > struct IsUnloadingStruct { > unsigned int _is_unloading:1; > unsigned int _unloading_cycle:2; > }; > > ...would be one byte large, as I explicitly asked for just 3 bits for > both bit fields combined. If that really has the size of 4, then that is > problematic indeed. But I am not sure how it could be 4 bytes, when I > explicitly sized the number of bits I want in the integers so that I > would only need 3 bits, with the closest type holding that being a byte. > > Is this happening with the xlc compiler only? Do I need some packing > attribute to the struct to make it pack the field into a byte? Does it > help if the integers are declared as uint8_t instead? > > I have not observed any problems on SPARC. > > Thanks, > /Erik > > On 2018-11-05 16:41, Doerr, Martin wrote: >> Hi Erik, >> >> we observe crashes on Big Endian PPC64 machines since this change was pushed. >> >> Taking a look at the IsUnloadingUnion which looks incorrect to me. >> Note that sizeof(IsUnloadingUnion) = sizeof(IsUnloadingStruct) = 4 on PPC64. >> On Big Endian, the 8 bit _value and the _inflated bits are disjoint. >> >> I haven't checked SPARC, yet. >> >> Can you take a look, please? >> >> Thanks and best regards, >> Martin >> >> >> -----Original Message----- >> From: hotspot-dev On Behalf Of Erik Osterlund >> Sent: Montag, 5. November 2018 07:20 >> To: David Holmes >> Cc: hotspot-dev developers >> Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent >> >> Hi David, >> >> Yes that lock is now unused. Sorry I forgot to delete it! >> >> /Erik >> >>> On 4 Nov 2018, at 22:38, David Holmes wrote: >>> >>> Hi Erik, >>> >>> Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) >>> >>> I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. >>> >>> Thanks, >>> David >>> >>>> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >>>> Hi, >>>> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >>>> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >>>> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >>>> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >>>> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >>>> Bug ID: >>>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>>> Webrev: >>>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>>> Thanks, >>>> /Erik From daniel.daugherty at oracle.com Mon Nov 5 16:44:03 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 5 Nov 2018 11:44:03 -0500 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> Message-ID: <8bbbcea0-b3f8-86d6-7ec9-6446b7a18f1f@oracle.com> On 11/4/18 8:43 PM, David Holmes wrote: > This impacts GC, compiler, runtime and serviceability. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 > webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ src/hotspot/share/gc/shared/parallelCleaning.cpp ??? No comments. src/hotspot/share/gc/shared/parallelCleaning.hpp ??? No comments. src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp ??? No comments. src/hotspot/share/runtime/mutexLocker.cpp ??? L324: ? def(JfrStacktrace_lock?????????? , PaddedMutex? , special,???? true,? Monitor::_safepoint_check_sometimes); ??? L325: ? def(JfrThreadSampler_lock, PaddedMonitor, leaf, true, Monitor::_safepoint_check_never); ??????? Please match the spaces before comma format used on L324. ??? L334: ? def(NMethodSweeper_stat_lock???? , PaddedMutex? , special,???? true,? Monitor::_safepoint_check_sometimes); ??? L336: ? def(Decoder_shared_decoder_lock,?? PaddedMutex, native,????? false, Monitor::_safepoint_check_never); ??? L337: ? def(DCmdFactory_lock,????????????? PaddedMutex, leaf,??????? true,? Monitor::_safepoint_check_never); ??????? Please match the spaces before comma format used on L334. src/hotspot/share/runtime/mutexLocker.hpp ??? No comments. src/hotspot/share/runtime/sweeper.cpp ??? No comments. src/hotspot/share/runtime/sweeper.hpp ??? No comments. src/hotspot/share/runtime/threadSMR.cpp ??? No comments. src/hotspot/share/runtime/threadSMR.hpp ??? No comments. src/hotspot/share/services/diagnosticFramework.cpp ??? No comments. src/hotspot/share/services/diagnosticFramework.hpp ??? No comments. src/hotspot/share/utilities/decoder.cpp ??? L100: ? MutexLockerEx locker(error_handling_thread ? NULL : shared_decoder_lock(), true); ??? L110: ? MutexLockerEx locker(error_handling_thread ? NULL : shared_decoder_lock(), true); ??? L121: ? MutexLockerEx locker(error_handling_thread ? NULL : shared_decoder_lock(), true); ??????? Not your mess, but those literal 'true' params should be replaced ??????? with: Mutex::_no_safepoint_check_flag ??????? Since you are touching these lines, would you mind fixing it? src/hotspot/share/utilities/decoder.hpp ??? No comments. Thumbs up. If you choose to fix the nits, I do not need to see another webrev. Dan > > To facilitate changes to the Mutex/Monitor code we need to ensure > there are no statically initialized Monitor/Mutex instances in the VM > - as the constructors may rely on VM initialization that has not > happened when C++ initializers execute. > > There were 6 locally defined "lock" members of classes that were > statically initialized, and these are across all areas of the VM: > > ?- Decoder::_shared_decoder_lock > ?- DCmdFactory::_dcmdFactory_lock > ?- JfrThreadSampler::_transition_block_lock > ?- NMethodSweeper::_stat_lock > ?- ThreadsSMRSupport::_delete_lock > ?- CodeCacheUnloadingTask::_lock > > The last is actually now unused and so is just deleted. The rest join > all other Monitor/Mutex instances defined in the global list in > MutexLocker and initialized in mutex_init(). I've verified that none > of the monitor/mutex instances can be used prior to mutex_init, by > code inspection - more details in the bug report. > > I investigated asserting that a Monitor/Mutex is only constructed > during mutex_init() but assertions don't fail cleanly during C++ > static initialization - the result is that during the build jmod goes > into an infinite loop printing out "[Too many errors, abort]". > > Tested using tier1-3 > > Thanks, > David From erik.joelsson at oracle.com Mon Nov 5 16:57:42 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 5 Nov 2018 08:57:42 -0800 Subject: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency In-Reply-To: References: <3095bbe5-6e76-eafc-b8a2-5766bfe273e4@oracle.com> <7b29f755-0404-0581-a188-95ac8b7a62e1@oracle.com> <42E17EFC-8E6C-4032-A136-529A96E72E93@oracle.com> <507b7651-f9fc-65f1-9ee4-9f6fe391246f@oracle.com> <10a9ab67-81c6-3cf2-9f42-817574b4f4dc@oracle.com> Message-ID: On 2018-11-05 01:41, Magnus Ihse Bursie wrote: > If I were to spend more time on PCH, I'd rather spend it on either > Windows (which is the slowest build platform in our CI), or Solaris, > which does not have precompiled headers at all. (I actually looked > into solstudio PCH as part of this fix to see if the new set of > headers would allow us to enable it. Unfortunately, it seems like they > need a single PCH per directory of source code files (e.g. all files > in share/logging can share PCH with each other, but they cannot share > PCH with the files in share/utilities. As I understood the problem, at > least; the error messages were unclear and undocumented.) > I looked into this before. There were several limitations that made it very tricky. The compiler was extremely picky with command lines matching, including something with directories, which is probably what you describe (I don't remember well at this point). I managed to hack around it to get some C++ files compiled with a PCH so I could get a feel for the potential speedup. It seemed like the best gain I would get was about 10% per compiled file. I did not think that was enough gain to motivate the amount of hackery required to get it to actually work. Looking inside the PCH from Solaris Studio, they were basically just preprocessed header files, no binaries. /Erik > /Magnus > >> >> Cheers, >> David >> >>> Thanks, Thomas >>> >>>> That would also remove the problem of things not compiling with PCH >>>> disabled as a single set of includes would be used for both PCH and >>>> non-PCH building. >>>> >>>> I don't know if the other compilers support PCH in a similar manner >>>> to gcc. >>>> >>>> Cheers, >>>> David >>>> >>>>> /Magnus >>>>> >>>>>> >>>>>> David >>>>>> >>>>>>> ?? 25 // Precompiled headers are turned off for Sun Studio, >>>>>>> May as well change to Solaris Studio if you're going to fix the >>>>>>> typo :) >>>>>>> Thanks. >>>>>>> David >>>>>>>> On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote: >>>>>>>> The reasons for the current set of files included in >>>>>>>> precompiled.hpp is somewhat lost in the mists of history. >>>>>>>> However, it is clear that it is not optimal. >>>>>>>> >>>>>>>> This patch replaces the current set with a new set, based on >>>>>>>> how often a header file is included in a C++ file. This >>>>>>>> selection contains all header files that are included by at >>>>>>>> least 130 C++ files. Testing has shown that this is around the >>>>>>>> optimal value -- include many more, and too many "innocent" >>>>>>>> files get hurt by unneeded work, leave out many more, and we >>>>>>>> miss out on optimization possibilities. >>>>>>>> >>>>>>>> The same set turned out to work well for both clang and gcc. >>>>>>>> However, files named "*.inline.hpp" did hurt rather than help >>>>>>>> performance, so those have been left out. For visual studio, >>>>>>>> the same set was also optimal, as long as the inline files were >>>>>>>> included. Presumably, visual studio is better than gcc/clang on >>>>>>>> handling precompiled headers containing inlined code. >>>>>>>> >>>>>>>> Here are some rough comparisons from our internal CI system, >>>>>>>> for building the target "hotspot" from scratch. >>>>>>>> >>>>>>>> macosx-x64: >>>>>>>> old: 00:05:00 >>>>>>>> new: 00:03:47 >>>>>>>> >>>>>>>> linux-x64: >>>>>>>> old: 00:05:43 >>>>>>>> new: 00:04:51 >>>>>>>> >>>>>>>> windows-x64: >>>>>>>> old: 00:05:18 >>>>>>>> new: 00:04:33 >>>>>>>> >>>>>>>> linux-aarch64: >>>>>>>> old: 00:07:57 >>>>>>>> new: 00:03:48 >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213339 >>>>>>>> WebRev: >>>>>>>> http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01 >>>>>>>> >>>>>>>> /Magnus >>>>> > From erik.osterlund at oracle.com Mon Nov 5 17:53:34 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 5 Nov 2018 18:53:34 +0100 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <2ea571725e7e4327ad679495e09691bd@sap.com> References: <5BC9E8BA.50407@oracle.com> <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> <9697fcc126d34725982e4826a035e035@sap.com> <5BE06744.806@oracle.com> <04a9290c3a0244849144e52228f10269@sap.com> <5BE06D3D.5070804@oracle.com> <2ea571725e7e4327ad679495e09691bd@sap.com> Message-ID: <1bc698e4-80b8-d05a-2649-41f1a2fde2b3@oracle.com> Hi Martin, I think this fantastic macro should support struct packing on all of our compilers based on what I have read in compiler manuals (from solaris studio, xlc, gcc, clang and msvc): http://cr.openjdk.java.net/~eosterlund/tmp/webrev.04/ And in case a new tool chain is added not supporting struct packing, the static assert will fire at compile time. With full compiler support for struct packing, would you still consider it a hack to rely on it? I think it would be a bit sad if you could not rely on struct packing working, despite not being standard in general. But I am open to other opinions. Thanks, /Erik On 2018-11-05 17:39, Doerr, Martin wrote: > Hi Erik, > > this seems to be a valid workaround for GCC: I got sizeof(IsUnloadingUnion) = 1 > But it's just a GCC hack. > > Best regards, > Martin > > > -----Original Message----- > From: Erik ?sterlund > Sent: Montag, 5. November 2018 17:18 > To: Doerr, Martin > Cc: hotspot-dev developers > Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent > > Hi Martin, > > Oh snap. If I change the IsUnloadingStruct to use > __attribute__((packed)), it becomes one byte as I expected. Does that > work for you? > > Thanks, > /Erik > > On 2018-11-05 17:11, Doerr, Martin wrote: >> Hi Erik, >> >> gdb shows sizeof(IsUnloadingUnion) = 4 on linux x86_64, PPC64 BE, PPC64 LE and s390. >> I havent't used a debugger on AIX, but I believe it's the same there. >> >> Unfortunately, C++ doesn't guarantee a certain size. So it's incorrect for BE platforms in general. >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Erik ?sterlund >> Sent: Montag, 5. November 2018 16:53 >> To: Doerr, Martin >> Cc: hotspot-dev developers >> Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent >> >> Hi Martin, >> >> Interesting. The issue seems to be that I assumed that this struct: >> >> struct IsUnloadingStruct { >> unsigned int _is_unloading:1; >> unsigned int _unloading_cycle:2; >> }; >> >> ...would be one byte large, as I explicitly asked for just 3 bits for >> both bit fields combined. If that really has the size of 4, then that is >> problematic indeed. But I am not sure how it could be 4 bytes, when I >> explicitly sized the number of bits I want in the integers so that I >> would only need 3 bits, with the closest type holding that being a byte. >> >> Is this happening with the xlc compiler only? Do I need some packing >> attribute to the struct to make it pack the field into a byte? Does it >> help if the integers are declared as uint8_t instead? >> >> I have not observed any problems on SPARC. >> >> Thanks, >> /Erik >> >> On 2018-11-05 16:41, Doerr, Martin wrote: >>> Hi Erik, >>> >>> we observe crashes on Big Endian PPC64 machines since this change was pushed. >>> >>> Taking a look at the IsUnloadingUnion which looks incorrect to me. >>> Note that sizeof(IsUnloadingUnion) = sizeof(IsUnloadingStruct) = 4 on PPC64. >>> On Big Endian, the 8 bit _value and the _inflated bits are disjoint. >>> >>> I haven't checked SPARC, yet. >>> >>> Can you take a look, please? >>> >>> Thanks and best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: hotspot-dev On Behalf Of Erik Osterlund >>> Sent: Montag, 5. November 2018 07:20 >>> To: David Holmes >>> Cc: hotspot-dev developers >>> Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent >>> >>> Hi David, >>> >>> Yes that lock is now unused. Sorry I forgot to delete it! >>> >>> /Erik >>> >>>> On 4 Nov 2018, at 22:38, David Holmes wrote: >>>> >>>> Hi Erik, >>>> >>>> Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) >>>> >>>> I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. >>>> >>>> Thanks, >>>> David >>>> >>>>> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >>>>> Hi, >>>>> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >>>>> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >>>>> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >>>>> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >>>>> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >>>>> Bug ID: >>>>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>>>> Webrev: >>>>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>>>> Thanks, >>>>> /Erik > From dean.long at oracle.com Mon Nov 5 18:33:20 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 5 Nov 2018 10:33:20 -0800 Subject: RFR(M) 8212605: Pure-Java implementation of AccessController.doPrivileged In-Reply-To: <3f15e776-4aae-b3a7-302a-0c451d73a0c1@oracle.com> References: <0e3759bf-9c6f-dd75-bb27-5579e23c5348@oracle.com> <2768f4be-458a-f78d-89ab-55450cec5915@oracle.com> <46bff210-18fb-474e-42f3-0a277d955b84@oracle.com> <3f15e776-4aae-b3a7-302a-0c451d73a0c1@oracle.com> Message-ID: <5c9afc7f-3ada-f95c-1110-db9bbc553536@oracle.com> Thanks Alan. dl On 11/4/18 1:03 AM, Alan Bateman wrote: > On 03/11/2018 20:00, dean.long at oracle.com wrote: >> I made a pass at improving the comments based on feedback I've >> received.? I updated webrev.4 in place, along with an incremental diff: > I looked through the updated webrev.4, mostly studying the changes in > AccessController.java and jvm.cpp and the changes look good to me. > > -Alan. From kim.barrett at oracle.com Mon Nov 5 20:11:05 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 5 Nov 2018 15:11:05 -0500 Subject: RFR: 8210986: Add OopStorage cleanup to ServiceThread In-Reply-To: <7e997976-e91a-faaf-8673-ee32a43038fa@oracle.com> References: <02593BDB-F99C-4854-A4F6-9F3A0EC631AA@oracle.com> <7e997976-e91a-faaf-8673-ee32a43038fa@oracle.com> Message-ID: > On Nov 5, 2018, at 4:56 AM, Robbin Ehn wrote: > > Hi Kim, > > Seems reasonable, I must admit I hate the MutexUnlocker, but I see way you choose to that pattern here. Thanks. As you might guess, without MutexUnlocker the code structure becomes more complicated. > It's not obvious to me why you needed to change the: > 644 OrderAccess::storeload(); > to > 668 OrderAccess::fence(); I?m finding it a little bit less difficult to reason about code that consistently uses acquire/release/fence, without the other ordering forms. From bsrbnd at gmail.com Mon Nov 5 21:21:02 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 5 Nov 2018 22:21:02 +0100 Subject: Bit set intrinsic In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Wed, 31 Oct 2018 at 15:51, B. Blaser wrote: > > The last but not least, I implemented the c2 part (using the 8-bit > AND/OR variant) to do sharper comparisons also on non-concurrent > execution: > > http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.02/ > > With 10e6 iterations the lock latency seems to be more or less > negligible and removing it would make the intrinsic about 10% faster > than BitSet without synchronization. Which actually seems to be due to the following missing ANDB/ORB patterns in x86_64.ad: instruct andB_mem_rReg(memory dst, rRegI src, rFlagsReg cr) %{ match(Set dst (StoreB dst (AndI (LoadB dst) src))); effect(KILL cr); ins_cost(150); format %{ "andb $dst, $src\t# byte" %} opcode(0x20); ins_encode(REX_breg_mem(src, dst), OpcP, reg_mem(src, dst)); ins_pipe(ialu_mem_reg); %} instruct orB_mem_rReg(memory dst, rRegI src, rFlagsReg cr) %{ match(Set dst (StoreB dst (OrI (LoadB dst) src))); effect(KILL cr); ins_cost(150); format %{ "orb $dst, $src\t# byte" %} opcode(0x08); ins_encode(REX_breg_mem(src, dst), OpcP, reg_mem(src, dst)); ins_pipe(ialu_mem_reg); %} The next two lines: 1) bits[index>>>3] |= (byte)(1 << (index & 7)); 2) bits[index>>>3] &= (byte)~(1 << (index & 7)); where assembled as: 1) 024 movsbl R8, [RSI + #16 + R10] # byte 02a movl R11, #1 # int 030 sall R11, RCX 033 movsbl R11, R11 # i2b 037 orl R11, R8 # int 03a movb [RSI + #16 + R10], R11 # byte 2) 024 movsbl R8, [RSI + #16 + R10] # byte 02a movl R11, #1 # int 030 sall R11, RCX 033 not R11 036 movsbl R11, R11 # i2b 03a andl R8, R11 # int 03d movb [RSI + #16 + R10], R8 # byte instead of: 1) 024 movl R11, #1 # int 02a sall R11, RCX 02d movsbl R11, R11 # i2b 031 orb [RSI + #16 + R10], R11 # byte 2) 024 movl R11, #1 # int 02a sall R11, RCX 02d not R11 030 movsbl R11, R11 # i2b 034 andb [RSI + #16 + R10], R11 # byte So, as first step, I would probably create a JBS issue and send out a RFR on hotspot-dev for this simple enhancement if there are no objections? Any opinion is welcome. Thanks, Bernard From rkennke at redhat.com Mon Nov 5 21:24:12 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 5 Nov 2018 22:24:12 +0100 Subject: Bit set intrinsic In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: Just want to say that I like this effort. Please go ahead and create an issue and send it out for review. Roman > On Wed, 31 Oct 2018 at 15:51, B. Blaser wrote: >> >> The last but not least, I implemented the c2 part (using the 8-bit >> AND/OR variant) to do sharper comparisons also on non-concurrent >> execution: >> >> http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.02/ >> >> With 10e6 iterations the lock latency seems to be more or less >> negligible and removing it would make the intrinsic about 10% faster >> than BitSet without synchronization. > > Which actually seems to be due to the following missing ANDB/ORB > patterns in x86_64.ad: > > instruct andB_mem_rReg(memory dst, rRegI src, rFlagsReg cr) > %{ > match(Set dst (StoreB dst (AndI (LoadB dst) src))); > effect(KILL cr); > > ins_cost(150); > format %{ "andb $dst, $src\t# byte" %} > opcode(0x20); > ins_encode(REX_breg_mem(src, dst), OpcP, reg_mem(src, dst)); > ins_pipe(ialu_mem_reg); > %} > > instruct orB_mem_rReg(memory dst, rRegI src, rFlagsReg cr) > %{ > match(Set dst (StoreB dst (OrI (LoadB dst) src))); > effect(KILL cr); > > ins_cost(150); > format %{ "orb $dst, $src\t# byte" %} > opcode(0x08); > ins_encode(REX_breg_mem(src, dst), OpcP, reg_mem(src, dst)); > ins_pipe(ialu_mem_reg); > %} > > The next two lines: > 1) bits[index>>>3] |= (byte)(1 << (index & 7)); > 2) bits[index>>>3] &= (byte)~(1 << (index & 7)); > > where assembled as: > 1) > 024 movsbl R8, [RSI + #16 + R10] # byte > 02a movl R11, #1 # int > 030 sall R11, RCX > 033 movsbl R11, R11 # i2b > 037 orl R11, R8 # int > 03a movb [RSI + #16 + R10], R11 # byte > 2) > 024 movsbl R8, [RSI + #16 + R10] # byte > 02a movl R11, #1 # int > 030 sall R11, RCX > 033 not R11 > 036 movsbl R11, R11 # i2b > 03a andl R8, R11 # int > 03d movb [RSI + #16 + R10], R8 # byte > > instead of: > 1) > 024 movl R11, #1 # int > 02a sall R11, RCX > 02d movsbl R11, R11 # i2b > 031 orb [RSI + #16 + R10], R11 # byte > 2) > 024 movl R11, #1 # int > 02a sall R11, RCX > 02d not R11 > 030 movsbl R11, R11 # i2b > 034 andb [RSI + #16 + R10], R11 # byte > > So, as first step, I would probably create a JBS issue and send out a > RFR on hotspot-dev for this simple enhancement if there are no > objections? > > Any opinion is welcome. > > Thanks, > Bernard > From kim.barrett at oracle.com Tue Nov 6 00:07:51 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 5 Nov 2018 19:07:51 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: Message-ID: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> > On Nov 5, 2018, at 7:15 AM, Aleksey Shipilev wrote: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8211926 > > Fix: > http://cr.openjdk.java.net/~shade/8211926/webrev.01/ > > Found this while porting some Shenandoah addons to jdk/jdk. > > I took the liberty of homogenizing the internals of those methods to always call "small" path when > range does not allow us to enter the "large" path. This protects us from accidentally breaking VM > when asserts are disabled, and makes "large" methods usable as drop-ins for "small" ones. > > Testing: hotspot-tier1, new gtest, jdk-submit (running) > > Thanks, > -Aleksey ------------------------------------------------------------------------------ src/hotspot/share/utilities/bitMap.cpp 272 if ((end_full_word < beg_full_word) || (end_full_word - beg_full_word < BitsPerWord)) { The use of BitsPerWord here at on line 289 seems like a units error. What does the number of bits in a word have to do with the number of words that is considered "small"? I suspect the old value of 32 was just a heuristic bound (which may or may not have had any measurements to back it up). Of course, the assertion error message in the old version of set_large_range is also confused; the comparison is in words but the message says bytes. ------------------------------------------------------------------------------ From david.holmes at oracle.com Tue Nov 6 01:48:40 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Nov 2018 11:48:40 +1000 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: <8bbbcea0-b3f8-86d6-7ec9-6446b7a18f1f@oracle.com> References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> <8bbbcea0-b3f8-86d6-7ec9-6446b7a18f1f@oracle.com> Message-ID: <2367335b-24a3-eca8-356a-3e8a8c419b30@oracle.com> Hi Dan, Thanks for looking at this. On 6/11/2018 2:44 AM, Daniel D. Daugherty wrote: > On 11/4/18 8:43 PM, David Holmes wrote: >> This impacts GC, compiler, runtime and serviceability. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 >> webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ > > src/hotspot/share/gc/shared/parallelCleaning.cpp > ??? No comments. > > src/hotspot/share/gc/shared/parallelCleaning.hpp > ??? No comments. > > src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp > ??? No comments. > > src/hotspot/share/runtime/mutexLocker.cpp > ??? L324: ? def(JfrStacktrace_lock?????????? , PaddedMutex? , > special,???? true,? Monitor::_safepoint_check_sometimes); > ??? L325: ? def(JfrThreadSampler_lock, PaddedMonitor, leaf, true, > Monitor::_safepoint_check_never); > ??????? Please match the spaces before comma format used on L324. Fixed > ??? L334: ? def(NMethodSweeper_stat_lock???? , PaddedMutex? , > special,???? true,? Monitor::_safepoint_check_sometimes); > ??? L336: ? def(Decoder_shared_decoder_lock,?? PaddedMutex, > native,????? false, Monitor::_safepoint_check_never); > ??? L337: ? def(DCmdFactory_lock,????????????? PaddedMutex, > leaf,??????? true,? Monitor::_safepoint_check_never); > ??????? Please match the spaces before comma format used on L334. Fixed (hard to get used to deliberately breaking the style rules!) > src/hotspot/share/runtime/mutexLocker.hpp > ??? No comments. > > src/hotspot/share/runtime/sweeper.cpp > ??? No comments. > > src/hotspot/share/runtime/sweeper.hpp > ??? No comments. > > src/hotspot/share/runtime/threadSMR.cpp > ??? No comments. > > src/hotspot/share/runtime/threadSMR.hpp > ??? No comments. > > src/hotspot/share/services/diagnosticFramework.cpp > ??? No comments. > > src/hotspot/share/services/diagnosticFramework.hpp > ??? No comments. > > src/hotspot/share/utilities/decoder.cpp > ??? L100: ? MutexLockerEx locker(error_handling_thread ? NULL : > shared_decoder_lock(), true); > ??? L110: ? MutexLockerEx locker(error_handling_thread ? NULL : > shared_decoder_lock(), true); > ??? L121: ? MutexLockerEx locker(error_handling_thread ? NULL : > shared_decoder_lock(), true); > ??????? Not your mess, but those literal 'true' params should be replaced > ??????? with: Mutex::_no_safepoint_check_flag > ??????? Since you are touching these lines, would you mind fixing it? Sure no problem. But in doing so I noticed an issue in this code - it defines a DecoderLocker class to encapsulate all that stuff but then doesn't use it at all. I'll file a RFE to either use it or lose it. :) > src/hotspot/share/utilities/decoder.hpp > ??? No comments. > > Thumbs up. If you choose to fix the nits, I do not need to > see another webrev. Webrev updated in place for the benefit of later reviewers. Thanks, David > Dan > > >> >> To facilitate changes to the Mutex/Monitor code we need to ensure >> there are no statically initialized Monitor/Mutex instances in the VM >> - as the constructors may rely on VM initialization that has not >> happened when C++ initializers execute. >> >> There were 6 locally defined "lock" members of classes that were >> statically initialized, and these are across all areas of the VM: >> >> ?- Decoder::_shared_decoder_lock >> ?- DCmdFactory::_dcmdFactory_lock >> ?- JfrThreadSampler::_transition_block_lock >> ?- NMethodSweeper::_stat_lock >> ?- ThreadsSMRSupport::_delete_lock >> ?- CodeCacheUnloadingTask::_lock >> >> The last is actually now unused and so is just deleted. The rest join >> all other Monitor/Mutex instances defined in the global list in >> MutexLocker and initialized in mutex_init(). I've verified that none >> of the monitor/mutex instances can be used prior to mutex_init, by >> code inspection - more details in the bug report. >> >> I investigated asserting that a Monitor/Mutex is only constructed >> during mutex_init() but assertions don't fail cleanly during C++ >> static initialization - the result is that during the build jmod goes >> into an infinite loop printing out "[Too many errors, abort]". >> >> Tested using tier1-3 >> >> Thanks, >> David > From david.holmes at oracle.com Tue Nov 6 01:48:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 6 Nov 2018 11:48:54 +1000 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> Message-ID: <00432f1a-e253-3f87-f076-042bf69e051b@oracle.com> Hi Thomas, Thanks for looking at this. On 5/11/2018 11:05 PM, Thomas Schatzl wrote: > Hi, > > On Mon, 2018-11-05 at 11:43 +1000, David Holmes wrote: >> This impacts GC, compiler, runtime and serviceability. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 >> webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ >> >> To facilitate changes to the Mutex/Monitor code we need to ensure >> there are no statically initialized Monitor/Mutex instances in the VM >> - as the constructors may rely on VM initialization that has not >> happened when C++ initializers execute. >> >> There were 6 locally defined "lock" members of classes that were >> statically initialized, and these are across all areas of the VM: >> >> - Decoder::_shared_decoder_lock >> - DCmdFactory::_dcmdFactory_lock >> - JfrThreadSampler::_transition_block_lock >> - NMethodSweeper::_stat_lock >> - ThreadsSMRSupport::_delete_lock >> - CodeCacheUnloadingTask::_lock >> >> The last is actually now unused and so is just deleted. The rest > > Removal of the CodeCacheUnloadingTask lock looks good. I also looked > through the other changes which seems simple enough to me too, but > should obviously be looked at from people from the respective areas > too. > > Some nits: > > Maybe the spacing in mutexLocker.cpp:325 for the JfrThreadSampler_lock > could be fixed though. Yes oversight on my part. (Actually I was sure I'd already fixed that as soon as I pasted it in ... ??? :( ) webrev updated in place with a few other alignment issues fixed. > Also the assert in decoder.cpp:94 seems to be as superfluous to me as > in the other lines where it has been removed. The assert at 94 replaces the asserts at the other lines. I replaced raw uses of _shared_decoder_lock with calls to the function shared_decoder_lock(), which embodies the != NULL assert - which is a good check to ensure initialization truly does happen sufficiently early. In overall effect the assertions remain unchanged. Thanks, David > Thanks, > Thomas > > From vladimir.kozlov at oracle.com Tue Nov 6 01:51:03 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 5 Nov 2018 17:51:03 -0800 Subject: Review request: JDK-8211122: Reduce the number of internal classes made accessible to jdk.unsupported In-Reply-To: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> References: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> Message-ID: <1375f008-56d8-abf4-e073-19b1c0b1d87e@oracle.com> I looked on tests changes in src/jdk.internal.vm.compiler and test/hotspot/jtreg/compiler/jvmci. They look fine. And you ran hotspot tier1-3 which tests AOT, JVMCI and Graal. So your changes should be fine regarding them since you did not get failures. Thanks, Vladimir On 11/1/18 9:16 PM, Mandy Chung wrote: > This patch includes the following changes that will reduce the > number of internal classes made accessible to jdk.unsupported > module via qualified exports. > > 1. move shared secrets to a new jdk.internal.access package. > > jdk.internal.misc package has been a dumping ground for various > kinds of internal APIs and many of which were moved to an appropriate > package. This is a follow-up clean up that moves the shared secrets > and JavaXXXAccess interfaces to jdk.internal.access package. > > 2. add a wrapper class jdk.internal.misc.FileSystemOption to expose > sun.nio.fs.ExtendedOptions for com.sun.nio.file to access > > This eliminates the qualified exports of sun.nio.fs to jdk.unsupported. > > 3. Refactor the implementation of sun.misc.Unsafe::invokeCleaner > to jdk.internal.misc.Unsafe. > > This eliminates the qualified exports of jdk.internal.ref and > sun.nio.ch to jdk.unsupported. > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.02 > > The change is straight forward although quite many files are modified. > Majority of the changes is import statement change or test @modules > change due to the rename. > > I ran hs-tier1-3 and jdk-tier1-3 tests. > > We considered the alternative to define a wrapper class for everything > that jdk.unsupported module depends on in a dedicated package > e.g. jdk.internal.unsupported.? It would need a wrapper for Unsafe, > Signal, SignalHandler and ExtendedOptions.? It means that it would > have 3 classes of each - two similar/duplicated implementations > (one in sun.misc and one in jdk.internal.unsupported) and one > in jdk.internal.misc ([1] is the prototype).? Only a limited number > of files are touched but I think the proposed approach is cleaner. > > Thanks > Mandy > [1] http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.01/ From mandy.chung at oracle.com Tue Nov 6 01:59:37 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 5 Nov 2018 17:59:37 -0800 Subject: Review request: JDK-8211122: Reduce the number of internal classes made accessible to jdk.unsupported In-Reply-To: <1375f008-56d8-abf4-e073-19b1c0b1d87e@oracle.com> References: <0892f7bb-5c8e-e36d-544d-76ab05e6049e@oracle.com> <1375f008-56d8-abf4-e073-19b1c0b1d87e@oracle.com> Message-ID: Thanks for the review, Vladimir.? hotspot tier1-3 and jdk tier1-3 all passed. I will submit another test run as sanity tests before I push. Mandy On 11/5/18 5:51 PM, Vladimir Kozlov wrote: > I looked on tests changes in src/jdk.internal.vm.compiler and > test/hotspot/jtreg/compiler/jvmci. They look fine. > > And you ran hotspot tier1-3 which tests AOT, JVMCI and Graal. So your > changes should be fine regarding them since you did not get failures. > > Thanks, > Vladimir > > On 11/1/18 9:16 PM, Mandy Chung wrote: >> This patch includes the following changes that will reduce the >> number of internal classes made accessible to jdk.unsupported >> module via qualified exports. >> >> 1. move shared secrets to a new jdk.internal.access package. >> >> jdk.internal.misc package has been a dumping ground for various >> kinds of internal APIs and many of which were moved to an appropriate >> package. This is a follow-up clean up that moves the shared secrets >> and JavaXXXAccess interfaces to jdk.internal.access package. >> >> 2. add a wrapper class jdk.internal.misc.FileSystemOption to expose >> sun.nio.fs.ExtendedOptions for com.sun.nio.file to access >> >> This eliminates the qualified exports of sun.nio.fs to jdk.unsupported. >> >> 3. Refactor the implementation of sun.misc.Unsafe::invokeCleaner >> to jdk.internal.misc.Unsafe. >> >> This eliminates the qualified exports of jdk.internal.ref and >> sun.nio.ch to jdk.unsupported. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.02 >> >> The change is straight forward although quite many files are modified. >> Majority of the changes is import statement change or test @modules >> change due to the rename. >> >> I ran hs-tier1-3 and jdk-tier1-3 tests. >> >> We considered the alternative to define a wrapper class for everything >> that jdk.unsupported module depends on in a dedicated package >> e.g. jdk.internal.unsupported.? It would need a wrapper for Unsafe, >> Signal, SignalHandler and ExtendedOptions.? It means that it would >> have 3 classes of each - two similar/duplicated implementations >> (one in sun.misc and one in jdk.internal.unsupported) and one >> in jdk.internal.misc ([1] is the prototype).? Only a limited number >> of files are touched but I think the proposed approach is cleaner. >> >> Thanks >> Mandy >> [1] http://cr.openjdk.java.net/~mchung/jdk12/webrevs/8211122/webrev.01/ From martin.doerr at sap.com Tue Nov 6 08:59:56 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 6 Nov 2018 08:59:56 +0000 Subject: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent In-Reply-To: <1bc698e4-80b8-d05a-2649-41f1a2fde2b3@oracle.com> References: <5BC9E8BA.50407@oracle.com> <3F2CFBE5-CFF6-40F9-B9ED-A81F1A789A1C@oracle.com> <9697fcc126d34725982e4826a035e035@sap.com> <5BE06744.806@oracle.com> <04a9290c3a0244849144e52228f10269@sap.com> <5BE06D3D.5070804@oracle.com> <2ea571725e7e4327ad679495e09691bd@sap.com> <1bc698e4-80b8-d05a-2649-41f1a2fde2b3@oracle.com> Message-ID: Hi Erik, I've seen a crash on SPARC, too (running JCK api tests). I've opened an issue for this problem: https://bugs.openjdk.java.net/browse/JDK-8213411 Feel free to take it. I don't really like such compiler pragmas, but I can test your proposal. Maybe using a class with bit masks and accessors would be better. Is the STATIC_ASSERT really correct? Doesn?t it evaluate sizeof(IsUnloadingStruct) == sizeof(IsUnloadingUnion) => true and then true == (implicit bool conversion) sizeof(uint8_t) => true? Best regards, Martin -----Original Message----- From: Erik ?sterlund Sent: Montag, 5. November 2018 18:54 To: Doerr, Martin Cc: hotspot-dev developers Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent Hi Martin, I think this fantastic macro should support struct packing on all of our compilers based on what I have read in compiler manuals (from solaris studio, xlc, gcc, clang and msvc): http://cr.openjdk.java.net/~eosterlund/tmp/webrev.04/ And in case a new tool chain is added not supporting struct packing, the static assert will fire at compile time. With full compiler support for struct packing, would you still consider it a hack to rely on it? I think it would be a bit sad if you could not rely on struct packing working, despite not being standard in general. But I am open to other opinions. Thanks, /Erik On 2018-11-05 17:39, Doerr, Martin wrote: > Hi Erik, > > this seems to be a valid workaround for GCC: I got sizeof(IsUnloadingUnion) = 1 > But it's just a GCC hack. > > Best regards, > Martin > > > -----Original Message----- > From: Erik ?sterlund > Sent: Montag, 5. November 2018 17:18 > To: Doerr, Martin > Cc: hotspot-dev developers > Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent > > Hi Martin, > > Oh snap. If I change the IsUnloadingStruct to use > __attribute__((packed)), it becomes one byte as I expected. Does that > work for you? > > Thanks, > /Erik > > On 2018-11-05 17:11, Doerr, Martin wrote: >> Hi Erik, >> >> gdb shows sizeof(IsUnloadingUnion) = 4 on linux x86_64, PPC64 BE, PPC64 LE and s390. >> I havent't used a debugger on AIX, but I believe it's the same there. >> >> Unfortunately, C++ doesn't guarantee a certain size. So it's incorrect for BE platforms in general. >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Erik ?sterlund >> Sent: Montag, 5. November 2018 16:53 >> To: Doerr, Martin >> Cc: hotspot-dev developers >> Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent >> >> Hi Martin, >> >> Interesting. The issue seems to be that I assumed that this struct: >> >> struct IsUnloadingStruct { >> unsigned int _is_unloading:1; >> unsigned int _unloading_cycle:2; >> }; >> >> ...would be one byte large, as I explicitly asked for just 3 bits for >> both bit fields combined. If that really has the size of 4, then that is >> problematic indeed. But I am not sure how it could be 4 bytes, when I >> explicitly sized the number of bits I want in the integers so that I >> would only need 3 bits, with the closest type holding that being a byte. >> >> Is this happening with the xlc compiler only? Do I need some packing >> attribute to the struct to make it pack the field into a byte? Does it >> help if the integers are declared as uint8_t instead? >> >> I have not observed any problems on SPARC. >> >> Thanks, >> /Erik >> >> On 2018-11-05 16:41, Doerr, Martin wrote: >>> Hi Erik, >>> >>> we observe crashes on Big Endian PPC64 machines since this change was pushed. >>> >>> Taking a look at the IsUnloadingUnion which looks incorrect to me. >>> Note that sizeof(IsUnloadingUnion) = sizeof(IsUnloadingStruct) = 4 on PPC64. >>> On Big Endian, the 8 bit _value and the _inflated bits are disjoint. >>> >>> I haven't checked SPARC, yet. >>> >>> Can you take a look, please? >>> >>> Thanks and best regards, >>> Martin >>> >>> >>> -----Original Message----- >>> From: hotspot-dev On Behalf Of Erik Osterlund >>> Sent: Montag, 5. November 2018 07:20 >>> To: David Holmes >>> Cc: hotspot-dev developers >>> Subject: Re: RFR: 8209189: Make CompiledMethod::do_unloading more concurrent >>> >>> Hi David, >>> >>> Yes that lock is now unused. Sorry I forgot to delete it! >>> >>> /Erik >>> >>>> On 4 Nov 2018, at 22:38, David Holmes wrote: >>>> >>>> Hi Erik, >>>> >>>> Just a follow-up on this change. It seems that the CodeCacheUnloadingTask::_lock is now unused - is that correct? (You (accidentally?) made it non-private as part of this change.) >>>> >>>> I was moving that lock as part of the static initalization cleanup, but will just delete it if now unused. >>>> >>>> Thanks, >>>> David >>>> >>>>> On 20/10/2018 12:22 AM, Erik ?sterlund wrote: >>>>> Hi, >>>>> Today the nmethods are unloaded either serially or in parallel due to GC (and e.g. class unloading). With ZGC concurrent class unloading, a concurrent fashion is required. Rather than inventing yet a third way of unloading nmethods due to class unloading, it would be ideal if there could be one unified way of doing it that makes everyone happy. >>>>> To solve this problem in a more general way, a new member function on CompiledMethod is added: is_unloading(). It returns whether a CompiledMethod has broken oops. In order to not have to iterate over all the oops every time this question is asked, it caches the result, so it needs to be computed only once per "epoch". Every time a CodeCache unloading is triggered by the GC, a new epoch is started, which forces re-computation of whether the CompiledMethod is_unloading() the first time it is called. >>>>> So by having is_unloading() be lazily evaluated, it is now possible to build a do_unloading() method on nmethod that simply makes the nmethod unloaded if it is_unloading(), and otherwise cleans the various caches. Cleaning e.g. the inline caches of said nmethod, uses the same is_unloading() method on its targets to figure out if the IC cache should be cleaned or not. Again, the epoched caching mechanism makes sure we don't recompute this value. >>>>> The new do_unloading() method may be called in both serial, parallel and concurrent contexts. So I removed the parallel variation of this that we had before, that unnecessarily postponed the unloading due to not having computed whether the nmethod should be unloaded or not. Since that is now computed on-demand lazily, there is no longer a need for postponing this, nor to have two phases for parallel nmethod unloading. >>>>> While there is more work involved in making concurrent nmethod unloading work, this is a good cleanup towards that goal. >>>>> Bug ID: >>>>> https://bugs.openjdk.java.net/browse/JDK-8209189 >>>>> Webrev: >>>>> cr.openjdk.java.net/~eosterlund/8209189/webrev.00/ >>>>> Thanks, >>>>> /Erik > From erik.osterlund at oracle.com Tue Nov 6 10:34:18 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Nov 2018 11:34:18 +0100 Subject: RFR: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) Message-ID: <5BE16E2A.9090107@oracle.com> Hi, In my patch for 8209189 I assumed that the following struct containing a bit field using 3 bits, would be packed to one byte: struct IsUnloadingStruct { unsigned int _is_unloading:1; unsigned int _unloading_cycle:2; }; Turns out though that the size of this 3 bit bitfield is typically 4 bytes, causing trouble on big endian machines. The compilers can be tamed to turn this to a byte (as expected) by using struct packing. So the two most obvious solutions are either 1) Make a struct packing macro, and trust the compilers won't mess up then. 2) Do manual bit fiddling because we do not trust compilers. Martin convinced me to go with #2. Bug: https://bugs.openjdk.java.net/browse/JDK-8213411 Webrev: http://cr.openjdk.java.net/~eosterlund/8213411/webrev.00/ Running a bunch of tests while waiting for review. Thanks, /Erik From martin.doerr at sap.com Tue Nov 6 10:56:30 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 6 Nov 2018 10:56:30 +0000 Subject: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) In-Reply-To: <5BE16E2A.9090107@oracle.com> References: <5BE16E2A.9090107@oracle.com> Message-ID: <6f58c6fd413c4c4e8708cd8561ba452b@sap.com> Hi Erik, thanks for fixing the issue so quickly. Looks good. Note that in if (state_unloading_cycle == CodeCache::unloading_cycle()) { return state_is_unloading; } you could also use "return false" which would be equivalent. Not sure what is better readable. (No need for a new webrev.) Best regards, Martin -----Original Message----- From: Erik ?sterlund Sent: Dienstag, 6. November 2018 11:34 To: hotspot-dev developers Cc: Doerr, Martin Subject: RFR: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) Hi, In my patch for 8209189 I assumed that the following struct containing a bit field using 3 bits, would be packed to one byte: struct IsUnloadingStruct { unsigned int _is_unloading:1; unsigned int _unloading_cycle:2; }; Turns out though that the size of this 3 bit bitfield is typically 4 bytes, causing trouble on big endian machines. The compilers can be tamed to turn this to a byte (as expected) by using struct packing. So the two most obvious solutions are either 1) Make a struct packing macro, and trust the compilers won't mess up then. 2) Do manual bit fiddling because we do not trust compilers. Martin convinced me to go with #2. Bug: https://bugs.openjdk.java.net/browse/JDK-8213411 Webrev: http://cr.openjdk.java.net/~eosterlund/8213411/webrev.00/ Running a bunch of tests while waiting for review. Thanks, /Erik From erik.osterlund at oracle.com Tue Nov 6 11:06:29 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Nov 2018 12:06:29 +0100 Subject: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) In-Reply-To: <6f58c6fd413c4c4e8708cd8561ba452b@sap.com> References: <5BE16E2A.9090107@oracle.com> <6f58c6fd413c4c4e8708cd8561ba452b@sap.com> Message-ID: <5BE175B5.7080506@oracle.com> Hi Martin, Thank you for the review. On 2018-11-06 11:56, Doerr, Martin wrote: > Hi Erik, > > thanks for fixing the issue so quickly. Looks good. > > Note that in > if (state_unloading_cycle == CodeCache::unloading_cycle()) { > return state_is_unloading; > } > you could also use "return false" which would be equivalent. Not sure what is better readable. (No need for a new webrev.) Yeah I think what you are proposing might be easier to read, so I will go with that when pushing. Thanks, /Erik > Best regards, > Martin > > > -----Original Message----- > From: Erik ?sterlund > Sent: Dienstag, 6. November 2018 11:34 > To: hotspot-dev developers > Cc: Doerr, Martin > Subject: RFR: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) > > Hi, > > In my patch for 8209189 I assumed that the following struct containing a > bit field using 3 bits, would be packed to one byte: > > struct IsUnloadingStruct { > unsigned int _is_unloading:1; > unsigned int _unloading_cycle:2; > }; > > Turns out though that the size of this 3 bit bitfield is typically 4 > bytes, causing trouble on big endian machines. The compilers can be > tamed to turn this to a byte (as expected) by using struct packing. > > So the two most obvious solutions are either > 1) Make a struct packing macro, and trust the compilers won't mess up then. > 2) Do manual bit fiddling because we do not trust compilers. > > Martin convinced me to go with #2. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213411 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8213411/webrev.00/ > > Running a bunch of tests while waiting for review. > > Thanks, > /Erik From magnus.ihse.bursie at oracle.com Tue Nov 6 11:16:33 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 6 Nov 2018 12:16:33 +0100 Subject: RFR[XS]: JDK-8213414 Fix incorrect copy constructors in hotspot Message-ID: In JDK-8211073, an attempt was made to add -Wextra to hotspot. That patch was not accepted at the time being, but it did dig out some bugs in the code. While we await JDK-8211073 to be included in the code base (so -Wextra is always on), we can at least fix the bugs it found in some copy constructors. Bug: https://bugs.openjdk.java.net/browse/JDK-8213414 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213414-fix-hotspot-copy-constructors/webrev.01 /Magnus From kim.barrett at oracle.com Tue Nov 6 11:53:29 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 6 Nov 2018 06:53:29 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> Message-ID: <5A378B7C-6F1F-4714-9FA7-D3708285EAA6@oracle.com> > On Nov 5, 2018, at 7:07 PM, Kim Barrett wrote: Some more comments ------------------------------------------------------------------------------ src/hotspot/share/utilities/bitMap.cpp 269 idx_t beg_full_word = word_index_round_up(beg); Pre-existing: Note that word_index_round_up can overflow, not that it's a practical concern. See JDK-8213415. ------------------------------------------------------------------------------ src/hotspot/share/utilities/bitMap.cpp 272 if ((end_full_word < beg_full_word) || (end_full_word - beg_full_word < BitsPerWord)) { Rather than adding an additional check to prevent reaching the underflow, consider something like the following, which doesn?t have any under/overflow to worry about. // Treat range as small if not more than this many words. const idx_t small_word_count = 32; idx_t beg_full_word = word_index(beg); idx_t end_full_word = word_index(end); if (beg_full_word + small_word_count >= end_full_word) { return set_range(beg, end); } if (!is_word_aligned(beg)) { ++beg_full_word; set_range_within_word(beg, bit_index(beg_full_word)); } set_large_range_of_words(beg_full_word, end_full_word); set_range_within_word(bit_index(end_full_word, end); ------------------------------------------------------------------------------ From kim.barrett at oracle.com Tue Nov 6 11:55:21 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 6 Nov 2018 06:55:21 -0500 Subject: RFR[XS]: JDK-8213414 Fix incorrect copy constructors in hotspot In-Reply-To: References: Message-ID: > On Nov 6, 2018, at 6:16 AM, Magnus Ihse Bursie wrote: > > In JDK-8211073, an attempt was made to add -Wextra to hotspot. That patch was not accepted at the time being, but it did dig out some bugs in the code. > > While we await JDK-8211073 to be included in the code base (so -Wextra is always on), we can at least fix the bugs it found in some copy constructors. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213414 > WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213414-fix-hotspot-copy-constructors/webrev.01 > > /Magnus Looks good. From erik.osterlund at oracle.com Tue Nov 6 14:33:49 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Nov 2018 15:33:49 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> Message-ID: <5BE1A64D.9050005@oracle.com> Hi Roman, Yes, the check is indeed the same for checking addresses vs offsets. But only because it has been written in a non-obvious way to allow that. And I think that is a bad idea compared to having the two cases separate, and writing the corresponding code in a more obvious way where these two different quantities can not be mixed. For offsets, we are interested in the following range: [-cell_header_size, os::vm_page_size) For addresses, we are interested in the following ranges: 1) with heap base: [heap_base - cell_header_size, heap_base + os::vm_page_size) 2) without heap base: [-cell_header_size, os::vm_page_size) Just expressing these trivial ranges separately would have been very easy to understand. By baking this into the same check, there are multiple things that make me scratch my head before I understand how/if it works. For example, let's say we request a 32 GB heap that just barely fits with compressed oops encoding. But it just barely does not fit into the lower 32 GB virtual address space because it is off by one page - the NULL page. And simplicity, let's say that then we map the heap with a heap base at address equal to os::vm_page_size(), because that is the heap base we are given from the OS when reserving our heap. Then the compiler asks if it needs a null check for an offset equal to os::vm_page_size() + 8. Then with the current logic, that will cause the offset to get subtracted by os::vm_page_size(), because the offset "looks like an address" despite being an offset. This results in the JIT thinking implicit null checks are fine, as the resulting offset 8 (after normalizing it like an address to an offset even though it is not an address), looks like it will hit the protected page. But it was in fact not fine, as this access will, given a NULL base pointer, perform accesses into the the heap past the first protected page, without trapping. So it seems that in order for the current approach that treats offsets and address the same to work, we have an implicit contract that no offsets observed by the compiler may be greater than the compressed oops heap base, or things will go horribly wrong. Do we actually enforce this contract in any way? I see we have some HeapBaseMinAddress JVM flag that is by default typically 2G. But I see we have also defined a MAX_OBJECT_SIZE like this: #define MAX_OBJECT_SIZE \ ( arrayOopDesc::header_size(T_DOUBLE) * HeapWordSize \ + ((julong)max_jint * sizeof(double)) ) Since it seems like MAX_OBJECT_SIZE is greater than HeapBaseMinAddress, it seems like the contract we have here is not enforced. We can map in the heap at 2G, and have offsets >2G, which will confuse the check to treat offsets like addresses, which can cause us to crash the VM. Do we have another seat belt that makes this okay? Not that I'm aware of. With all offsets now also being "adjusted" by 8 to determine whether to treat things like an address or not, the head scratching gets worse. We now have a whole bunch of different cases handled by the same function, including: 1) normal offsets 2) negative offsets 3) offsets greater than heap base (probably does not work reliably today) 4) unmasked positive addresses before the heap base 5) unmasked positive addresses after the heap base 6) unmasked positive addresses without a heap base 7) unmasked negative addresses without a heap base 8) masked negative addresses without a heap base And the check to treat all these different scenarios as the same is very far from obvious compared to simply checking the offset range vs address range separately, knowing that one gets passed in offsets, the other gets passed in addresses. Here is a patch with a suggested implementation prototype with my take: http://cr.openjdk.java.net/~eosterlund/8213199/webrev.00/ * It performs sign extension in the AArch64 signal handler to fix masked addresses, so we don't need to special case the masked addresses * It splits the address vs offset cases so you don't have to scratch your head as much * It trivially does not rely on where the heap is mapped in to work (corner cases due to blurring offsets "looking like addresses" vs actual addresses) * It supports GCs with non-zero cell header size, in a trivially understandable way Thoughts? Thanks, /Erik On 2018-11-05 16:25, Roman Kennke wrote: > Another way to look at it is this: > > The method checks if: > > 1. An offset that, would it be dereferenced on NULL (0) hits the > protected 0-page (0<=offset 2. An actual address (the fault address in the signal handler) hits the > protected 0-page. > > ... which basically amounts to the exact same check. Extra handling is > there to: > > - handle narrow-oops > - mask aarch64 address (should be moved to signal handler) > - with Shenandoah, handle -8 offset, which would wrap-around the fault > address to 0xFF... if dereferenced on NULL > > The proposed patch seems like a reasonably good solution to me. And I > have tried a bunch of different approaches too (basically keeping the > main 'algorithm' in place, and only add a hook or two to handle > Shenandoah), which didn't work out quite as nicely. > > > Roman > >> Hi Roman, >> >> So I get what you are saying right, we are using the bool >> needs_explicit_null_check(intptr_t offset) function both with offsets >> (what the parameter seems to suggest should be passed in), but also with >> raw addresses from the signal handler, posing as offsets. And we >> happened to get away with that because the check was written in such a >> clever way that it would return the right answer regardless of whether >> it was an offset or an address, saving us from writing those ~3 lines >> extra code for a new function expecting an address. >> >> Except now with negative offsets, that also additionally get masked by >> the AArch64 linux signal handler, this suddenly falls apart and we can >> no longer seamlessly pass in offsets and addresses to get consistent >> answers. The same function suddenly handles different combinations of >> masked/unmasked positive/negative addresses, with/without heap base, as >> well as positive/negative offsets. >> >> The first obvious suggestion that comes to mind is to sign extend the >> high order byte inside of the signal handler on AArch64, to fix the >> address before it gets passed in to needs_explicit_null_check, as >> opposed to letting the shared needs_explicit_null_check function take >> that responsibility, handling not just offsets and virtual addresses, >> but additionally also masked virtual addresses from the signal handler. >> >> So with the AArch64 stuff gone from this function, I still can't help >> but think that the conditions in this code would be possible to write in >> a trivially understandable way that doesn't require me to scratch my >> head, if there was one function expecting addresses and one function >> expecting offsets. And in that case, I think it would look trivial to >> have a cell header check in those functions, which would fit in well. I >> think. >> >> Thoughts? >> >> Thanks, >> /Erik >> >> On 2018-11-02 23:31, Roman Kennke wrote: >>> Hi Erik, >>> >>> I had a discussion with Andrew Dinn (who actually wrote that code back >>> then) and will copy his explanation here, I think it covers the >>> situation: >>> >>> "" >>> Let's just recap without considering Shenandoah: >>> >>> MacroAssembler::needs_explicit_null_check is called from several places, >>> not just the SEGV handler. >>> >>> In those other cases offset is a putative field offset for an object. >>> >>> With uncompressed pointers the method has to decide whether this offset >>> will lie within the read-protected page at address 0 i.e. 0 <= offset < >>> os::vm_page_size(). If that is true then a null check can be omitted for >>> a read or write operation. If not an explicit check is needed. >>> >>> With compressed pointers when Universe::narrow_oop_base() != NULL the >>> test also checks whether this offset may lie within the read-protected >>> page at the heap base only -- oh -- that's the same test! Of course, >>> we'll see that this is different when the SEGV handler makes the call. >>> >>> n.b. I think this case is included because there is the possibility that >>> a null pointer can be represented by the narrow oop base address. >>> However, I think in reality special case checks in all encode decode >>> operations ensure that a null oop is always represented as zero (for >>> AArch64 at least). I'm not absolutely sure of that though. >>> >>> Now, when called from the SEGV handler offset is actually the offending >>> read/write address that caused the SEGV. In that case a null pointer >>> will only have been dereferenced if the address lies in the protected >>> zero page or in the protected heap base page. The first if clause >>> handles the compressed pointer case and reduces it to the uncompressed >>> pointer case by subtracting base when offset >= base. >>> >>> So, either way the offending address is potentially a null pointer >>> dereference under exactly the same conditions as for those other calls. >>> >>> Next, how does Shenandoah modify this? Well, it looks like that depends >>> on how this method is called. If it can be invoked from any of those >>> other places to check whether a Brooks pointer read or write is ok to >>> validate with an implicit null check then the offset may be passed as -8 >>> i.e. 0xfffffffffffffff8. However, that's actually not enough. >>> >>> When invoked from the handler because of an access at of (0 + offset) >>> then offset may legitimately be 0x00fffffffffffff8 or lie between 0 and >>> os::vm_page_size(). >>> >>> When invoked form the handler because of a dereference of >>> (narrow_oop_base + offset) then the subtract i the if clause means that >>> offset may legitimately be 0xfffffffffffffff8 or lie between 0 and >>> os::vm_page_size(). >>> >>> So, the masking in the Shenandoah case is to reduce those two special >>> cases into one. >>> "" >>> >>> Roman >>> >>> >>> >>>> Hi Roman, >>>> >>>> On 2018-11-01 17:58, Roman Kennke wrote: >>>>> Hi Erik, >>>>> >>>>>> Would you mind explaining how you need to override this and why? I'm >>>>>> afraid I did not quite get it from your description in the RFC >>>>>> (which is >>>>>> also why I didn't come up with a solution either). >>>>>> >>>>>> Am I correct that you need to return false if the passed in offset is >>>>>> the cell header offset -8, or is there anything else at all >>>>>> required to >>>>>> that logic? >>>>> No, it's really just that. Plus take care of it in the case of combined >>>>> narrow-oops-offset plus -8. >>>>>> You mentioned something about the high order byte being >>>>>> masked on AArch64, but didn't quite connect the dot to how that is >>>>>> related to this code. Is it? >>>>> Yes, we also need to mask the high order byte in AArch64 because of the >>>>> way addressing works on that platform, and because -8 flips on those >>>>> negative bits. >>>> Okay. Looking at >>>> https://builds.shipilev.net/patch-openjdk-shenandoah-jdk/latest/src/hotspot/share/asm/assembler.cpp.udiff.html >>>> >>>> that you posted before, assuming this is how it will be extended. >>>> >>>> So in the case with a heap base, the resulting address will never be >>>> negative, and you know it will trap (presuming there is an mprotected >>>> page at offset -8 from the heap base pointer. So it seems in this case, >>>> the check could have just been offset == -8. >>>> >>>> So let's say we access null -8 without a heap base then. The address >>>> becomes 0xFFFFFFFFFFFFFFF8. When that traps, the value you see in the >>>> signal handler becomes 0x00FFFFFFFFFFFFF8 because of the virtual address >>>> masking of the high order byte. I presume that is what the AArch64 code >>>> is dealing with. But I don't quite understand why. Addresses starting >>>> with 0x00FF are not valid in user space so we never need to worry about >>>> any memory being mapped in there accidentally. So I'm not sure what this >>>> code protects against. >>>> >>>> So that takes me back to my original intuition: isn't this the same as >>>> checking at the very top: if (offset == BrooksPointer::byte_offset()) >>>> return false; >>>> >>>> If it really is as simple as just that, and I did not miss something not >>>> obvious, then perhaps we would be better off abstracting a cell header >>>> check here, as opposed to wrapping the whole thing in a virtual member >>>> function. Especially since there are now a number of these occurrences >>>> where some basic knowledge about cell header size that is 0 for all GCs >>>> except Shenandoah where it is 8, leaves us wrapping a whole bunch of >>>> stuff behind a GC interface, only to handle that one difference. >>>> >>>> While I don't know exactly what this cell abstraction would reasonably >>>> look like in its full shape, perhaps we could start with something >>>> simple like a virtual member function size_t cell_header_size() on >>>> CollectedHeap with a suitable comment explaining that a cell header is a >>>> GC thing we sometimes put above the object header. And then see if there >>>> are more related functions that lead us to a helper class for cells or >>>> something like that. >>>> >>>> I'm open to suggestions of course. Thought I'd check with you if this >>>> sounds sane to you. Of course if my assumption is wrong that this check >>>> could be written much simpler than it looks like, then I suppose I need >>>> to give up on that idea. It all boils down to that really. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> Also, the mach5 job came back with FAILED (see below). Can somebody >>>>> with >>>>> access check and see what's up? >>>>> >>>>> Thanks, >>>>> Roman >>>>> >>>>> >>>>> Build Details: 2018-11-01-1146402.roman.source >>>>> 0 Failed Tests >>>>> Mach5 Tasks Results Summary >>>>> >>>>> EXECUTED_WITH_FAILURE: 6 >>>>> KILLED: 0 >>>>> UNABLE_TO_RUN: 21 >>>>> PASSED: 48 >>>>> FAILED: 0 >>>>> NA: 0 >>>>> Build >>>>> >>>>> 6 Executed with failure >>>>> solaris-sparcv9-solaris-sparcv9-build-8 error while >>>>> building, return value: 2 >>>>> solaris-sparcv9-debug-solaris-sparcv9-build-9 error while >>>>> building, return value: 2 >>>>> windows-x64-windows-x64-build-10 error while building, >>>>> return value: 2 >>>>> windows-x64-debug-windows-x64-build-11 error while >>>>> building, >>>>> return value: 2 >>>>> windows-x64-open-windows-x64-build-12 error while >>>>> building, >>>>> return value: 2 >>>>> windows-x64-open-debug-windows-x64-build-13 error while >>>>> building, return value: 2 >>>>> 2 Not run >>>>> solaris-sparcv9-install-solaris-sparcv9-build-16 >>>>> Dependency >>>>> task failed: mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>>> windows-x64-install-windows-x64-build-17 Dependency task >>>>> failed: YJftjiBUYc >>>>> >>>>> Test >>>>> >>>>> 19 Not run >>>>> >>>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-57 >>>>> >>>>> >>>>> Dependency task failed: >>>>> mach5...-8300-solaris-sparcv9-solaris-sparcv9-build-8 >>>>> >>>>> tier1-solaris-sparc-open_test_hotspot_jtreg_tier1_common-solaris-sparcv9-debug-58 >>>>> >>>>> >>>>> Dependency task failed: >>>>> mach5...solaris-sparcv9-debug-solaris-sparcv9-build-9 >>>>> >>>>> tier1-product-open_test_hotspot_jtreg_tier1_common-windows-x64-20 >>>>> Dependency task failed: YJftjiBUYc >>>>> >>>>> tier1-debug-open_test_hotspot_jtreg_tier1_common-windows-x64-debug-26 >>>>> Dependency task failed: YJftjiBVYc >>>>> >>>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-windows-x64-debug-29 >>>>> >>>>> Dependency >>>>> task failed: YJftjiBVYc >>>>> >>>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-windows-x64-debug-32 >>>>> >>>>> Dependency >>>>> task failed: YJftjiBVYc >>>>> >>>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-windows-x64-debug-35 >>>>> >>>>> Dependency >>>>> task failed: YJftjiBVYc >>>>> >>>>> tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-windows-x64-debug-38 >>>>> >>>>> >>>>> Dependency task failed: YJftjiBVYc >>>>> >>>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-windows-x64-debug-41 >>>>> Dependency task failed: YJftjiBVYc >>>>> >>>>> tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-windows-x64-debug-44 >>>>> Dependency task failed: YJftjiBVYc >>>>> See all 19... >>>>> >>>>> >>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-11-01 12:20, Roman Kennke wrote: >>>>>>> Hi Kim, thanks for reviewing! I'll push it through jdk/submit now. >>>>>>> >>>>>>> Erik: ok for you too? >>>>>>> >>>>>>> Thanks, >>>>>>> Roman >>>>>>> >>>>>>> >>>>>>>>> On Oct 31, 2018, at 6:14 PM, Roman Kennke >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi Erik, >>>>>>>>> >>>>>>>>> right. Fixed this, and what what Kim mentioned plus a missing >>>>>>>>> include: >>>>>>>>> >>>>>>>>> Incremental: >>>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01.diff/ >>>>>>>>> Full: >>>>>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.01/ >>>>>>>>> >>>>>>>>> Ok now? >>>>>>>> Looks good. >>>>>>>> From boris.ulasevich at bell-sw.com Tue Nov 6 14:43:07 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Tue, 6 Nov 2018 17:43:07 +0300 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system Message-ID: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> Hi all, Please review this patch to fix jtreg @requires vm.opt.final.UseCompressedOops flag evaluation fail reproduced on ARM32: "invalid boolean value: null for expression vm.opt.final.UseCompressedOops". http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8213410 The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok by default, and becomes disabled with -javaoptions:"-XX:-UseCompressedOops" jtreg option). Thanks, Boris Ulasevich From adinn at redhat.com Tue Nov 6 15:13:05 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 6 Nov 2018 15:13:05 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE1A64D.9050005@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> Message-ID: <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> Hi Erik, On 06/11/18 14:33, Erik ?sterlund wrote: > For offsets, we are interested in the following range: > [-cell_header_size, os::vm_page_size) > > For addresses, we are interested in the following ranges: > 1) with heap base: [heap_base - cell_header_size, heap_base + > os::vm_page_size) > 2) without heap base: [-cell_header_size, os::vm_page_size) I don't think that is correct. Even when we are using a heap_base null oops are still represented by address 0. So, the address range of interest in case 1 is [-cell_header_size, os::vm_page_size). That's the value used when a 'null' value is instantiated in memory. It is also the value tested for by encode_heap_oop when it decides to write 0 into an object field to represent a null oop and is also the value returned by decode_heap_oop when it detects 0 in an object field. In that case, I hear you ask, why does the old code check for values in the range [heap_base - cell_header_size, heap_base + os::vm_page_size)? Good question. I don't know the answer. I think that test is bogus with no Shenandoah GC present. So, anyway, this means your code for uses_implicit_null_check in assembler.cpp is borked. > Just expressing these trivial ranges separately would have been very > easy to understand. Except it's not that easy. Also, with Shenandoah there is a new wrinkle. It is possible for an object to be allocated at the first non-guarded address in the heap area i.e. heap_base + os::vm_page_size. I that case a dereference of its Brooks pointer will lie in the protected page. > Thoughts? I am agnostic as to whether this is done using one or two functions. However, besides the above issue I believe there is a small error in your rewrite of os_linux_aarch64.cpp. At line 259 you correct si_addr to or back in the top byte if bit 55 is non-zero. 359 address addr = (address) info->si_addr; 360 361 // Make sure the high order byte is sign extended, as it may be masked away by the hardware. 362 if ((uintptr_t(addr) & (uintptr_t(1) << 55)) != 0) { 363 addr = address(uintptr_t(addr) | (uintptr_t(0xFF) << 56)); 364 } 365 However, at line 463 you fail to pass the corrected address to uses_implicit_null_check 462 } else if (sig == SIGSEGV && 463 MacroAssembler::uses_implicit_null_check(info->si_addr)) { regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From rkennke at redhat.com Tue Nov 6 15:48:44 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 6 Nov 2018 16:48:44 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> Message-ID: > Except it's not that easy. Also, with Shenandoah there is a new wrinkle. > It is possible for an object to be allocated at the first non-guarded > address in the heap area i.e. heap_base + os::vm_page_size. I that case > a dereference of its Brooks pointer will lie in the protected page. No, that is not possible. We always allocate the first object at heap_start + 8, and place the first fwdptr at heap_start. Doing what you described would be a gross error, if I understood you correctly. Roman > >> Thoughts? > > I am agnostic as to whether this is done using one or two functions. > However, besides the above issue I believe there is a small error in > your rewrite of os_linux_aarch64.cpp. At line 259 you correct si_addr to > or back in the top byte if bit 55 is non-zero. > > 359 address addr = (address) info->si_addr; > 360 > 361 // Make sure the high order byte is sign extended, as it may be > masked away by the hardware. > 362 if ((uintptr_t(addr) & (uintptr_t(1) << 55)) != 0) { > 363 addr = address(uintptr_t(addr) | (uintptr_t(0xFF) << 56)); > 364 } > 365 > > However, at line 463 you fail to pass the corrected address to > uses_implicit_null_check > > 462 } else if (sig == SIGSEGV && > 463 > MacroAssembler::uses_implicit_null_check(info->si_addr)) { > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From erik.osterlund at oracle.com Tue Nov 6 16:04:55 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Nov 2018 17:04:55 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> Message-ID: <5BE1BBA7.40309@oracle.com> Hi Andrew, New webrev: http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ On 2018-11-06 16:13, Andrew Dinn wrote: > Hi Erik, > > On 06/11/18 14:33, Erik ?sterlund wrote: > >> For offsets, we are interested in the following range: >> [-cell_header_size, os::vm_page_size) >> >> For addresses, we are interested in the following ranges: >> 1) with heap base: [heap_base - cell_header_size, heap_base + >> os::vm_page_size) >> 2) without heap base: [-cell_header_size, os::vm_page_size) > I don't think that is correct. Even when we are using a heap_base null > oops are still represented by address 0. So, the address range of > interest in case 1 is [-cell_header_size, os::vm_page_size). > > That's the value used when a 'null' value is instantiated in memory. It > is also the value tested for by encode_heap_oop when it decides to write > 0 into an object field to represent a null oop and is also the value > returned by decode_heap_oop when it detects 0 in an object field. > > In that case, I hear you ask, why does the old code check for values in > the range [heap_base - cell_header_size, heap_base + os::vm_page_size)? > Good question. I don't know the answer. I think that test is bogus with > no Shenandoah GC present. Interesting. It turns out that both ranges are relevant [-cell_header_size, os::vm_page_size) and [heap_base - cell_header_size, heap_base + os::vm_page_size). Looks like we both missed that from the original check. I updated my webrev to explicitly reflect those two ranges, as two different checks. > So, anyway, this means your code for uses_implicit_null_check in > assembler.cpp is borked. Yup. Good catch. >> Just expressing these trivial ranges separately would have been very >> easy to understand. > Except it's not that easy. Also, with Shenandoah there is a new wrinkle. > It is possible for an object to be allocated at the first non-guarded > address in the heap area i.e. heap_base + os::vm_page_size. I that case > a dereference of its Brooks pointer will lie in the protected page. Looks like Roman just said that's not possible. >> Thoughts? > I am agnostic as to whether this is done using one or two functions. > However, besides the above issue I believe there is a small error in > your rewrite of os_linux_aarch64.cpp. At line 259 you correct si_addr to > or back in the top byte if bit 55 is non-zero. > > 359 address addr = (address) info->si_addr; > 360 > 361 // Make sure the high order byte is sign extended, as it may be > masked away by the hardware. > 362 if ((uintptr_t(addr) & (uintptr_t(1) << 55)) != 0) { > 363 addr = address(uintptr_t(addr) | (uintptr_t(0xFF) << 56)); > 364 } > 365 > > However, at line 463 you fail to pass the corrected address to > uses_implicit_null_check > > 462 } else if (sig == SIGSEGV && > 463 > MacroAssembler::uses_implicit_null_check(info->si_addr)) { Fixed. Might need some help testing this on AArch64 though if we decide to go with it. Do you agree though, that the current version that merges the address and offset case relies on offsets being smaller than the heap base for its magic check to work, and that we actually do not have such a guarantee in the VM? Apart from readability preferences, this seems like it is broken in the current form, unless there are seat belts I am not aware of. In particular, if you have an offset value that is in the virtual address range (heap_base, heap_base + os::vm_page_size()), which I argue can happen, then it looks like we will normalize that as if it was an address (even though it is not), and incorrectly say that this offset does not need explicit null checks, despite needing explicit null checks. Thanks, /Erik > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From vladimir.kozlov at oracle.com Tue Nov 6 16:21:50 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 6 Nov 2018 08:21:50 -0800 Subject: RFR: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) In-Reply-To: <5BE16E2A.9090107@oracle.com> References: <5BE16E2A.9090107@oracle.com> Message-ID: Hi Erik, It would be nice if you use enum for values, masks and shift (like for markOop). It will become more readable. Thanks, Vladimir On 11/6/18 2:34 AM, Erik ?sterlund wrote: > Hi, > > In my patch for 8209189 I assumed that the following struct containing a bit field using 3 bits, would be packed to one > byte: > > struct IsUnloadingStruct { > ? unsigned int _is_unloading:1; > ? unsigned int _unloading_cycle:2; > }; > > Turns out though that the size of this 3 bit bitfield is typically 4 bytes, causing trouble on big endian machines. The > compilers can be tamed to turn this to a byte (as expected) by using struct packing. > > So the two most obvious solutions are either > 1) Make a struct packing macro, and trust the compilers won't mess up then. > 2) Do manual bit fiddling because we do not trust compilers. > > Martin convinced me to go with #2. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213411 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8213411/webrev.00/ > > Running a bunch of tests while waiting for review. > > Thanks, > /Erik From adinn at redhat.com Tue Nov 6 16:46:29 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 6 Nov 2018 16:46:29 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE1BBA7.40309@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> Message-ID: On 06/11/18 16:04, Erik ?sterlund wrote: > New webrev: > http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ Thanks for the updated version. > On 2018-11-06 16:13, Andrew Dinn wrote: >> In that case, I hear you ask, why does the old code check for values in >> the range [heap_base - cell_header_size, heap_base + os::vm_page_size)? >> Good question. I don't know the answer. I think that test is bogus with >> no Shenandoah GC present. > > Interesting. It turns out that both ranges are relevant > [-cell_header_size, os::vm_page_size) and [heap_base - cell_header_size, > heap_base + os::vm_page_size). Looks like we both missed that from the > original check. > I updated my webrev to explicitly reflect those two ranges, as two > different checks. Well, you say that but -- as I said before -- I am not sure why *both* are relevant to the non-Shenandoah case. And, given Roman's correction (doh, yes of course! the space for the Brooks pointer field had better lie in RW-mapped memory :-) I don't see why it is relevant to the Shenandoah case either. I'd be very glad to hear from someone who knows about the history of the original function as to why the extra test for addresses above heap_base is included [hint: John Rose? David Holmes? Coleen? anyone else who might have been around when it was added?] If that check is not actually needed then getting rid of it would make the whole thing a lot simpler. >>> Just expressing these trivial ranges separately would have been very >>> easy to understand. >> Except it's not that easy. Also, with Shenandoah there is a new wrinkle. >> It is possible for an object to be allocated at the first non-guarded >> address in the heap area i.e. heap_base + os::vm_page_size. I that case >> a dereference of its Brooks pointer will lie in the protected page. > > Looks like Roman just said that's not possible. Yes, my dumb! > Fixed. Might need some help testing this on AArch64 though if we decide > to go with it. I am sure Roman and Aleksey can do that but if needed I will. > Do you agree though, that the current version that merges the address > and offset case relies on offsets being smaller than the heap base for > its magic check to work, and that we actually do not have such a > guarantee in the VM? Apart from readability preferences, this seems like > it is broken in the current form, unless there are seat belts I am not > aware of. In particular, if you have an offset value that is in the > virtual address range (heap_base, heap_base + os::vm_page_size()), which > I argue can happen, then it looks like we will normalize that as if it > was an address (even though it is not), and incorrectly say that this > offset does not need explicit null checks, despite needing explicit null > checks. Well, without the answer as to whether the check against heap_base is needed I am not sure. Assuming it is then then ... Yes, it is possible, in theory, that an oop offset could turn out to have value (heap_base + D) where 0 <= D < os_page_size(). In that case the offset might wrongly be classified as *not* needing a null check even though it really does. I suspect it is unlikely in practice given how mmap hands out addresses but you never know. As for readibility, I think the two functions are probably clearer -- expecially if they stay adjacent in the source code. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From erik.osterlund at oracle.com Tue Nov 6 17:20:43 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 6 Nov 2018 18:20:43 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> Message-ID: Hi Andrew, On 2018-11-06 17:46, Andrew Dinn wrote: > On 06/11/18 16:04, Erik ?sterlund wrote: >> New webrev: >> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ > > Thanks for the updated version. > >> On 2018-11-06 16:13, Andrew Dinn wrote: >>> In that case, I hear you ask, why does the old code check for values in >>> the range [heap_base - cell_header_size, heap_base + os::vm_page_size)? >>> Good question. I don't know the answer. I think that test is bogus with >>> no Shenandoah GC present. >> >> Interesting. It turns out that both ranges are relevant >> [-cell_header_size, os::vm_page_size) and [heap_base - cell_header_size, >> heap_base + os::vm_page_size). Looks like we both missed that from the >> original check. >> I updated my webrev to explicitly reflect those two ranges, as two >> different checks. > > Well, you say that but -- as I said before -- I am not sure why *both* > are relevant to the non-Shenandoah case. And, given Roman's correction > (doh, yes of course! the space for the Brooks pointer field had better > lie in RW-mapped memory :-) I don't see why it is relevant to the > Shenandoah case either. The reason it is needed is because there may or may not be null checks in both the decode() step to get the base pointer of an access, and the access itself, given some base pointer. This results in two ranges for those two cases correspondingly. If the decoding uses decode_not_null, with an implicit null check in the decoding itself that blindly adds the heap base without checking for null, we will crash into the first page of the heap which is protected when performing accesses into that. But sometimes we have a decode()ed base pointer (with null check), that has already been fixed up to NULL, and then perform an access at an offset into that with an implicit null check. This results in a trap at the first page. So in summary, both ranges are needed because the implicitness of the null check may be both in the access and the narrowOop decoding. But as I pointed out, this (seemingly incorrectly) assumes there may not be any offset values in the (heap_base, heap_base + os::vm_page_size) range. If you hit such an offset, we will generate incorrect code, due to the lack of distinction between addresses and offsets. > I'd be very glad to hear from someone who knows about the history of the > original function as to why the extra test for addresses above heap_base > is included [hint: John Rose? David Holmes? Coleen? anyone else who > might have been around when it was added?] > > If that check is not actually needed then getting rid of it would make > the whole thing a lot simpler. It would indeed. But unfortunately we need it. >>>> Just expressing these trivial ranges separately would have been very >>>> easy to understand. >>> Except it's not that easy. Also, with Shenandoah there is a new wrinkle. >>> It is possible for an object to be allocated at the first non-guarded >>> address in the heap area i.e. heap_base + os::vm_page_size. I that case >>> a dereference of its Brooks pointer will lie in the protected page. >> >> Looks like Roman just said that's not possible. > > Yes, my dumb! > >> Fixed. Might need some help testing this on AArch64 though if we decide >> to go with it. > > I am sure Roman and Aleksey can do that but if needed I will. > >> Do you agree though, that the current version that merges the address >> and offset case relies on offsets being smaller than the heap base for >> its magic check to work, and that we actually do not have such a >> guarantee in the VM? Apart from readability preferences, this seems like >> it is broken in the current form, unless there are seat belts I am not >> aware of. In particular, if you have an offset value that is in the >> virtual address range (heap_base, heap_base + os::vm_page_size()), which >> I argue can happen, then it looks like we will normalize that as if it >> was an address (even though it is not), and incorrectly say that this >> offset does not need explicit null checks, despite needing explicit null >> checks. > Well, without the answer as to whether the check against heap_base is > needed I am not sure. Assuming it is then then ... > > Yes, it is possible, in theory, that an oop offset could turn out to > have value (heap_base + D) where 0 <= D < os_page_size(). In that case > the offset might wrongly be classified as *not* needing a null check > even though it really does. I suspect it is unlikely in practice given > how mmap hands out addresses but you never know. > > As for readibility, I think the two functions are probably clearer -- > expecially if they stay adjacent in the source code. Okay, sounds like we are on the same page. I agree it is unlikely that this will crash the VM (as evidenced by the lack of crashes observed). But I don't like relying on luck for correctness. Thanks, /Erik > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From adinn at redhat.com Tue Nov 6 17:45:33 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 6 Nov 2018 17:45:33 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> Message-ID: <90d4e32b-fd85-8cb1-a8e9-6bb49f211f0c@redhat.com> On 06/11/18 17:20, Erik ?sterlund wrote: > On 2018-11-06 17:46, Andrew Dinn wrote: . . . >> Well, you say that but -- as I said before -- I am not sure why *both* >> are relevant to the non-Shenandoah case. And, given Roman's correction >> (doh, yes of course! the space for the Brooks pointer field had better >> lie in RW-mapped memory :-) I don't see why it is relevant to the >> Shenandoah case either. > > The reason it is needed is because there may or may not be null checks > in both the decode() step to get the base pointer of an access, and the > access itself, given some base pointer. This results in two ranges for > those two cases correspondingly. If the decoding uses decode_not_null, > with an implicit null check in the decoding itself that blindly adds the > heap base without checking for null, we will crash into the first page > of the heap which is protected when performing accesses into that. But > sometimes we have a decode()ed base pointer (with null check), that has > already been fixed up to NULL, and then perform an access at an offset > into that with an implicit null check. This results in a trap at the > first page. So in summary, both ranges are needed because the > implicitness of the null check may be both in the access and the > narrowOop decoding. Hmm, that doesn't sound quite kosher. If decode_not_null is being called in places where we may potentially load a null value (or rather a zero store value stored in a field) then I would have said that we really ought to crash and then fix the code to use a decode that tests for null. If this is recommended a way to optimize loads of potentially null object fields then it doesn't sound like a reliably performant way to do it. Could you point me at some place in the compiler where narrow oop loads are done using a decode_not_null on the assumption that a load of a null value that gets translated to a low heap_base address will be caught and corrected by a check initiated from the signal handler? >> I'd be very glad to hear from someone who knows about the history of the >> original function as to why the extra test for addresses above heap_base >> is included [hint: John Rose? David Holmes? Coleen? anyone else who >> might have been around when it was added?] >> >> If that check is not actually needed then getting rid of it would make >> the whole thing a lot simpler. > > It would indeed. But unfortunately we need it. I'd still like to hear the history from whoever wrote this code. >> Well, without the answer as to whether the check against heap_base is >> needed I am not sure. Assuming it is then then ... >> >> Yes, it is possible, in theory, that an oop offset could turn out to >> have value (heap_base + D) where 0 <= D < os_page_size(). In that case >> the offset might wrongly be classified as *not* needing a null check >> even though it really does. I suspect it is unlikely in practice given >> how mmap hands out addresses but you never know. >> >> As for readibility, I think the two functions are probably clearer -- >> expecially if they stay adjacent in the source code. > > Okay, sounds like we are on the same page. I agree it is unlikely that > this will crash the VM (as evidenced by the lack of crashes observed). > But I don't like relying on luck for correctness. Absolutely. Luck is not enough. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From kim.barrett at oracle.com Tue Nov 6 18:28:16 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 6 Nov 2018 13:28:16 -0500 Subject: RFR[XS]: JDK-8213414 Fix incorrect copy constructors in hotspot In-Reply-To: References: Message-ID: <3049827A-4C7D-41E7-BE66-2C513C6F0C4D@oracle.com> > On Nov 6, 2018, at 6:55 AM, Kim Barrett wrote: > >> On Nov 6, 2018, at 6:16 AM, Magnus Ihse Bursie wrote: >> >> In JDK-8211073, an attempt was made to add -Wextra to hotspot. That patch was not accepted at the time being, but it did dig out some bugs in the code. >> >> While we await JDK-8211073 to be included in the code base (so -Wextra is always on), we can at least fix the bugs it found in some copy constructors. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213414 >> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213414-fix-hotspot-copy-constructors/webrev.01 >> >> /Magnus > > Looks good. ? and trivial. From rkennke at redhat.com Tue Nov 6 21:37:14 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 6 Nov 2018 22:37:14 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> Message-ID: <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> Hi Erik, I also like it. I needed to change a little bit to make it work with Shenandoah: http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch I will give it a spin on aarch64 as soon as I get access to the machine. Do you want to take the bug and finish it, or do you want me to take it from here? Roman > Hi Andrew, > > On 2018-11-06 17:46, Andrew Dinn wrote: >> On 06/11/18 16:04, Erik ?sterlund wrote: >>> New webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >> >> Thanks for the updated version. >> >>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>> In that case, I hear you ask, why does the old code check for values in >>>> the range [heap_base - cell_header_size, heap_base + os::vm_page_size)? >>>> Good question. I don't know the answer. I think that test is bogus with >>>> no Shenandoah GC present. >>> >>> Interesting. It turns out that both ranges are relevant >>> [-cell_header_size, os::vm_page_size) and [heap_base - cell_header_size, >>> heap_base + os::vm_page_size). Looks like we both missed that from the >>> original check. >>> I updated my webrev to explicitly reflect those two ranges, as two >>> different checks. >> >> Well, you say that but -- as I said before -- I am not sure why *both* >> are relevant to the non-Shenandoah case. And, given Roman's correction >> (doh, yes of course! the space for the Brooks pointer field had better >> lie in RW-mapped memory :-) I don't see why it is relevant to the >> Shenandoah case either. > > The reason it is needed is because there may or may not be null checks > in both the decode() step to get the base pointer of an access, and the > access itself, given some base pointer. This results in two ranges for > those two cases correspondingly. If the decoding uses decode_not_null, > with an implicit null check in the decoding itself that blindly adds the > heap base without checking for null, we will crash into the first page > of the heap which is protected when performing accesses into that. But > sometimes we have a decode()ed base pointer (with null check), that has > already been fixed up to NULL, and then perform an access at an offset > into that with an implicit null check. This results in a trap at the > first page. So in summary, both ranges are needed because the > implicitness of the null check may be both in the access and the > narrowOop decoding. > > But as I pointed out, this (seemingly incorrectly) assumes there may not > be any offset values in the (heap_base, heap_base + os::vm_page_size) > range. If you hit such an offset, we will generate incorrect code, due > to the lack of distinction between addresses and offsets. > >> I'd be very glad to hear from someone who knows about the history of the >> original function as to why the extra test for addresses above heap_base >> is included [hint: John Rose? David Holmes? Coleen? anyone else who >> might have been around when it was added?] >> >> If that check is not actually needed then getting rid of it would make >> the whole thing a lot simpler. > > It would indeed. But unfortunately we need it. > >>>>> Just expressing these trivial ranges separately would have been very >>>>> easy to understand. >>>> Except it's not that easy. Also, with Shenandoah there is a new >>>> wrinkle. >>>> It is possible for an object to be allocated at the first non-guarded >>>> address in the heap area i.e. heap_base + os::vm_page_size. I that case >>>> a dereference of its Brooks pointer will lie in the protected page. >>> >>> Looks like Roman just said that's not possible. >> >> Yes, my dumb! >> >>> Fixed. Might need some help testing this on AArch64 though if we decide >>> to go with it. >> >> I am sure Roman and Aleksey can do that but if needed I will. >> >>> Do you agree though, that the current version that merges the address >>> and offset case relies on offsets being smaller than the heap base for >>> its magic check to work, and that we actually do not have such a >>> guarantee in the VM? Apart from readability preferences, this seems like >>> it is broken in the current form, unless there are seat belts I am not >>> aware of. In particular, if you have an offset value that is in the >>> virtual address range (heap_base, heap_base + os::vm_page_size()), which >>> I argue can happen, then it looks like we will normalize that as if it >>> was an address (even though it is not), and incorrectly say that this >>> offset does not need explicit null checks, despite needing explicit null >>> checks. >> Well, without the answer as to whether the check against heap_base is >> needed I am not sure. Assuming it is then then ... >> >> Yes, it is possible, in theory, that an oop offset could turn out to >> have value (heap_base + D) where 0 <= D < os_page_size(). In that case >> the offset might wrongly be classified as *not* needing a null check >> even though it really does. I suspect it is unlikely in practice given >> how mmap hands out addresses but you never know. >> >> As for readibility, I think the two functions are probably clearer -- >> expecially if they stay adjacent in the source code. > > Okay, sounds like we are on the same page. I agree it is unlikely that > this will crash the VM (as evidenced by the lack of crashes observed). > But I don't like relying on luck for correctness. > > Thanks, > /Erik > >> regards, >> >> >> Andrew Dinn >> ----------- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in England and Wales under Company Registration No. 03798903 >> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >> From david.holmes at oracle.com Tue Nov 6 22:36:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Nov 2018 08:36:02 +1000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> Message-ID: <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> Hi Boris, I'm somewhat perplexed as to what the difference is between: @requires vm.opt.final.UseCompressedOops and @requires vm.opt.final.UseCompressedOops == true if they are indeed different then that seems to be a bug in the requirement handling in the jtreg support code. I'm also wondering whether any such requirement should always be proceeded by a check for 64-bits? Otherwise I would expect @requires vm.opt.final.UseCompressedOops to be false in 32-bit, so again a problem with the jtreg support code. Thanks, David On 7/11/2018 12:43 AM, Boris Ulasevich wrote: > Hi all, > > Please review this patch to fix jtreg @requires > vm.opt.final.UseCompressedOops flag evaluation fail reproduced on ARM32: > "invalid boolean value: null for expression > vm.opt.final.UseCompressedOops". > > http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8213410 > > The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok by > default, and becomes disabled with -javaoptions:"-XX:-UseCompressedOops" > jtreg option). > > Thanks, > Boris Ulasevich From robbin.ehn at oracle.com Wed Nov 7 07:11:18 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 7 Nov 2018 08:11:18 +0100 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> Message-ID: <6ccc82c9-e468-049c-74b4-daddafb86dac@oracle.com> Thanks David, looks good! /Robbin On 2018-11-05 02:43, David Holmes wrote: > This impacts GC, compiler, runtime and serviceability. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 > webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ > > To facilitate changes to the Mutex/Monitor code we need to ensure there are no > statically initialized Monitor/Mutex instances in the VM - as the constructors > may rely on VM initialization that has not happened when C++ initializers execute. > > There were 6 locally defined "lock" members of classes that were statically > initialized, and these are across all areas of the VM: > > ?- Decoder::_shared_decoder_lock > ?- DCmdFactory::_dcmdFactory_lock > ?- JfrThreadSampler::_transition_block_lock > ?- NMethodSweeper::_stat_lock > ?- ThreadsSMRSupport::_delete_lock > ?- CodeCacheUnloadingTask::_lock > > The last is actually now unused and so is just deleted. The rest join all other > Monitor/Mutex instances defined in the global list in MutexLocker and > initialized in mutex_init(). I've verified that none of the monitor/mutex > instances can be used prior to mutex_init, by code inspection - more details in > the bug report. > > I investigated asserting that a Monitor/Mutex is only constructed during > mutex_init() but assertions don't fail cleanly during C++ static initialization > - the result is that during the build jmod goes into an infinite loop printing > out "[Too many errors, abort]". > > Tested using tier1-3 > > Thanks, > David From david.holmes at oracle.com Wed Nov 7 07:12:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Nov 2018 17:12:15 +1000 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: <6ccc82c9-e468-049c-74b4-daddafb86dac@oracle.com> References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> <6ccc82c9-e468-049c-74b4-daddafb86dac@oracle.com> Message-ID: Thanks Robbin! David On 7/11/2018 5:11 PM, Robbin Ehn wrote: > Thanks David, looks good! > > /Robbin > > On 2018-11-05 02:43, David Holmes wrote: >> This impacts GC, compiler, runtime and serviceability. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 >> webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ >> >> To facilitate changes to the Mutex/Monitor code we need to ensure >> there are no statically initialized Monitor/Mutex instances in the VM >> - as the constructors may rely on VM initialization that has not >> happened when C++ initializers execute. >> >> There were 6 locally defined "lock" members of classes that were >> statically initialized, and these are across all areas of the VM: >> >> ??- Decoder::_shared_decoder_lock >> ??- DCmdFactory::_dcmdFactory_lock >> ??- JfrThreadSampler::_transition_block_lock >> ??- NMethodSweeper::_stat_lock >> ??- ThreadsSMRSupport::_delete_lock >> ??- CodeCacheUnloadingTask::_lock >> >> The last is actually now unused and so is just deleted. The rest join >> all other Monitor/Mutex instances defined in the global list in >> MutexLocker and initialized in mutex_init(). I've verified that none >> of the monitor/mutex instances can be used prior to mutex_init, by >> code inspection - more details in the bug report. >> >> I investigated asserting that a Monitor/Mutex is only constructed >> during mutex_init() but assertions don't fail cleanly during C++ >> static initialization - the result is that during the build jmod goes >> into an infinite loop printing out "[Too many errors, abort]". >> >> Tested using tier1-3 >> >> Thanks, >> David From erik.osterlund at oracle.com Wed Nov 7 07:35:25 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 08:35:25 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> Message-ID: <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> Hi Roman, Glad you liked the idea! On 2018-11-06 22:37, Roman Kennke wrote: > Hi Erik, > > I also like it. I needed to change a little bit to make it work with > Shenandoah: > > http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch > > I will give it a spin on aarch64 as soon as I get access to the machine. > > Do you want to take the bug and finish it, or do you want me to take it > from here? You can take it from here. I just wanted to show a possible direction. Thanks, /Erik > Roman > >> Hi Andrew, >> >> On 2018-11-06 17:46, Andrew Dinn wrote: >>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>> New webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>> >>> Thanks for the updated version. >>> >>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>> In that case, I hear you ask, why does the old code check for values in >>>>> the range [heap_base - cell_header_size, heap_base + os::vm_page_size)? >>>>> Good question. I don't know the answer. I think that test is bogus with >>>>> no Shenandoah GC present. >>>> >>>> Interesting. It turns out that both ranges are relevant >>>> [-cell_header_size, os::vm_page_size) and [heap_base - cell_header_size, >>>> heap_base + os::vm_page_size). Looks like we both missed that from the >>>> original check. >>>> I updated my webrev to explicitly reflect those two ranges, as two >>>> different checks. >>> >>> Well, you say that but -- as I said before -- I am not sure why *both* >>> are relevant to the non-Shenandoah case. And, given Roman's correction >>> (doh, yes of course! the space for the Brooks pointer field had better >>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>> Shenandoah case either. >> >> The reason it is needed is because there may or may not be null checks >> in both the decode() step to get the base pointer of an access, and the >> access itself, given some base pointer. This results in two ranges for >> those two cases correspondingly. If the decoding uses decode_not_null, >> with an implicit null check in the decoding itself that blindly adds the >> heap base without checking for null, we will crash into the first page >> of the heap which is protected when performing accesses into that. But >> sometimes we have a decode()ed base pointer (with null check), that has >> already been fixed up to NULL, and then perform an access at an offset >> into that with an implicit null check. This results in a trap at the >> first page. So in summary, both ranges are needed because the >> implicitness of the null check may be both in the access and the >> narrowOop decoding. >> >> But as I pointed out, this (seemingly incorrectly) assumes there may not >> be any offset values in the (heap_base, heap_base + os::vm_page_size) >> range. If you hit such an offset, we will generate incorrect code, due >> to the lack of distinction between addresses and offsets. >> >>> I'd be very glad to hear from someone who knows about the history of the >>> original function as to why the extra test for addresses above heap_base >>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>> might have been around when it was added?] >>> >>> If that check is not actually needed then getting rid of it would make >>> the whole thing a lot simpler. >> >> It would indeed. But unfortunately we need it. >> >>>>>> Just expressing these trivial ranges separately would have been very >>>>>> easy to understand. >>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>> wrinkle. >>>>> It is possible for an object to be allocated at the first non-guarded >>>>> address in the heap area i.e. heap_base + os::vm_page_size. I that case >>>>> a dereference of its Brooks pointer will lie in the protected page. >>>> >>>> Looks like Roman just said that's not possible. >>> >>> Yes, my dumb! >>> >>>> Fixed. Might need some help testing this on AArch64 though if we decide >>>> to go with it. >>> >>> I am sure Roman and Aleksey can do that but if needed I will. >>> >>>> Do you agree though, that the current version that merges the address >>>> and offset case relies on offsets being smaller than the heap base for >>>> its magic check to work, and that we actually do not have such a >>>> guarantee in the VM? Apart from readability preferences, this seems like >>>> it is broken in the current form, unless there are seat belts I am not >>>> aware of. In particular, if you have an offset value that is in the >>>> virtual address range (heap_base, heap_base + os::vm_page_size()), which >>>> I argue can happen, then it looks like we will normalize that as if it >>>> was an address (even though it is not), and incorrectly say that this >>>> offset does not need explicit null checks, despite needing explicit null >>>> checks. >>> Well, without the answer as to whether the check against heap_base is >>> needed I am not sure. Assuming it is then then ... >>> >>> Yes, it is possible, in theory, that an oop offset could turn out to >>> have value (heap_base + D) where 0 <= D < os_page_size(). In that case >>> the offset might wrongly be classified as *not* needing a null check >>> even though it really does. I suspect it is unlikely in practice given >>> how mmap hands out addresses but you never know. >>> >>> As for readibility, I think the two functions are probably clearer -- >>> expecially if they stay adjacent in the source code. >> >> Okay, sounds like we are on the same page. I agree it is unlikely that >> this will crash the VM (as evidenced by the lack of crashes observed). >> But I don't like relying on luck for correctness. >> >> Thanks, >> /Erik >> >>> regards, >>> >>> >>> Andrew Dinn >>> ----------- >>> Senior Principal Software Engineer >>> Red Hat UK Ltd >>> Registered in England and Wales under Company Registration No. 03798903 >>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >>> > From erik.osterlund at oracle.com Wed Nov 7 08:00:04 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 09:00:04 +0100 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> Message-ID: Hi David, I noticed that for each lock except Decoder::shared_decoder_lock() you removed the static lock and its static getter, rerouting references from them to the new locks managed in the mutexLocker files. Was there a particular reason why the getter for shared_decoder_lock() in particular remains, instead of just referencing the Decoder_shared_decoder_lock directly, as you did for all other locks? I suppose that by removing the getter, we would miss out on the assert that the lock is not NULL. But on the other hand, 1) nobody else checks if their global lock from mutexLocker is null, 2) if it ever was null, we would immediately get a SIGSEGV upon use, so I don't think that is particularly helpful. I also noticed that all other locks managed by the mutexLocker files have a CamelCase_lock convention. These locks are the first to break that convention. On the other hand, I have no idea why we have that convention in the first place, so I'm not necessarily opposed to that. I thought I'd at least check if this was intended or accidental. Thanks, /Erik On 2018-11-05 02:43, David Holmes wrote: > This impacts GC, compiler, runtime and serviceability. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 > webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ > > To facilitate changes to the Mutex/Monitor code we need to ensure there > are no statically initialized Monitor/Mutex instances in the VM - as the > constructors may rely on VM initialization that has not happened when > C++ initializers execute. > > There were 6 locally defined "lock" members of classes that were > statically initialized, and these are across all areas of the VM: > > ?- Decoder::_shared_decoder_lock > ?- DCmdFactory::_dcmdFactory_lock > ?- JfrThreadSampler::_transition_block_lock > ?- NMethodSweeper::_stat_lock > ?- ThreadsSMRSupport::_delete_lock > ?- CodeCacheUnloadingTask::_lock > > The last is actually now unused and so is just deleted. The rest join > all other Monitor/Mutex instances defined in the global list in > MutexLocker and initialized in mutex_init(). I've verified that none of > the monitor/mutex instances can be used prior to mutex_init, by code > inspection - more details in the bug report. > > I investigated asserting that a Monitor/Mutex is only constructed during > mutex_init() but assertions don't fail cleanly during C++ static > initialization - the result is that during the build jmod goes into an > infinite loop printing out "[Too many errors, abort]". > > Tested using tier1-3 > > Thanks, > David From boris.ulasevich at bell-sw.com Wed Nov 7 08:06:43 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Wed, 7 Nov 2018 11:06:43 +0300 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> Message-ID: <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Hi David, Yes, at first glance it is weird. We have actually three states for vm.opt.final.UseCompressedOops: true, false and null. Null means "not applicable" - when current VM does not support the option with the given name. Here is another approach for the issue (I am not sure it is a good one): http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ thank you, Boris On 07.11.2018 1:36, David Holmes wrote: > Hi Boris, > > I'm somewhat perplexed as to what the difference is between: > > @requires vm.opt.final.UseCompressedOops > > and > > @requires vm.opt.final.UseCompressedOops == true > > if they are indeed different then that seems to be a bug in the > requirement handling in the jtreg support code. > > I'm also wondering whether any such requirement should always be > proceeded by a check for 64-bits? > > Otherwise I would expect > > @requires vm.opt.final.UseCompressedOops > > to be false in 32-bit, so again a problem with the jtreg support code. > > Thanks, > David > > On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >> Hi all, >> >> Please review this patch to fix jtreg @requires >> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >> ARM32: "invalid boolean value: null for expression >> vm.opt.final.UseCompressedOops". >> >> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8213410 >> >> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok by >> default, and becomes disabled with >> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >> >> Thanks, >> Boris Ulasevich From robbin.ehn at oracle.com Wed Nov 7 08:32:36 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 7 Nov 2018 09:32:36 +0100 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) Message-ID: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> Hi all, please review. We rename SafepointMechanism::poll to SafepointMechanism::should_block to make it more clear what the method do. CR: https://bugs.openjdk.java.net/browse/JDK-8211403 Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ Sanity on tier 1. Thanks, Robbin From martin.doerr at sap.com Wed Nov 7 08:37:13 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 7 Nov 2018 08:37:13 +0000 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> Message-ID: <33c185af14564c4a9cd1d6b9e668f73b@sap.com> +1 -----Original Message----- From: hotspot-dev On Behalf Of Robbin Ehn Sent: Mittwoch, 7. November 2018 09:33 To: hotspot-dev developers Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) Hi all, please review. We rename SafepointMechanism::poll to SafepointMechanism::should_block to make it more clear what the method do. CR: https://bugs.openjdk.java.net/browse/JDK-8211403 Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ Sanity on tier 1. Thanks, Robbin From david.holmes at oracle.com Wed Nov 7 08:45:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Nov 2018 18:45:17 +1000 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> Message-ID: <71f9e512-46aa-d4c4-de44-c2827c5b14f1@oracle.com> Hi Erik, Thanks for looking at this ;-) On 7/11/2018 6:00 PM, Erik ?sterlund wrote: > Hi David, > > I noticed that for each lock except Decoder::shared_decoder_lock() you > removed the static lock and its static getter, rerouting references from > them to the new locks managed in the mutexLocker files. > > Was there a particular reason why the getter for shared_decoder_lock() > in particular remains, instead of just referencing the > Decoder_shared_decoder_lock directly, as you did for all other locks? > > I suppose that by removing the getter, we would miss out on the assert > that the lock is not NULL. But on the other hand, 1) nobody else checks > if their global lock from mutexLocker is null, 2) if it ever was null, > we would immediately get a SIGSEGV upon use, so I don't think that is > particularly helpful. Yes it is because of the assert that checks for NULL. My intent was to move the locks but leave the existing semantics unchanged. So I kept the getter for the Decoder_lock and moved the assertion there to simplify the other code that repeated the NULL checks. If you really don't like this (and I agree this code is the odd one out here) then I suggest it gets fixed in the follow up RFE that I filed: (JDK-8213399) DecoderLocker is unused > I also noticed that all other locks managed by the mutexLocker files > have a CamelCase_lock convention. These locks are the first to break > that convention. On the other hand, I have no idea why we have that > convention in the first place, so I'm not necessarily opposed to that. I > thought I'd at least check if this was intended or accidental. Accidental. I thought the convention was (loosely) ClassName_lock_name but it's more ThingBeingLocked_lock, so I have Decoder_shared_decoder_lock rather than (I suppose?) SharedDecoder_lock I can change these: NMethodSweeper_stat_lock -> NMethodSweeperStats_lock; ThreadsSMRSupport_delete_lock -> ThreadsSMRDelete_lock Decoder_shared_decoder_lock -> SharedDecoder_lock Okay? Thanks, David > Thanks, > /Erik > > On 2018-11-05 02:43, David Holmes wrote: >> This impacts GC, compiler, runtime and serviceability. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 >> webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ >> >> To facilitate changes to the Mutex/Monitor code we need to ensure >> there are no statically initialized Monitor/Mutex instances in the VM >> - as the constructors may rely on VM initialization that has not >> happened when C++ initializers execute. >> >> There were 6 locally defined "lock" members of classes that were >> statically initialized, and these are across all areas of the VM: >> >> ??- Decoder::_shared_decoder_lock >> ??- DCmdFactory::_dcmdFactory_lock >> ??- JfrThreadSampler::_transition_block_lock >> ??- NMethodSweeper::_stat_lock >> ??- ThreadsSMRSupport::_delete_lock >> ??- CodeCacheUnloadingTask::_lock >> >> The last is actually now unused and so is just deleted. The rest join >> all other Monitor/Mutex instances defined in the global list in >> MutexLocker and initialized in mutex_init(). I've verified that none >> of the monitor/mutex instances can be used prior to mutex_init, by >> code inspection - more details in the bug report. >> >> I investigated asserting that a Monitor/Mutex is only constructed >> during mutex_init() but assertions don't fail cleanly during C++ >> static initialization - the result is that during the build jmod goes >> into an infinite loop printing out "[Too many errors, abort]". >> >> Tested using tier1-3 >> >> Thanks, >> David From rkennke at redhat.com Wed Nov 7 08:52:22 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 7 Nov 2018 09:52:22 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> Message-ID: Alright! thanks Erik! So here I'd like to propose your version plus my little fixes for review: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.02/ It passes hotspot/jtreg:tier1 locally on x86 and aarch64 and works with Shenandoah too (and passes the Shenandoah testsuites) Good? Roman > Hi Roman, > > Glad you liked the idea! > > On 2018-11-06 22:37, Roman Kennke wrote: >> Hi Erik, >> >> I also like it. I needed to change a little bit to make it work with >> Shenandoah: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch >> >> >> I will give it a spin on aarch64 as soon as I get access to the machine. >> >> Do you want to take the bug and finish it, or do you want me to take it >> from here? > > You can take it from here. I just wanted to show a possible direction. > > Thanks, > /Erik > >> Roman >> >>> Hi Andrew, >>> >>> On 2018-11-06 17:46, Andrew Dinn wrote: >>>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>>> New webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>>> >>>> Thanks for the updated version. >>>> >>>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>>> In that case, I hear you ask, why does the old code check for >>>>>> values in >>>>>> the range [heap_base - cell_header_size, heap_base + >>>>>> os::vm_page_size)? >>>>>> Good question. I don't know the answer. I think that test is bogus >>>>>> with >>>>>> no Shenandoah GC present. >>>>> >>>>> Interesting. It turns out that both ranges are relevant >>>>> [-cell_header_size, os::vm_page_size) and [heap_base - >>>>> cell_header_size, >>>>> heap_base + os::vm_page_size). Looks like we both missed that from the >>>>> original check. >>>>> I updated my webrev to explicitly reflect those two ranges, as two >>>>> different checks. >>>> >>>> Well, you say that but -- as I said before -- I am not sure why *both* >>>> are relevant to the non-Shenandoah case. And, given Roman's correction >>>> (doh, yes of course! the space for the Brooks pointer field had better >>>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>>> Shenandoah case either. >>> >>> The reason it is needed is because there may or may not be null checks >>> in both the decode() step to get the base pointer of an access, and the >>> access itself, given some base pointer. This results in two ranges for >>> those two cases correspondingly. If the decoding uses decode_not_null, >>> with an implicit null check in the decoding itself that blindly adds the >>> heap base without checking for null, we will crash into the first page >>> of the heap which is protected when performing accesses into that. But >>> sometimes we have a decode()ed base pointer (with null check), that has >>> already been fixed up to NULL, and then perform an access at an offset >>> into that with an implicit null check. This results in a trap at the >>> first page. So in summary, both ranges are needed because the >>> implicitness of the null check may be both in the access and the >>> narrowOop decoding. >>> >>> But as I pointed out, this (seemingly incorrectly) assumes there may not >>> be any offset values in the (heap_base, heap_base + os::vm_page_size) >>> range. If you hit such an offset, we will generate incorrect code, due >>> to the lack of distinction between addresses and offsets. >>> >>>> I'd be very glad to hear from someone who knows about the history of >>>> the >>>> original function as to why the extra test for addresses above >>>> heap_base >>>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>>> might have been around when it was added?] >>>> >>>> If that check is not actually needed then getting rid of it would make >>>> the whole thing a lot simpler. >>> >>> It would indeed. But unfortunately we need it. >>> >>>>>>> Just expressing these trivial ranges separately would have been very >>>>>>> easy to understand. >>>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>>> wrinkle. >>>>>> It is possible for an object to be allocated at the first non-guarded >>>>>> address in the heap area i.e. heap_base + os::vm_page_size. I that >>>>>> case >>>>>> a dereference of its Brooks pointer will lie in the protected page. >>>>> >>>>> Looks like Roman just said that's not possible. >>>> >>>> Yes, my dumb! >>>> >>>>> Fixed. Might need some help testing this on AArch64 though if we >>>>> decide >>>>> to go with it. >>>> >>>> I am sure Roman and Aleksey can do that but if needed I will. >>>> >>>>> Do you agree though, that the current version that merges the address >>>>> and offset case relies on offsets being smaller than the heap base for >>>>> its magic check to work, and that we actually do not have such a >>>>> guarantee in the VM? Apart from readability preferences, this seems >>>>> like >>>>> it is broken in the current form, unless there are seat belts I am not >>>>> aware of. In particular, if you have an offset value that is in the >>>>> virtual address range (heap_base, heap_base + os::vm_page_size()), >>>>> which >>>>> I argue can happen, then it looks like we will normalize that as if it >>>>> was an address (even though it is not), and incorrectly say that this >>>>> offset does not need explicit null checks, despite needing explicit >>>>> null >>>>> checks. >>>> Well, without the answer as to whether the check against heap_base is >>>> needed I am not sure. Assuming it is then then ... >>>> >>>> Yes, it is possible, in theory, that an oop offset could turn out to >>>> have value (heap_base + D) where 0 <= D < os_page_size(). In that case >>>> the offset might wrongly be classified as *not* needing a null check >>>> even though it really does. I suspect it is unlikely in practice given >>>> how mmap hands out addresses but you never know. >>>> >>>> As for readibility, I think the two functions are probably clearer -- >>>> expecially if they stay adjacent in the source code. >>> >>> Okay, sounds like we are on the same page. I agree it is unlikely that >>> this will crash the VM (as evidenced by the lack of crashes observed). >>> But I don't like relying on luck for correctness. >>> >>> Thanks, >>> /Erik >>> >>>> regards, >>>> >>>> >>>> Andrew Dinn >>>> ----------- >>>> Senior Principal Software Engineer >>>> Red Hat UK Ltd >>>> Registered in England and Wales under Company Registration No. 03798903 >>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >>>> >> From david.holmes at oracle.com Wed Nov 7 08:54:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Nov 2018 18:54:39 +1000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Message-ID: Hi Boris, On 7/11/2018 6:06 PM, Boris Ulasevich wrote: > Hi David, > > ? Yes, at first glance it is weird. We have actually three states for > vm.opt.final.UseCompressedOops: true, false and null. Null means "not > applicable" - when current VM does not support the option with the given > name. Here is another approach for the issue (I am not sure it is a good > one): > ? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ I like that approach! I'm just not sure everyone else will necessarily agree :) This proposal might need a wider audience than just hotspot-dev. Let me make enquiries with jtreg-use at openjdk.java.net and see what the general intent was here. Aside: if the UseCompressOops checks is preceded by a 64-bit check does it still fail? (Just wondering whether jtreg does short-circuit evaluation for these @require statements.) Thanks, David > thank you, > Boris > > On 07.11.2018 1:36, David Holmes wrote: >> Hi Boris, >> >> I'm somewhat perplexed as to what the difference is between: >> >> @requires vm.opt.final.UseCompressedOops >> >> and >> >> @requires vm.opt.final.UseCompressedOops == true >> >> if they are indeed different then that seems to be a bug in the >> requirement handling in the jtreg support code. >> >> I'm also wondering whether any such requirement should always be >> proceeded by a check for 64-bits? >> >> Otherwise I would expect >> >> @requires vm.opt.final.UseCompressedOops >> >> to be false in 32-bit, so again a problem with the jtreg support code. >> >> Thanks, >> David >> >> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>> Hi all, >>> >>> Please review this patch to fix jtreg @requires >>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>> ARM32: "invalid boolean value: null for expression >>> vm.opt.final.UseCompressedOops". >>> >>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>> >>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok >>> by default, and becomes disabled with >>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>> >>> Thanks, >>> Boris Ulasevich From robbin.ehn at oracle.com Wed Nov 7 08:57:44 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 7 Nov 2018 09:57:44 +0100 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: <33c185af14564c4a9cd1d6b9e668f73b@sap.com> References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> <33c185af14564c4a9cd1d6b9e668f73b@sap.com> Message-ID: <68de646a-d9c5-89b9-b512-049a4936935f@oracle.com> Thanks Martin! /Robbin On 2018-11-07 09:37, Doerr, Martin wrote: > +1 > > -----Original Message----- > From: hotspot-dev On Behalf Of Robbin Ehn > Sent: Mittwoch, 7. November 2018 09:33 > To: hotspot-dev developers > Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) > > Hi all, please review. > > We rename SafepointMechanism::poll to SafepointMechanism::should_block to make > it more clear what the method do. > > CR: https://bugs.openjdk.java.net/browse/JDK-8211403 > Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ > > Sanity on tier 1. > > Thanks, Robbin > From boris.ulasevich at bell-sw.com Wed Nov 7 09:07:24 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Wed, 7 Nov 2018 12:07:24 +0300 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Message-ID: <24aeecab-7c3a-9902-cbb5-68c6023922ae@bell-sw.com> Hi David, > Let me make enquiries with jtreg-use at openjdk.java.net Ok! > Aside: if the UseCompressOops checks is preceded by a 64-bit check > does it still fail? Jtreg test fails with "invalid boolean value" message: @requires vm.bits == "64" & vm.opt.final.UseCompressedOops Boris On 07.11.2018 11:54, David Holmes wrote: > Hi Boris, > > On 7/11/2018 6:06 PM, Boris Ulasevich wrote: >> Hi David, >> >> ?? Yes, at first glance it is weird. We have actually three states for >> vm.opt.final.UseCompressedOops: true, false and null. Null means "not >> applicable" - when current VM does not support the option with the >> given name. Here is another approach for the issue (I am not sure it >> is a good one): >> ?? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ > > I like that approach! I'm just not sure everyone else will necessarily > agree :) This proposal might need a wider audience than just hotspot-dev. > > Let me make enquiries with jtreg-use at openjdk.java.net and see what the > general intent was here. > > Aside: if the UseCompressOops checks is preceded by a 64-bit check does > it still fail? (Just wondering whether jtreg does short-circuit > evaluation for these @require statements.) > > Thanks, > David > >> thank you, >> Boris >> >> On 07.11.2018 1:36, David Holmes wrote: >>> Hi Boris, >>> >>> I'm somewhat perplexed as to what the difference is between: >>> >>> @requires vm.opt.final.UseCompressedOops >>> >>> and >>> >>> @requires vm.opt.final.UseCompressedOops == true >>> >>> if they are indeed different then that seems to be a bug in the >>> requirement handling in the jtreg support code. >>> >>> I'm also wondering whether any such requirement should always be >>> proceeded by a check for 64-bits? >>> >>> Otherwise I would expect >>> >>> @requires vm.opt.final.UseCompressedOops >>> >>> to be false in 32-bit, so again a problem with the jtreg support code. >>> >>> Thanks, >>> David >>> >>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>> Hi all, >>>> >>>> Please review this patch to fix jtreg @requires >>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>>> ARM32: "invalid boolean value: null for expression >>>> vm.opt.final.UseCompressedOops". >>>> >>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>> >>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok >>>> by default, and becomes disabled with >>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>> >>>> Thanks, >>>> Boris Ulasevich From david.holmes at oracle.com Wed Nov 7 09:32:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Nov 2018 19:32:08 +1000 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> Message-ID: Hi Robbin, On 7/11/2018 6:32 PM, Robbin Ehn wrote: > Hi all, please review. > > We rename SafepointMechanism::poll to SafepointMechanism::should_block > to make it more clear what the method do. Fair enough, but doesn't that just imply that global_poll and local_poll are also mis-named? How far should this rename go? Thanks, David > > CR: https://bugs.openjdk.java.net/browse/JDK-8211403 > Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ > > Sanity on tier 1. > > Thanks, Robbin From erik.osterlund at oracle.com Wed Nov 7 09:59:07 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 10:59:07 +0100 Subject: RFR (S) 8213137: Remove static initialization of monitor/mutex instances In-Reply-To: <71f9e512-46aa-d4c4-de44-c2827c5b14f1@oracle.com> References: <05b33a95-d7e0-1492-6ac4-014a485be165@oracle.com> <71f9e512-46aa-d4c4-de44-c2827c5b14f1@oracle.com> Message-ID: <5BE2B76B.6040608@oracle.com> Hi David, On 2018-11-07 09:45, David Holmes wrote: > Hi Erik, > > Thanks for looking at this ;-) > > On 7/11/2018 6:00 PM, Erik ?sterlund wrote: >> Hi David, >> >> I noticed that for each lock except Decoder::shared_decoder_lock() >> you removed the static lock and its static getter, rerouting >> references from them to the new locks managed in the mutexLocker files. >> >> Was there a particular reason why the getter for >> shared_decoder_lock() in particular remains, instead of just >> referencing the Decoder_shared_decoder_lock directly, as you did for >> all other locks? >> >> I suppose that by removing the getter, we would miss out on the >> assert that the lock is not NULL. But on the other hand, 1) nobody >> else checks if their global lock from mutexLocker is null, 2) if it >> ever was null, we would immediately get a SIGSEGV upon use, so I >> don't think that is particularly helpful. > > Yes it is because of the assert that checks for NULL. My intent was to > move the locks but leave the existing semantics unchanged. So I kept > the getter for the Decoder_lock and moved the assertion there to > simplify the other code that repeated the NULL checks. If you really > don't like this (and I agree this code is the odd one out here) then I > suggest it gets fixed in the follow up RFE that I filed: (JDK-8213399) > DecoderLocker is unused That sounds like a good plan. >> I also noticed that all other locks managed by the mutexLocker files >> have a CamelCase_lock convention. These locks are the first to break >> that convention. On the other hand, I have no idea why we have that >> convention in the first place, so I'm not necessarily opposed to >> that. I thought I'd at least check if this was intended or accidental. > > Accidental. I thought the convention was (loosely) ClassName_lock_name > but it's more ThingBeingLocked_lock, so I have > > Decoder_shared_decoder_lock > > rather than (I suppose?) > > SharedDecoder_lock > > I can change these: > > NMethodSweeper_stat_lock -> NMethodSweeperStats_lock; > ThreadsSMRSupport_delete_lock -> ThreadsSMRDelete_lock > Decoder_shared_decoder_lock -> SharedDecoder_lock > > Okay? Those names look good to me. I don't need to see another webrev. Thanks, /Erik > Thanks, > David > >> Thanks, >> /Erik >> >> On 2018-11-05 02:43, David Holmes wrote: >>> This impacts GC, compiler, runtime and serviceability. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213137 >>> webrev: http://cr.openjdk.java.net/~dholmes/8213137/webrev/ >>> >>> To facilitate changes to the Mutex/Monitor code we need to ensure >>> there are no statically initialized Monitor/Mutex instances in the >>> VM - as the constructors may rely on VM initialization that has not >>> happened when C++ initializers execute. >>> >>> There were 6 locally defined "lock" members of classes that were >>> statically initialized, and these are across all areas of the VM: >>> >>> - Decoder::_shared_decoder_lock >>> - DCmdFactory::_dcmdFactory_lock >>> - JfrThreadSampler::_transition_block_lock >>> - NMethodSweeper::_stat_lock >>> - ThreadsSMRSupport::_delete_lock >>> - CodeCacheUnloadingTask::_lock >>> >>> The last is actually now unused and so is just deleted. The rest >>> join all other Monitor/Mutex instances defined in the global list in >>> MutexLocker and initialized in mutex_init(). I've verified that none >>> of the monitor/mutex instances can be used prior to mutex_init, by >>> code inspection - more details in the bug report. >>> >>> I investigated asserting that a Monitor/Mutex is only constructed >>> during mutex_init() but assertions don't fail cleanly during C++ >>> static initialization - the result is that during the build jmod >>> goes into an infinite loop printing out "[Too many errors, abort]". >>> >>> Tested using tier1-3 >>> >>> Thanks, >>> David From robbin.ehn at oracle.com Wed Nov 7 10:04:29 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 7 Nov 2018 11:04:29 +0100 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> Message-ID: <2663554a-9600-1a3f-e2f4-0576e4688912@oracle.com> Hi David, On 2018-11-07 10:32, David Holmes wrote: > Hi Robbin, > > On 7/11/2018 6:32 PM, Robbin Ehn wrote: >> Hi all, please review. >> >> We rename SafepointMechanism::poll to SafepointMechanism::should_block to make >> it more clear what the method do. > > Fair enough, but doesn't that just imply that global_poll and local_poll are > also mis-named? How far should this rename go? I think they are fine, because of the different context. As I see it should_block is implemented by actively checking (polling) the threads page or the global state. Thanks, Robbin > > Thanks, > David > >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8211403 >> Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ >> >> Sanity on tier 1. >> >> Thanks, Robbin From adinn at redhat.com Wed Nov 7 10:31:37 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 7 Nov 2018 10:31:37 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <90d4e32b-fd85-8cb1-a8e9-6bb49f211f0c@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <90d4e32b-fd85-8cb1-a8e9-6bb49f211f0c@redhat.com> Message-ID: On 06/11/18 17:45, Andrew Dinn wrote: > Could you point me at some place in the compiler where narrow oop loads > are done using a decode_not_null on the assumption that a load of a null > value that gets translated to a low heap_base address will be caught and > corrected by a check initiated from the signal handler? I'll answer that myself: The magic happens in opto/compile.cpp and the controlling predicate to search for is gen_narrow_oop_implicit_null_checks which itself relies on Matcher::narrow_oop_use_complex_address defined in opto/matcher.hpp. n.b. the ad file for a given architecture can control this feature by how it defines method narrow_oop_use_complex_address. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From erik.osterlund at oracle.com Wed Nov 7 10:37:00 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 11:37:00 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <90d4e32b-fd85-8cb1-a8e9-6bb49f211f0c@redhat.com> Message-ID: <5BE2C04C.50507@oracle.com> Hi Andrew, Thanks for digging that up. /Erik On 2018-11-07 11:31, Andrew Dinn wrote: > On 06/11/18 17:45, Andrew Dinn wrote: >> Could you point me at some place in the compiler where narrow oop loads >> are done using a decode_not_null on the assumption that a load of a null >> value that gets translated to a low heap_base address will be caught and >> corrected by a check initiated from the signal handler? > I'll answer that myself: > > The magic happens in opto/compile.cpp and the controlling predicate to > search for is gen_narrow_oop_implicit_null_checks which itself relies on > Matcher::narrow_oop_use_complex_address defined in opto/matcher.hpp. > > n.b. the ad file for a given architecture can control this feature by > how it defines method narrow_oop_use_complex_address. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Wed Nov 7 10:46:18 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 7 Nov 2018 10:46:18 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE2C04C.50507@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <90d4e32b-fd85-8cb1-a8e9-6bb49f211f0c@redhat.com> <5BE2C04C.50507@oracle.com> Message-ID: On 07/11/18 10:37, Erik ?sterlund wrote: > Thanks for digging that up. No problem. The funny thing is that I knew about this ad file function and -- roughly -- what it would achieve in terms of the graph reshape and subsequent code generation. However, I never put 2 + 2 together and followed through with the conclusion that this would cause loads of oops stored as zero to potentially be modelled by and dereferenced at address heap_base. It's all very devious! Do you think a small explanatory comment in uses_implicit_null_check pointing readers at gen_narrow_oop_implicit_null_checks might be worth including? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From david.holmes at oracle.com Wed Nov 7 11:03:33 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Nov 2018 21:03:33 +1000 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: <2663554a-9600-1a3f-e2f4-0576e4688912@oracle.com> References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> <2663554a-9600-1a3f-e2f4-0576e4688912@oracle.com> Message-ID: <5ec696e0-5d2f-802a-32b8-9a105a9c061a@oracle.com> On 7/11/2018 8:04 PM, Robbin Ehn wrote: > Hi David, > > On 2018-11-07 10:32, David Holmes wrote: >> Hi Robbin, >> >> On 7/11/2018 6:32 PM, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> We rename SafepointMechanism::poll to >>> SafepointMechanism::should_block to make it more clear what the >>> method do. >> >> Fair enough, but doesn't that just imply that global_poll and >> local_poll are also mis-named? How far should this rename go? > > I think they are fine, because of the different context. As I see it > should_block is implemented by actively checking (polling) the threads > page or the global state. Except there's a level in between that. If poll() should be should_block() then global_poll() should be global_should_block() and local_poll() should be local_should_block(). I don't see any difference. It's the methods under that that actually "poll" something. David > > Thanks, Robbin > >> >> Thanks, >> David >> >>> >>> CR: https://bugs.openjdk.java.net/browse/JDK-8211403 >>> Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ >>> >>> Sanity on tier 1. >>> >>> Thanks, Robbin From erik.osterlund at oracle.com Wed Nov 7 11:07:23 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 12:07:23 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <90d4e32b-fd85-8cb1-a8e9-6bb49f211f0c@redhat.com> <5BE2C04C.50507@oracle.com> Message-ID: <5BE2C76B.50405@oracle.com> Hi Andrew, On 2018-11-07 11:46, Andrew Dinn wrote: > On 07/11/18 10:37, Erik ?sterlund wrote: >> Thanks for digging that up. > No problem. The funny thing is that I knew about this ad file function > and -- roughly -- what it would achieve in terms of the graph reshape > and subsequent code generation. However, I never put 2 + 2 together and > followed through with the conclusion that this would cause loads of oops > stored as zero to potentially be modelled by and dereferenced at address > heap_base. > > It's all very devious! Agreed. > Do you think a small explanatory comment in uses_implicit_null_check > pointing readers at gen_narrow_oop_implicit_null_checks might be worth > including? Yes I think that would help for future adventurers stumbling across this code. Thanks, /Erik > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From robbin.ehn at oracle.com Wed Nov 7 11:37:09 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 7 Nov 2018 12:37:09 +0100 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: <5ec696e0-5d2f-802a-32b8-9a105a9c061a@oracle.com> References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> <2663554a-9600-1a3f-e2f4-0576e4688912@oracle.com> <5ec696e0-5d2f-802a-32b8-9a105a9c061a@oracle.com> Message-ID: Hi David, > Except there's a level in between that. If poll() should be should_block() then > global_poll() should be global_should_block() and local_poll() should be > local_should_block(). I don't see any difference. It's the methods under that > that actually "poll" something. The difference is private vs public. I'm fixing the issue that some raised about this: if (SafepointMechanism::poll(Self)) return 0; Not being clear as: if (SafepointMechanism::should_block(Self)) return 0; The implementation is fine by me, I do not consider changes there my goal at all. Thanks, Robbin > > David > >> >> Thanks, Robbin >> >>> >>> Thanks, >>> David >>> >>>> >>>> CR: https://bugs.openjdk.java.net/browse/JDK-8211403 >>>> Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ >>>> >>>> Sanity on tier 1. >>>> >>>> Thanks, Robbin From erik.osterlund at oracle.com Wed Nov 7 13:42:17 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 14:42:17 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> Message-ID: <5BE2EBB9.3040804@oracle.com> Hi Roman, The approach looks great! :p Looks like "you" forgot to call the new function in os_solaris_sparc.cpp though, he he. With that fixed, we pass tier1-3 on our platforms. Oh and the cast in the following line is unnecessary: intptr_t start = (intptr_t)-cell_header_size; Thanks, /Erik On 2018-11-07 09:52, Roman Kennke wrote: > Alright! thanks Erik! So here I'd like to propose your version plus my > little fixes for review: > > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.02/ > > It passes hotspot/jtreg:tier1 locally on x86 and aarch64 and works with > Shenandoah too (and passes the Shenandoah testsuites) > > Good? > > Roman > > >> Hi Roman, >> >> Glad you liked the idea! >> >> On 2018-11-06 22:37, Roman Kennke wrote: >>> Hi Erik, >>> >>> I also like it. I needed to change a little bit to make it work with >>> Shenandoah: >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch >>> >>> >>> I will give it a spin on aarch64 as soon as I get access to the machine. >>> >>> Do you want to take the bug and finish it, or do you want me to take it >>> from here? >> You can take it from here. I just wanted to show a possible direction. >> >> Thanks, >> /Erik >> >>> Roman >>> >>>> Hi Andrew, >>>> >>>> On 2018-11-06 17:46, Andrew Dinn wrote: >>>>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>>>> New webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>>>> Thanks for the updated version. >>>>> >>>>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>>>> In that case, I hear you ask, why does the old code check for >>>>>>> values in >>>>>>> the range [heap_base - cell_header_size, heap_base + >>>>>>> os::vm_page_size)? >>>>>>> Good question. I don't know the answer. I think that test is bogus >>>>>>> with >>>>>>> no Shenandoah GC present. >>>>>> Interesting. It turns out that both ranges are relevant >>>>>> [-cell_header_size, os::vm_page_size) and [heap_base - >>>>>> cell_header_size, >>>>>> heap_base + os::vm_page_size). Looks like we both missed that from the >>>>>> original check. >>>>>> I updated my webrev to explicitly reflect those two ranges, as two >>>>>> different checks. >>>>> Well, you say that but -- as I said before -- I am not sure why *both* >>>>> are relevant to the non-Shenandoah case. And, given Roman's correction >>>>> (doh, yes of course! the space for the Brooks pointer field had better >>>>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>>>> Shenandoah case either. >>>> The reason it is needed is because there may or may not be null checks >>>> in both the decode() step to get the base pointer of an access, and the >>>> access itself, given some base pointer. This results in two ranges for >>>> those two cases correspondingly. If the decoding uses decode_not_null, >>>> with an implicit null check in the decoding itself that blindly adds the >>>> heap base without checking for null, we will crash into the first page >>>> of the heap which is protected when performing accesses into that. But >>>> sometimes we have a decode()ed base pointer (with null check), that has >>>> already been fixed up to NULL, and then perform an access at an offset >>>> into that with an implicit null check. This results in a trap at the >>>> first page. So in summary, both ranges are needed because the >>>> implicitness of the null check may be both in the access and the >>>> narrowOop decoding. >>>> >>>> But as I pointed out, this (seemingly incorrectly) assumes there may not >>>> be any offset values in the (heap_base, heap_base + os::vm_page_size) >>>> range. If you hit such an offset, we will generate incorrect code, due >>>> to the lack of distinction between addresses and offsets. >>>> >>>>> I'd be very glad to hear from someone who knows about the history of >>>>> the >>>>> original function as to why the extra test for addresses above >>>>> heap_base >>>>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>>>> might have been around when it was added?] >>>>> >>>>> If that check is not actually needed then getting rid of it would make >>>>> the whole thing a lot simpler. >>>> It would indeed. But unfortunately we need it. >>>> >>>>>>>> Just expressing these trivial ranges separately would have been very >>>>>>>> easy to understand. >>>>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>>>> wrinkle. >>>>>>> It is possible for an object to be allocated at the first non-guarded >>>>>>> address in the heap area i.e. heap_base + os::vm_page_size. I that >>>>>>> case >>>>>>> a dereference of its Brooks pointer will lie in the protected page. >>>>>> Looks like Roman just said that's not possible. >>>>> Yes, my dumb! >>>>> >>>>>> Fixed. Might need some help testing this on AArch64 though if we >>>>>> decide >>>>>> to go with it. >>>>> I am sure Roman and Aleksey can do that but if needed I will. >>>>> >>>>>> Do you agree though, that the current version that merges the address >>>>>> and offset case relies on offsets being smaller than the heap base for >>>>>> its magic check to work, and that we actually do not have such a >>>>>> guarantee in the VM? Apart from readability preferences, this seems >>>>>> like >>>>>> it is broken in the current form, unless there are seat belts I am not >>>>>> aware of. In particular, if you have an offset value that is in the >>>>>> virtual address range (heap_base, heap_base + os::vm_page_size()), >>>>>> which >>>>>> I argue can happen, then it looks like we will normalize that as if it >>>>>> was an address (even though it is not), and incorrectly say that this >>>>>> offset does not need explicit null checks, despite needing explicit >>>>>> null >>>>>> checks. >>>>> Well, without the answer as to whether the check against heap_base is >>>>> needed I am not sure. Assuming it is then then ... >>>>> >>>>> Yes, it is possible, in theory, that an oop offset could turn out to >>>>> have value (heap_base + D) where 0 <= D < os_page_size(). In that case >>>>> the offset might wrongly be classified as *not* needing a null check >>>>> even though it really does. I suspect it is unlikely in practice given >>>>> how mmap hands out addresses but you never know. >>>>> >>>>> As for readibility, I think the two functions are probably clearer -- >>>>> expecially if they stay adjacent in the source code. >>>> Okay, sounds like we are on the same page. I agree it is unlikely that >>>> this will crash the VM (as evidenced by the lack of crashes observed). >>>> But I don't like relying on luck for correctness. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> regards, >>>>> >>>>> >>>>> Andrew Dinn >>>>> ----------- >>>>> Senior Principal Software Engineer >>>>> Red Hat UK Ltd >>>>> Registered in England and Wales under Company Registration No. 03798903 >>>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >>>>> From erik.osterlund at oracle.com Wed Nov 7 13:45:27 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 14:45:27 +0100 Subject: RFR: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) In-Reply-To: References: <5BE16E2A.9090107@oracle.com> Message-ID: <5BE2EC77.6030809@oracle.com> Hi Vladimir, Thank you for the review. I made an abstraction for the bit fiddling as suggested to make it more readable. There is now an AllStatic IsUnloadingState class that has a mask and shift for each field, expresses getters and setters using them, and a create function for setting some passed in fields into a new bit field. Webrev: http://cr.openjdk.java.net/~eosterlund/8213411/webrev.01/ Thanks, /Erik On 2018-11-06 17:21, Vladimir Kozlov wrote: > Hi Erik, > > It would be nice if you use enum for values, masks and shift (like for > markOop). It will become more readable. > > Thanks, > Vladimir > > On 11/6/18 2:34 AM, Erik ?sterlund wrote: >> Hi, >> >> In my patch for 8209189 I assumed that the following struct >> containing a bit field using 3 bits, would be packed to one byte: >> >> struct IsUnloadingStruct { >> unsigned int _is_unloading:1; >> unsigned int _unloading_cycle:2; >> }; >> >> Turns out though that the size of this 3 bit bitfield is typically 4 >> bytes, causing trouble on big endian machines. The compilers can be >> tamed to turn this to a byte (as expected) by using struct packing. >> >> So the two most obvious solutions are either >> 1) Make a struct packing macro, and trust the compilers won't mess up >> then. >> 2) Do manual bit fiddling because we do not trust compilers. >> >> Martin convinced me to go with #2. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8213411 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8213411/webrev.00/ >> >> Running a bunch of tests while waiting for review. >> >> Thanks, >> /Erik From rkennke at redhat.com Wed Nov 7 14:51:58 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 7 Nov 2018 15:51:58 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE2EBB9.3040804@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> Message-ID: <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> Hi Erik, > The approach looks great! :p Ha! :-) > Looks like "you" forgot to call the new function in os_solaris_sparc.cpp > though, he he. With that fixed, we pass tier1-3 on our platforms. > Oh and the cast in the following line is unnecessary: intptr_t start = > (intptr_t)-cell_header_size; Both fixed here (solaris/sparc stuff blindly though): Incremental: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03.diff Full: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03/ Good now? Thanks for reviewing and helping! Roman > Thanks, > /Erik > > On 2018-11-07 09:52, Roman Kennke wrote: >> Alright! thanks Erik! So here I'd like to propose your version plus my >> little fixes for review: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.02/ >> >> It passes hotspot/jtreg:tier1 locally on x86 and aarch64 and works with >> Shenandoah too (and passes the Shenandoah testsuites) >> >> Good? >> >> Roman >> >> >>> Hi Roman, >>> >>> Glad you liked the idea! >>> >>> On 2018-11-06 22:37, Roman Kennke wrote: >>>> Hi Erik, >>>> >>>> I also like it. I needed to change a little bit to make it work with >>>> Shenandoah: >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch >>>> >>>> >>>> >>>> I will give it a spin on aarch64 as soon as I get access to the >>>> machine. >>>> >>>> Do you want to take the bug and finish it, or do you want me to take it >>>> from here? >>> You can take it from here. I just wanted to show a possible direction. >>> >>> Thanks, >>> /Erik >>> >>>> Roman >>>> >>>>> Hi Andrew, >>>>> >>>>> On 2018-11-06 17:46, Andrew Dinn wrote: >>>>>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>>>>> New webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>>>>> Thanks for the updated version. >>>>>> >>>>>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>>>>> In that case, I hear you ask, why does the old code check for >>>>>>>> values in >>>>>>>> the range [heap_base - cell_header_size, heap_base + >>>>>>>> os::vm_page_size)? >>>>>>>> Good question. I don't know the answer. I think that test is bogus >>>>>>>> with >>>>>>>> no Shenandoah GC present. >>>>>>> Interesting. It turns out that both ranges are relevant >>>>>>> [-cell_header_size, os::vm_page_size) and [heap_base - >>>>>>> cell_header_size, >>>>>>> heap_base + os::vm_page_size). Looks like we both missed that >>>>>>> from the >>>>>>> original check. >>>>>>> I updated my webrev to explicitly reflect those two ranges, as two >>>>>>> different checks. >>>>>> Well, you say that but -- as I said before -- I am not sure why >>>>>> *both* >>>>>> are relevant to the non-Shenandoah case. And, given Roman's >>>>>> correction >>>>>> (doh, yes of course! the space for the Brooks pointer field had >>>>>> better >>>>>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>>>>> Shenandoah case either. >>>>> The reason it is needed is because there may or may not be null checks >>>>> in both the decode() step to get the base pointer of an access, and >>>>> the >>>>> access itself, given some base pointer. This results in two ranges for >>>>> those two cases correspondingly. If the decoding uses decode_not_null, >>>>> with an implicit null check in the decoding itself that blindly >>>>> adds the >>>>> heap base without checking for null, we will crash into the first page >>>>> of the heap which is protected when performing accesses into that. But >>>>> sometimes we have a decode()ed base pointer (with null check), that >>>>> has >>>>> already been fixed up to NULL, and then perform an access at an offset >>>>> into that with an implicit null check. This results in a trap at the >>>>> first page. So in summary, both ranges are needed because the >>>>> implicitness of the null check may be both in the access and the >>>>> narrowOop decoding. >>>>> >>>>> But as I pointed out, this (seemingly incorrectly) assumes there >>>>> may not >>>>> be any offset values in the (heap_base, heap_base + os::vm_page_size) >>>>> range. If you hit such an offset, we will generate incorrect code, due >>>>> to the lack of distinction between addresses and offsets. >>>>> >>>>>> I'd be very glad to hear from someone who knows about the history of >>>>>> the >>>>>> original function as to why the extra test for addresses above >>>>>> heap_base >>>>>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>>>>> might have been around when it was added?] >>>>>> >>>>>> If that check is not actually needed then getting rid of it would >>>>>> make >>>>>> the whole thing a lot simpler. >>>>> It would indeed. But unfortunately we need it. >>>>> >>>>>>>>> Just expressing these trivial ranges separately would have been >>>>>>>>> very >>>>>>>>> easy to understand. >>>>>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>>>>> wrinkle. >>>>>>>> It is possible for an object to be allocated at the first >>>>>>>> non-guarded >>>>>>>> address in the heap area i.e. heap_base + os::vm_page_size. I that >>>>>>>> case >>>>>>>> a dereference of its Brooks pointer will lie in the protected page. >>>>>>> Looks like Roman just said that's not possible. >>>>>> Yes, my dumb! >>>>>> >>>>>>> Fixed. Might need some help testing this on AArch64 though if we >>>>>>> decide >>>>>>> to go with it. >>>>>> I am sure Roman and Aleksey can do that but if needed I will. >>>>>> >>>>>>> Do you agree though, that the current version that merges the >>>>>>> address >>>>>>> and offset case relies on offsets being smaller than the heap >>>>>>> base for >>>>>>> its magic check to work, and that we actually do not have such a >>>>>>> guarantee in the VM? Apart from readability preferences, this seems >>>>>>> like >>>>>>> it is broken in the current form, unless there are seat belts I >>>>>>> am not >>>>>>> aware of. In particular, if you have an offset value that is in the >>>>>>> virtual address range (heap_base, heap_base + os::vm_page_size()), >>>>>>> which >>>>>>> I argue can happen, then it looks like we will normalize that as >>>>>>> if it >>>>>>> was an address (even though it is not), and incorrectly say that >>>>>>> this >>>>>>> offset does not need explicit null checks, despite needing explicit >>>>>>> null >>>>>>> checks. >>>>>> Well, without the answer as to whether the check against heap_base is >>>>>> needed I am not sure. Assuming it is then then ... >>>>>> >>>>>> Yes, it is possible, in theory, that an oop offset could turn out to >>>>>> have value (heap_base + D) where 0 <= D < os_page_size(). In that >>>>>> case >>>>>> the offset might wrongly be classified as *not* needing a null check >>>>>> even though it really does. I suspect it is unlikely in practice >>>>>> given >>>>>> how mmap hands out addresses but you never know. >>>>>> >>>>>> As for readibility, I think the two functions are probably clearer -- >>>>>> expecially if they stay adjacent in the source code. >>>>> Okay, sounds like we are on the same page. I agree it is unlikely that >>>>> this will crash the VM (as evidenced by the lack of crashes observed). >>>>> But I don't like relying on luck for correctness. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> regards, >>>>>> >>>>>> >>>>>> Andrew Dinn >>>>>> ----------- >>>>>> Senior Principal Software Engineer >>>>>> Red Hat UK Ltd >>>>>> Registered in England and Wales under Company Registration No. >>>>>> 03798903 >>>>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >>>>>> > From erik.osterlund at oracle.com Wed Nov 7 15:17:07 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 16:17:07 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> Message-ID: <5BE301F3.9080706@oracle.com> Hi Roman, The si_addr is always void*, so the cast to void* is redundant. Otherwise, looks good. Don't need to see another webrev for that. Thanks, /Erik On 2018-11-07 15:51, Roman Kennke wrote: > Hi Erik, > >> The approach looks great! :p > Ha! :-) > >> Looks like "you" forgot to call the new function in os_solaris_sparc.cpp >> though, he he. With that fixed, we pass tier1-3 on our platforms. >> Oh and the cast in the following line is unnecessary: intptr_t start = >> (intptr_t)-cell_header_size; > Both fixed here (solaris/sparc stuff blindly though): > Incremental: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03.diff > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03/ > > Good now? > > Thanks for reviewing and helping! > > Roman > >> Thanks, >> /Erik >> >> On 2018-11-07 09:52, Roman Kennke wrote: >>> Alright! thanks Erik! So here I'd like to propose your version plus my >>> little fixes for review: >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.02/ >>> >>> It passes hotspot/jtreg:tier1 locally on x86 and aarch64 and works with >>> Shenandoah too (and passes the Shenandoah testsuites) >>> >>> Good? >>> >>> Roman >>> >>> >>>> Hi Roman, >>>> >>>> Glad you liked the idea! >>>> >>>> On 2018-11-06 22:37, Roman Kennke wrote: >>>>> Hi Erik, >>>>> >>>>> I also like it. I needed to change a little bit to make it work with >>>>> Shenandoah: >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch >>>>> >>>>> >>>>> >>>>> I will give it a spin on aarch64 as soon as I get access to the >>>>> machine. >>>>> >>>>> Do you want to take the bug and finish it, or do you want me to take it >>>>> from here? >>>> You can take it from here. I just wanted to show a possible direction. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> Roman >>>>> >>>>>> Hi Andrew, >>>>>> >>>>>> On 2018-11-06 17:46, Andrew Dinn wrote: >>>>>>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>>>>>> New webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>>>>>> Thanks for the updated version. >>>>>>> >>>>>>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>>>>>> In that case, I hear you ask, why does the old code check for >>>>>>>>> values in >>>>>>>>> the range [heap_base - cell_header_size, heap_base + >>>>>>>>> os::vm_page_size)? >>>>>>>>> Good question. I don't know the answer. I think that test is bogus >>>>>>>>> with >>>>>>>>> no Shenandoah GC present. >>>>>>>> Interesting. It turns out that both ranges are relevant >>>>>>>> [-cell_header_size, os::vm_page_size) and [heap_base - >>>>>>>> cell_header_size, >>>>>>>> heap_base + os::vm_page_size). Looks like we both missed that >>>>>>>> from the >>>>>>>> original check. >>>>>>>> I updated my webrev to explicitly reflect those two ranges, as two >>>>>>>> different checks. >>>>>>> Well, you say that but -- as I said before -- I am not sure why >>>>>>> *both* >>>>>>> are relevant to the non-Shenandoah case. And, given Roman's >>>>>>> correction >>>>>>> (doh, yes of course! the space for the Brooks pointer field had >>>>>>> better >>>>>>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>>>>>> Shenandoah case either. >>>>>> The reason it is needed is because there may or may not be null checks >>>>>> in both the decode() step to get the base pointer of an access, and >>>>>> the >>>>>> access itself, given some base pointer. This results in two ranges for >>>>>> those two cases correspondingly. If the decoding uses decode_not_null, >>>>>> with an implicit null check in the decoding itself that blindly >>>>>> adds the >>>>>> heap base without checking for null, we will crash into the first page >>>>>> of the heap which is protected when performing accesses into that. But >>>>>> sometimes we have a decode()ed base pointer (with null check), that >>>>>> has >>>>>> already been fixed up to NULL, and then perform an access at an offset >>>>>> into that with an implicit null check. This results in a trap at the >>>>>> first page. So in summary, both ranges are needed because the >>>>>> implicitness of the null check may be both in the access and the >>>>>> narrowOop decoding. >>>>>> >>>>>> But as I pointed out, this (seemingly incorrectly) assumes there >>>>>> may not >>>>>> be any offset values in the (heap_base, heap_base + os::vm_page_size) >>>>>> range. If you hit such an offset, we will generate incorrect code, due >>>>>> to the lack of distinction between addresses and offsets. >>>>>> >>>>>>> I'd be very glad to hear from someone who knows about the history of >>>>>>> the >>>>>>> original function as to why the extra test for addresses above >>>>>>> heap_base >>>>>>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>>>>>> might have been around when it was added?] >>>>>>> >>>>>>> If that check is not actually needed then getting rid of it would >>>>>>> make >>>>>>> the whole thing a lot simpler. >>>>>> It would indeed. But unfortunately we need it. >>>>>> >>>>>>>>>> Just expressing these trivial ranges separately would have been >>>>>>>>>> very >>>>>>>>>> easy to understand. >>>>>>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>>>>>> wrinkle. >>>>>>>>> It is possible for an object to be allocated at the first >>>>>>>>> non-guarded >>>>>>>>> address in the heap area i.e. heap_base + os::vm_page_size. I that >>>>>>>>> case >>>>>>>>> a dereference of its Brooks pointer will lie in the protected page. >>>>>>>> Looks like Roman just said that's not possible. >>>>>>> Yes, my dumb! >>>>>>> >>>>>>>> Fixed. Might need some help testing this on AArch64 though if we >>>>>>>> decide >>>>>>>> to go with it. >>>>>>> I am sure Roman and Aleksey can do that but if needed I will. >>>>>>> >>>>>>>> Do you agree though, that the current version that merges the >>>>>>>> address >>>>>>>> and offset case relies on offsets being smaller than the heap >>>>>>>> base for >>>>>>>> its magic check to work, and that we actually do not have such a >>>>>>>> guarantee in the VM? Apart from readability preferences, this seems >>>>>>>> like >>>>>>>> it is broken in the current form, unless there are seat belts I >>>>>>>> am not >>>>>>>> aware of. In particular, if you have an offset value that is in the >>>>>>>> virtual address range (heap_base, heap_base + os::vm_page_size()), >>>>>>>> which >>>>>>>> I argue can happen, then it looks like we will normalize that as >>>>>>>> if it >>>>>>>> was an address (even though it is not), and incorrectly say that >>>>>>>> this >>>>>>>> offset does not need explicit null checks, despite needing explicit >>>>>>>> null >>>>>>>> checks. >>>>>>> Well, without the answer as to whether the check against heap_base is >>>>>>> needed I am not sure. Assuming it is then then ... >>>>>>> >>>>>>> Yes, it is possible, in theory, that an oop offset could turn out to >>>>>>> have value (heap_base + D) where 0 <= D < os_page_size(). In that >>>>>>> case >>>>>>> the offset might wrongly be classified as *not* needing a null check >>>>>>> even though it really does. I suspect it is unlikely in practice >>>>>>> given >>>>>>> how mmap hands out addresses but you never know. >>>>>>> >>>>>>> As for readibility, I think the two functions are probably clearer -- >>>>>>> expecially if they stay adjacent in the source code. >>>>>> Okay, sounds like we are on the same page. I agree it is unlikely that >>>>>> this will crash the VM (as evidenced by the lack of crashes observed). >>>>>> But I don't like relying on luck for correctness. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> regards, >>>>>>> >>>>>>> >>>>>>> Andrew Dinn >>>>>>> ----------- >>>>>>> Senior Principal Software Engineer >>>>>>> Red Hat UK Ltd >>>>>>> Registered in England and Wales under Company Registration No. >>>>>>> 03798903 >>>>>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >>>>>>> From rkennke at redhat.com Wed Nov 7 15:23:27 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 7 Nov 2018 16:23:27 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE301F3.9080706@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> Message-ID: Hi Erik, > The si_addr is always void*, so the cast to void* is redundant. > Otherwise, looks good. > Don't need to see another webrev for that. Right. Thanks again for reviewing and helping. I'll push it through jdk/submit while waiting for another review. Andrew? Roman > Thanks, > /Erik > > On 2018-11-07 15:51, Roman Kennke wrote: >> Hi Erik, >> >>> The approach looks great! :p >> Ha! :-) >> >>> Looks like "you" forgot to call the new function in os_solaris_sparc.cpp >>> though, he he. With that fixed, we pass tier1-3 on our platforms. >>> Oh and the cast in the following line is unnecessary: intptr_t start = >>> (intptr_t)-cell_header_size; >> Both fixed here (solaris/sparc stuff blindly though): >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03.diff >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03/ >> >> Good now? >> >> Thanks for reviewing and helping! >> >> Roman >> >>> Thanks, >>> /Erik >>> >>> On 2018-11-07 09:52, Roman Kennke wrote: >>>> Alright! thanks Erik! So here I'd like to propose your version plus my >>>> little fixes for review: >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.02/ >>>> >>>> It passes hotspot/jtreg:tier1 locally on x86 and aarch64 and works with >>>> Shenandoah too (and passes the Shenandoah testsuites) >>>> >>>> Good? >>>> >>>> Roman >>>> >>>> >>>>> Hi Roman, >>>>> >>>>> Glad you liked the idea! >>>>> >>>>> On 2018-11-06 22:37, Roman Kennke wrote: >>>>>> Hi Erik, >>>>>> >>>>>> I also like it. I needed to change a little bit to make it work with >>>>>> Shenandoah: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I will give it a spin on aarch64 as soon as I get access to the >>>>>> machine. >>>>>> >>>>>> Do you want to take the bug and finish it, or do you want me to >>>>>> take it >>>>>> from here? >>>>> You can take it from here. I just wanted to show a possible direction. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> Roman >>>>>> >>>>>>> Hi Andrew, >>>>>>> >>>>>>> On 2018-11-06 17:46, Andrew Dinn wrote: >>>>>>>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>>>>>>> New webrev: >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>>>>>>> Thanks for the updated version. >>>>>>>> >>>>>>>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>>>>>>> In that case, I hear you ask, why does the old code check for >>>>>>>>>> values in >>>>>>>>>> the range [heap_base - cell_header_size, heap_base + >>>>>>>>>> os::vm_page_size)? >>>>>>>>>> Good question. I don't know the answer. I think that test is >>>>>>>>>> bogus >>>>>>>>>> with >>>>>>>>>> no Shenandoah GC present. >>>>>>>>> Interesting. It turns out that both ranges are relevant >>>>>>>>> [-cell_header_size, os::vm_page_size) and [heap_base - >>>>>>>>> cell_header_size, >>>>>>>>> heap_base + os::vm_page_size). Looks like we both missed that >>>>>>>>> from the >>>>>>>>> original check. >>>>>>>>> I updated my webrev to explicitly reflect those two ranges, as two >>>>>>>>> different checks. >>>>>>>> Well, you say that but -- as I said before -- I am not sure why >>>>>>>> *both* >>>>>>>> are relevant to the non-Shenandoah case. And, given Roman's >>>>>>>> correction >>>>>>>> (doh, yes of course! the space for the Brooks pointer field had >>>>>>>> better >>>>>>>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>>>>>>> Shenandoah case either. >>>>>>> The reason it is needed is because there may or may not be null >>>>>>> checks >>>>>>> in both the decode() step to get the base pointer of an access, and >>>>>>> the >>>>>>> access itself, given some base pointer. This results in two >>>>>>> ranges for >>>>>>> those two cases correspondingly. If the decoding uses >>>>>>> decode_not_null, >>>>>>> with an implicit null check in the decoding itself that blindly >>>>>>> adds the >>>>>>> heap base without checking for null, we will crash into the first >>>>>>> page >>>>>>> of the heap which is protected when performing accesses into >>>>>>> that. But >>>>>>> sometimes we have a decode()ed base pointer (with null check), that >>>>>>> has >>>>>>> already been fixed up to NULL, and then perform an access at an >>>>>>> offset >>>>>>> into that with an implicit null check. This results in a trap at the >>>>>>> first page. So in summary, both ranges are needed because the >>>>>>> implicitness of the null check may be both in the access and the >>>>>>> narrowOop decoding. >>>>>>> >>>>>>> But as I pointed out, this (seemingly incorrectly) assumes there >>>>>>> may not >>>>>>> be any offset values in the (heap_base, heap_base + >>>>>>> os::vm_page_size) >>>>>>> range. If you hit such an offset, we will generate incorrect >>>>>>> code, due >>>>>>> to the lack of distinction between addresses and offsets. >>>>>>> >>>>>>>> I'd be very glad to hear from someone who knows about the >>>>>>>> history of >>>>>>>> the >>>>>>>> original function as to why the extra test for addresses above >>>>>>>> heap_base >>>>>>>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>>>>>>> might have been around when it was added?] >>>>>>>> >>>>>>>> If that check is not actually needed then getting rid of it would >>>>>>>> make >>>>>>>> the whole thing a lot simpler. >>>>>>> It would indeed. But unfortunately we need it. >>>>>>> >>>>>>>>>>> Just expressing these trivial ranges separately would have been >>>>>>>>>>> very >>>>>>>>>>> easy to understand. >>>>>>>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>>>>>>> wrinkle. >>>>>>>>>> It is possible for an object to be allocated at the first >>>>>>>>>> non-guarded >>>>>>>>>> address in the heap area i.e. heap_base + os::vm_page_size. I >>>>>>>>>> that >>>>>>>>>> case >>>>>>>>>> a dereference of its Brooks pointer will lie in the protected >>>>>>>>>> page. >>>>>>>>> Looks like Roman just said that's not possible. >>>>>>>> Yes, my dumb! >>>>>>>> >>>>>>>>> Fixed. Might need some help testing this on AArch64 though if we >>>>>>>>> decide >>>>>>>>> to go with it. >>>>>>>> I am sure Roman and Aleksey can do that but if needed I will. >>>>>>>> >>>>>>>>> Do you agree though, that the current version that merges the >>>>>>>>> address >>>>>>>>> and offset case relies on offsets being smaller than the heap >>>>>>>>> base for >>>>>>>>> its magic check to work, and that we actually do not have such a >>>>>>>>> guarantee in the VM? Apart from readability preferences, this >>>>>>>>> seems >>>>>>>>> like >>>>>>>>> it is broken in the current form, unless there are seat belts I >>>>>>>>> am not >>>>>>>>> aware of. In particular, if you have an offset value that is in >>>>>>>>> the >>>>>>>>> virtual address range (heap_base, heap_base + os::vm_page_size()), >>>>>>>>> which >>>>>>>>> I argue can happen, then it looks like we will normalize that as >>>>>>>>> if it >>>>>>>>> was an address (even though it is not), and incorrectly say that >>>>>>>>> this >>>>>>>>> offset does not need explicit null checks, despite needing >>>>>>>>> explicit >>>>>>>>> null >>>>>>>>> checks. >>>>>>>> Well, without the answer as to whether the check against >>>>>>>> heap_base is >>>>>>>> needed I am not sure. Assuming it is then then ... >>>>>>>> >>>>>>>> Yes, it is possible, in theory, that an oop offset could turn >>>>>>>> out to >>>>>>>> have value (heap_base + D) where 0 <= D < os_page_size(). In that >>>>>>>> case >>>>>>>> the offset might wrongly be classified as *not* needing a null >>>>>>>> check >>>>>>>> even though it really does. I suspect it is unlikely in practice >>>>>>>> given >>>>>>>> how mmap hands out addresses but you never know. >>>>>>>> >>>>>>>> As for readibility, I think the two functions are probably >>>>>>>> clearer -- >>>>>>>> expecially if they stay adjacent in the source code. >>>>>>> Okay, sounds like we are on the same page. I agree it is unlikely >>>>>>> that >>>>>>> this will crash the VM (as evidenced by the lack of crashes >>>>>>> observed). >>>>>>> But I don't like relying on luck for correctness. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>>> regards, >>>>>>>> >>>>>>>> >>>>>>>> Andrew Dinn >>>>>>>> ----------- >>>>>>>> Senior Principal Software Engineer >>>>>>>> Red Hat UK Ltd >>>>>>>> Registered in England and Wales under Company Registration No. >>>>>>>> 03798903 >>>>>>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric >>>>>>>> Shander >>>>>>>> > From vicente.romero at oracle.com Wed Nov 7 15:56:55 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Wed, 7 Nov 2018 10:56:55 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 Message-ID: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> Hi, Version 7.0 of ASM has been released. This version supports condy, yay!, and we want to include it in JDK 12. Please review [1] which includes: - the new version perse substituting the preview ASM internal version in the JDK - changes to additional files in particular some tests, mostly hotspot tests. Thanks, Vicente [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ From adinn at redhat.com Wed Nov 7 16:34:19 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 7 Nov 2018 16:34:19 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> Message-ID: <946ddb61-1b2e-56f2-6374-1b0e8498e64d@redhat.com> Hi Roman, On 07/11/18 15:23, Roman Kennke wrote: > Hi Erik, > >> The si_addr is always void*, so the cast to void* is redundant. >> Otherwise, looks good. >> Don't need to see another webrev for that. > > Right. Thanks again for reviewing and helping. > > I'll push it through jdk/submit while waiting for another review. Andrew? Just one issue regarding the comment in assembler.cpp 319 // The first page after heap_base is unmapped and 320 // the 'offset' is equal to [heap_base + offset] for 321 // narrow oop implicit null checks. That's not the general case. It's only potentially true for narrow_oops read out of fields when Matcher::narrow_oop_use_complex_address returns true. I would prefer for these specifics to be made clear as understanding it and tracking it down was not straightforward. So, how about // A SEGV can legitimately happen in C2 code at address // (heap_base + offset) if Matcher::narrow_oop_use_complex_address // is configured to allow narrow oops field loads to be implicitly // null checked Otherwise all good. regards, Andrew Dinn ----------- From rkennke at redhat.com Wed Nov 7 16:38:27 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 7 Nov 2018 17:38:27 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <946ddb61-1b2e-56f2-6374-1b0e8498e64d@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> <946ddb61-1b2e-56f2-6374-1b0e8498e64d@redhat.com> Message-ID: Right. Forgot about that. New (and hopefully final) webrev: http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.04/ ? Roman > Hi Roman, > > On 07/11/18 15:23, Roman Kennke wrote: >> Hi Erik, >> >>> The si_addr is always void*, so the cast to void* is redundant. >>> Otherwise, looks good. >>> Don't need to see another webrev for that. >> >> Right. Thanks again for reviewing and helping. >> >> I'll push it through jdk/submit while waiting for another review. Andrew? > Just one issue regarding the comment in assembler.cpp > > 319 // The first page after heap_base is unmapped and > 320 // the 'offset' is equal to [heap_base + offset] for > 321 // narrow oop implicit null checks. > > That's not the general case. It's only potentially true for narrow_oops > read out of fields when Matcher::narrow_oop_use_complex_address returns > true. > > I would prefer for these specifics to be made clear as understanding it > and tracking it down was not straightforward. So, how about > > // A SEGV can legitimately happen in C2 code at address > // (heap_base + offset) if Matcher::narrow_oop_use_complex_address > // is configured to allow narrow oops field loads to be implicitly > // null checked > > Otherwise all good. > > regards, > > > Andrew Dinn > ----------- > From vladimir.kozlov at oracle.com Wed Nov 7 16:43:29 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 7 Nov 2018 08:43:29 -0800 Subject: RFR: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) In-Reply-To: <5BE2EC77.6030809@oracle.com> References: <5BE16E2A.9090107@oracle.com> <5BE2EC77.6030809@oracle.com> Message-ID: Looks good. Thanks, Vladimir On 11/7/18 5:45 AM, Erik ?sterlund wrote: > Hi Vladimir, > > Thank you for the review. I made an abstraction for the bit fiddling as suggested to make it more readable. > There is now an AllStatic IsUnloadingState class that has a mask and shift for each field, expresses getters and setters > using them, and a create function for setting some passed in fields into a new bit field. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8213411/webrev.01/ > > Thanks, > /Erik > > On 2018-11-06 17:21, Vladimir Kozlov wrote: >> Hi Erik, >> >> It would be nice if you use enum for values, masks and shift (like for markOop). It will become more readable. >> >> Thanks, >> Vladimir >> >> On 11/6/18 2:34 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> In my patch for 8209189 I assumed that the following struct containing a bit field using 3 bits, would be packed to >>> one byte: >>> >>> struct IsUnloadingStruct { >>> ?? unsigned int _is_unloading:1; >>> ?? unsigned int _unloading_cycle:2; >>> }; >>> >>> Turns out though that the size of this 3 bit bitfield is typically 4 bytes, causing trouble on big endian machines. >>> The compilers can be tamed to turn this to a byte (as expected) by using struct packing. >>> >>> So the two most obvious solutions are either >>> 1) Make a struct packing macro, and trust the compilers won't mess up then. >>> 2) Do manual bit fiddling because we do not trust compilers. >>> >>> Martin convinced me to go with #2. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8213411 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213411/webrev.00/ >>> >>> Running a bunch of tests while waiting for review. >>> >>> Thanks, >>> /Erik > From erik.osterlund at oracle.com Wed Nov 7 16:53:39 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 7 Nov 2018 17:53:39 +0100 Subject: RFR: 8213411: JDK-8209189 incorrect for Big Endian (JVM crashes) In-Reply-To: References: <5BE16E2A.9090107@oracle.com> <5BE2EC77.6030809@oracle.com> Message-ID: <5BE31893.7000408@oracle.com> Hi Vladimir, Thanks for the review. /Erik On 2018-11-07 17:43, Vladimir Kozlov wrote: > Looks good. > > Thanks, > Vladimir > > On 11/7/18 5:45 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> >> Thank you for the review. I made an abstraction for the bit fiddling >> as suggested to make it more readable. >> There is now an AllStatic IsUnloadingState class that has a mask and >> shift for each field, expresses getters and setters using them, and a >> create function for setting some passed in fields into a new bit field. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8213411/webrev.01/ >> >> Thanks, >> /Erik >> >> On 2018-11-06 17:21, Vladimir Kozlov wrote: >>> Hi Erik, >>> >>> It would be nice if you use enum for values, masks and shift (like >>> for markOop). It will become more readable. >>> >>> Thanks, >>> Vladimir >>> >>> On 11/6/18 2:34 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> In my patch for 8209189 I assumed that the following struct >>>> containing a bit field using 3 bits, would be packed to one byte: >>>> >>>> struct IsUnloadingStruct { >>>> unsigned int _is_unloading:1; >>>> unsigned int _unloading_cycle:2; >>>> }; >>>> >>>> Turns out though that the size of this 3 bit bitfield is typically >>>> 4 bytes, causing trouble on big endian machines. The compilers can >>>> be tamed to turn this to a byte (as expected) by using struct packing. >>>> >>>> So the two most obvious solutions are either >>>> 1) Make a struct packing macro, and trust the compilers won't mess >>>> up then. >>>> 2) Do manual bit fiddling because we do not trust compilers. >>>> >>>> Martin convinced me to go with #2. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8213411 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213411/webrev.00/ >>>> >>>> Running a bunch of tests while waiting for review. >>>> >>>> Thanks, >>>> /Erik >> From daniel.daugherty at oracle.com Wed Nov 7 17:33:24 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 7 Nov 2018 12:33:24 -0500 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> Message-ID: On 11/7/18 3:32 AM, Robbin Ehn wrote: > Hi all, please review. > > We rename SafepointMechanism::poll to SafepointMechanism::should_block > to make > it more clear what the method do. > > CR: https://bugs.openjdk.java.net/browse/JDK-8211403 > Webrev: http://cr.openjdk.java.net/~rehn/8211403/webrev/ src/hotspot/cpu/zero/cppInterpreter_zero.cpp src/hotspot/share/jvmci/jvmciCodeInstaller.cpp src/hotspot/share/runtime/interfaceSupport.inline.hpp src/hotspot/share/runtime/mutex.cpp src/hotspot/share/runtime/objectMonitor.cpp ??? No comments on the above files. src/hotspot/share/runtime/safepointMechanism.hpp ??? Needs a copyright year update. ??? L76: ? // Call this method to see if this thread has depending poll and appropriate action should be taken ??? L77: ? static inline bool should_block(Thread* thread); ??????? The comment needs to be changed. Perhaps: ????????? // Call this method to see if this thread should block for a safepoint. src/hotspot/share/runtime/safepointMechanism.inline.hpp ??? Needs a copyright year update. src/hotspot/share/runtime/sweeper.cpp ??? No comments. Thumbs up!? Your call on whether to apply the renaming to global_poll() and local_poll(). I understand both points of view and I don't have a preference... Dan > > Sanity on tier 1. > > Thanks, Robbin > From bsrbnd at gmail.com Wed Nov 7 18:10:36 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 7 Nov 2018 19:10:36 +0100 Subject: RFR (S): JDK-8213479: Missing x86_64.ad patterns for 8-bit logical operators with destination in memory Message-ID: Hi, Please review the following small fix for [1] briefly discussed in thread [2]: http://cr.openjdk.java.net/~bsrbnd/jdk8213479/webrev.00/ It adds missing patterns to x86_64.ad for 8-bit logical operators with destination in memory. For example, the following method would use 4 instructions and 2 registers instead of currently 10 instructions and 5 registers, see [1]: private static byte and = 0b0011, or = 0b0011, xor = 0b0011; private static byte mask = 0b0101; public static void test() { and &= mask; or |= mask; xor ^= mask; } Tier1 is OK (hotspot+jdk+langtools). Any feedback is welcome, Bernard [1] https://bugs.openjdk.java.net/browse/JDK-8213479 [2] http://mail.openjdk.java.net/pipermail/jdk-dev/2018-November/002181.html From vladimir.kozlov at oracle.com Wed Nov 7 19:00:08 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 7 Nov 2018 11:00:08 -0800 Subject: RFR (S): JDK-8213479: Missing x86_64.ad patterns for 8-bit logical operators with destination in memory In-Reply-To: References: Message-ID: Hi Bernard, I am concern about next statement in Intel's Manual for these byte instructions: "*In 64-bit mode, r/m8 can not be encoded to access the following byte registers if a REX prefix is used: AH, BH, CH, DH." Also I have question about applications which benefit from it. Which applications has such pattern? I did not find it in discussion you pointed. Thanks, Vladimir On 11/7/18 10:10 AM, B. Blaser wrote: > Hi, > > Please review the following small fix for [1] briefly discussed in thread [2]: > > http://cr.openjdk.java.net/~bsrbnd/jdk8213479/webrev.00/ > > It adds missing patterns to x86_64.ad for 8-bit logical operators with > destination in memory. > For example, the following method would use 4 instructions and 2 > registers instead of currently 10 instructions and 5 registers, see > [1]: > > private static byte and = 0b0011, or = 0b0011, xor = 0b0011; > private static byte mask = 0b0101; > > public static void test() { > and &= mask; > > or |= mask; > > xor ^= mask; > } > > Tier1 is OK (hotspot+jdk+langtools). > > Any feedback is welcome, > Bernard > > [1] https://bugs.openjdk.java.net/browse/JDK-8213479 > [2] http://mail.openjdk.java.net/pipermail/jdk-dev/2018-November/002181.html > From bsrbnd at gmail.com Wed Nov 7 19:33:45 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 7 Nov 2018 20:33:45 +0100 Subject: RFR (S): JDK-8213479: Missing x86_64.ad patterns for 8-bit logical operators with destination in memory In-Reply-To: References: Message-ID: Hi Vladimir, On Wed, 7 Nov 2018 at 19:59, Vladimir Kozlov wrote: > > Hi Bernard, > > I am concern about next statement in Intel's Manual for these byte instructions: > > "*In 64-bit mode, r/m8 can not be encoded to access the following byte registers if a REX prefix is > used: AH, BH, CH, DH." yes, that's right but I guess we use AL, BL, CL and DL? > Also I have question about applications which benefit from it. Which applications has such pattern? > I did not find it in discussion you pointed. for example: http://hg.openjdk.java.net/jdk/jdk/file/f6583d7cff56/src/java.base/share/classes/java/lang/invoke/LambdaFormBuffer.java#l42 http://hg.openjdk.java.net/jdk/jdk/file/f6583d7cff56/src/java.base/share/classes/java/lang/invoke/LambdaFormBuffer.java#l234 http://hg.openjdk.java.net/jdk/jdk/file/f6583d7cff56/src/java.base/share/classes/java/lang/invoke/LambdaFormBuffer.java#l293 would benefit from these instructions. Please, let me know if you have any other comment. Thanks, Bernard > Thanks, > Vladimir > > On 11/7/18 10:10 AM, B. Blaser wrote: > > Hi, > > > > Please review the following small fix for [1] briefly discussed in thread [2]: > > > > http://cr.openjdk.java.net/~bsrbnd/jdk8213479/webrev.00/ > > > > It adds missing patterns to x86_64.ad for 8-bit logical operators with > > destination in memory. > > For example, the following method would use 4 instructions and 2 > > registers instead of currently 10 instructions and 5 registers, see > > [1]: > > > > private static byte and = 0b0011, or = 0b0011, xor = 0b0011; > > private static byte mask = 0b0101; > > > > public static void test() { > > and &= mask; > > > > or |= mask; > > > > xor ^= mask; > > } > > > > Tier1 is OK (hotspot+jdk+langtools). > > > > Any feedback is welcome, > > Bernard > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8213479 > > [2] http://mail.openjdk.java.net/pipermail/jdk-dev/2018-November/002181.html > > From igor.ignatyev at oracle.com Wed Nov 7 19:33:47 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 7 Nov 2018 11:33:47 -0800 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> Message-ID: Hi Vicente, I recall an (internal?) discussion about updating ASM somewhen in JDK 11TF, and AFAIR it was decided not to update ASM b/c nothing in JDK needs that, has it been changed? put somewhat differently, why are we doing this? in any case, I don't like the changes in mlvm tests. I understand that ClassWriter has been significantly changed in ASM 7.0, so ClassWriterExt can't disable CP entries caching (at least not in the way it used to), but removing setCache* calls from the tests changed them and in some cases made them invalid as they don't test that they supposed to. therefore I'd prefer to leave all calls setCache* as-is, change setCache* implementation to throw an exception (similarly to the fix in JDK-8194826 ) and mark all tests which throw this exception w/ '@ignore 8194951' jtreg tag. Thanks, -- Igor > On Nov 7, 2018, at 7:56 AM, Vicente Romero wrote: > > Hi, > > Version 7.0 of ASM has been released. This version supports condy, yay!, and we want to include it in JDK 12. Please review [1] which includes: > - the new version perse substituting the preview ASM internal version in the JDK > - changes to additional files in particular some tests, mostly hotspot tests. > > Thanks, > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ From vladimir.kozlov at oracle.com Wed Nov 7 19:46:42 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 7 Nov 2018 11:46:42 -0800 Subject: RFR (S): JDK-8213479: Missing x86_64.ad patterns for 8-bit logical operators with destination in memory In-Reply-To: References: Message-ID: <7a67ec0b-b9d1-303b-af71-b50b6b485aa0@oracle.com> On 11/7/18 11:33 AM, B. Blaser wrote: > Hi Vladimir, > > On Wed, 7 Nov 2018 at 19:59, Vladimir Kozlov wrote: >> >> Hi Bernard, >> >> I am concern about next statement in Intel's Manual for these byte instructions: >> >> "*In 64-bit mode, r/m8 can not be encoded to access the following byte registers if a REX prefix is >> used: AH, BH, CH, DH." > > yes, that's right but I guess we use AL, BL, CL and DL? Yes, I reread document and it is only AH, BH, CH, DH which we are not using. The code is good. > >> Also I have question about applications which benefit from it. Which applications has such pattern? >> I did not find it in discussion you pointed. > > for example: > > http://hg.openjdk.java.net/jdk/jdk/file/f6583d7cff56/src/java.base/share/classes/java/lang/invoke/LambdaFormBuffer.java#l42 > http://hg.openjdk.java.net/jdk/jdk/file/f6583d7cff56/src/java.base/share/classes/java/lang/invoke/LambdaFormBuffer.java#l234 > http://hg.openjdk.java.net/jdk/jdk/file/f6583d7cff56/src/java.base/share/classes/java/lang/invoke/LambdaFormBuffer.java#l293 > > would benefit from these instructions. Right. > > Please, let me know if you have any other comment. No more comments. Thanks, Vladimir > > Thanks, > Bernard > >> Thanks, >> Vladimir >> >> On 11/7/18 10:10 AM, B. Blaser wrote: >>> Hi, >>> >>> Please review the following small fix for [1] briefly discussed in thread [2]: >>> >>> http://cr.openjdk.java.net/~bsrbnd/jdk8213479/webrev.00/ >>> >>> It adds missing patterns to x86_64.ad for 8-bit logical operators with >>> destination in memory. >>> For example, the following method would use 4 instructions and 2 >>> registers instead of currently 10 instructions and 5 registers, see >>> [1]: >>> >>> private static byte and = 0b0011, or = 0b0011, xor = 0b0011; >>> private static byte mask = 0b0101; >>> >>> public static void test() { >>> and &= mask; >>> >>> or |= mask; >>> >>> xor ^= mask; >>> } >>> >>> Tier1 is OK (hotspot+jdk+langtools). >>> >>> Any feedback is welcome, >>> Bernard >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8213479 >>> [2] http://mail.openjdk.java.net/pipermail/jdk-dev/2018-November/002181.html >>> From bsrbnd at gmail.com Wed Nov 7 20:00:26 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 7 Nov 2018 21:00:26 +0100 Subject: RFR (S): JDK-8213479: Missing x86_64.ad patterns for 8-bit logical operators with destination in memory In-Reply-To: <7a67ec0b-b9d1-303b-af71-b50b6b485aa0@oracle.com> References: <7a67ec0b-b9d1-303b-af71-b50b6b485aa0@oracle.com> Message-ID: On Wed, 7 Nov 2018 at 20:46, Vladimir Kozlov wrote: > > On 11/7/18 11:33 AM, B. Blaser wrote: > > > > Please, let me know if you have any other comment. > No more comments. Thanks for the review Vladimir, I'll push it soon. Bernard > Thanks, > Vladimir From ioi.lam at oracle.com Wed Nov 7 22:34:34 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 7 Nov 2018 14:34:34 -0800 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <11b40c83-f15b-d1f0-2268-d6d05275f89e@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> <47520602-3945-9691-9c3b-217153a362a6@oracle.com> <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> <11b40c83-f15b-d1f0-2268-d6d05275f89e@oracle.com> Message-ID: Hi Jiangli, To address your concern about changing the verifier, I have updated the patch to remove the change in verifier.cpp. Instead, I just moved the finalize_verification_constraints() call slightly ahead (to immediately before we enter the VM thread). http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v03/ Note that I have to use a different way of iterating over all classes in SystemDictionaryShared::finalize_verification_constraints_for, because at this point we haven't combined all the shared classes into a single directory yet. Thanks -Ioi On 11/4/2018 2:29 PM, Ioi Lam wrote: > > > On 11/4/18 12:12 PM, Jiangli Zhou wrote: >> On 11/3/18 3:07 PM, Ioi Lam wrote: >> >>> >>> >>> On 11/2/18 10:42 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> >>>> On 11/2/18 5:27 PM, Ioi Lam wrote: >>>>> Hi Jiangli, >>>>> >>>>> Thanks for the review >>>>> >>>>> >>>>> On 11/2/18 1:24 PM, Jiangli Zhou wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> - src/hotspot/share/classfile/verifier.cpp >>>>>> >>>>>> You moved >>>>>> SystemDictionaryShared::finalize_verification_constraints(_klass) >>>>>> to ClassVerifier::verify_class(). We should be very careful in >>>>>> this area. Have you tested with cases where classes fail >>>>>> verification? >>>>> >>>>> void ClassVerifier::verify_class(TRAPS) { >>>>> ? log_info(verification)("Verifying class %s with new format", >>>>> _klass->external_name()); >>>>> >>>>> ? Array* methods = _klass->methods(); >>>>> ? int num_methods = methods->length(); >>>>> >>>>> ? for (int index = 0; index < num_methods; index++) { >>>>> ??? // Check for recursive re-verification before each method. >>>>> ??? if (was_recursively_verified())? return; >>>>> >>>>> ??? Method* m = methods->at(index); >>>>> ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { >>>>> ????? // If m is native or abstract, skip it.? It is checked in >>>>> class file >>>>> ????? // parser that methods do not override a final method. >>>>> Overpass methods >>>>> ????? // are trusted since the VM generates them. >>>>> ????? continue; >>>>> ??? } >>>>> ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); >>>>> ? } >>>>> >>>>> ? if (DumpSharedSpaces) { >>>>> SystemDictionaryShared::finalize_verification_constraints(_klass); >>>>> ? } >>>>> >>>>> If verification failed, the CHECK_VERIFY macro will force the >>>>> function to return. So we will call >>>>> finalize_verification_constraints only when verification has >>>>> succeeded. >>>> >>>> Ok, thanks for checking. >>>> >>>> An alternative is to allocate from the RO space directly after >>>> relocation. That way you can avoid adding another CDS specific in >>>> the verifier code. There is also a FIXME for using the symbol >>>> offset in SharedDictionaryEntry::finalize_verification_constraints, >>>> which you could handle together. >>> >>> I'll leave this patch simple so it can easily be backported to JDK 11. >>> >>> I'll do what you suggested and deal with the FIXME in a separate RFE >>> (https://bugs.openjdk.java.net/browse/JDK-8213346). >> >> Follow up with a separate RFE sounds good to me. For backporting to >> JDK 11, the risk of the fix seems to be higher than the benefit >> (considering the nature of the issue). Maybe we should have the >> proper fix backported instead (if backporting is required)? >> > > With JDK 11, if you have more than a few thousand classes, > -Xshare:dump will randomly fail. The workaround is to explicitly set > -XX:MetaspaceSize=128m. > > I think we should fix it in JDK 11. The fix in the current RFR is > pretty harmless. It just moves the allocation of the > _verifier_constraints earlier. > > Thanks > - Ioi > > >>> >>>>> >>>>> >>>>>> I haven't ran your code yet, we should avoid copying the >>>>>> constraints for classes that fails verification at dump time. >>>>> >>>>> If a class has failed verification, the class will not be >>>>> archived. So whether we have called >>>>> finalize_verification_constraints on it really doesn't matter. >>>> >>>> Since it doesn't do the copying when verification fails, it is okay >>>> in this case. With the work for dynamic archiving, we want to avoid >>>> unnecessary work at dump time, since performance/footprint will >>>> become higher priority with archiving at runtime. >>>>> >>>>>> I also need to think through any potential impact on dynamic >>>>>> archiving (possibly none). >>>>>> >>>>>> - src/hotspot/share/memory/metaspace.cpp >>>>>> >>>>>> -??? if (is_init_completed() && !(DumpSharedSpaces && >>>>>> THREAD->is_VM_thread())) { >>>>>> +??? if (is_init_completed()) { >>>>>> >>>>>> Please also add an assert to make sure we don't allocate >>>>>> metaspace objects from the VM_thread at dump time now. >>>>>> >>>>> >>>>> I think it's better to add a blanket >>>>> assert(!THREAD->is_VM_thread()), because no one should be >>>>> allocating metaspace objects inside VM thread. >>>>> >>>>> Adding this has a wider impact, so I will do it in a separate RFE >>>>> (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we >>>>> need to back it out. >>>> >>>> I'd suggest including the assert for dump time with the fix so we >>>> are sure it is safe. Then you can follow up with the broader assert >>>> that you are planning for JDK-8213331. >>>> >>> >>> I see. I've added >>> >>> MetaWord* Metaspace::allocate(ClassLoaderData* loader_data, size_t >>> word_size, >>> ????????????????????????????? MetaspaceObj::Type type, TRAPS) { >>> ? assert(!_frozen, "sanity"); >>> + assert(!(DumpSharedSpaces && THREAD->is_VM_thread()), "sanity"); >>> >>> I'll just remove the (DumpSharedSpaces &&) in JDK-8213331. >> >> Thumbs up. >> >> Thanks, >> Jiangli >> >>> >>> Thanks >>> - Ioi >>> >>>> Thanks, >>>> Jiangli >>>>> >>>>> Thanks >>>>> - Ioi >>>>>> Thanks, >>>>>> >>>>>> Jiangli >>>>>> >>>>>> >>>>>> On 11/1/18 9:47 PM, Ioi Lam wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I've revised the webrev to get to the the root of the problem -- >>>>>>> we shouldn't allocate metaspace objects from within the VM Thread! >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>>>>>> >>>>>>> >>>>>>> The original bug that caused the "!(DumpSharedSpaces && >>>>>>> THREAD->is_VM_thread())" check to be added in >>>>>>> Metaspace::allocate() was >>>>>>> >>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >>>>>>> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock >>>>>>> (not allowed to block on)" >>>>>>> >>>>>>> The fix in JDK-8196626 is just a band-aid solution -- avoiding >>>>>>> the GC. The proper fix is not to do any allocation inside the VM >>>>>>> thread. So instead of calling finalize_verification_constraints, >>>>>>> which allocates MetaspaceObjs, very late in the CDS dumping >>>>>>> process, now we call be as soon as a class has been verified. >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>>>>>> >>>>>>>> >>>>>>>> When Metaspace::allocate() fails, usually we first try to GC to >>>>>>>> see if >>>>>>>> some MetaspaceObjs can be freed. This is an optimization to >>>>>>>> avoid prematurely >>>>>>>> expanding the metaspace. >>>>>>>> >>>>>>>> However, GC cannot run in the last phase of CDS which runs >>>>>>>> inside the VM thread. >>>>>>>> The old code just aborts the CDS dump. The fix is to >>>>>>>> unconditionally expand the >>>>>>>> metaspace without doing a GC. >>>>>>>> >>>>>>>> (During GC dump time, no classes are unloaded, so even if we >>>>>>>> force a GC, it would >>>>>>>> not free up any metaspace anyway). >>>>>>>> >>>>>>>> Testing: >>>>>>>> >>>>>>>> Before the fix, the test case (jruby) would fail to dump once a >>>>>>>> couple of times. Now >>>>>>>> it has dumped hundreds of times without failure. >>>>>>>> >>>>>>>> I am also running hs-tier 1/2 >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Wed Nov 7 22:39:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 8 Nov 2018 08:39:22 +1000 Subject: RFR[XS]: JDK-8213414 Fix incorrect copy constructors in hotspot In-Reply-To: <3049827A-4C7D-41E7-BE66-2C513C6F0C4D@oracle.com> References: <3049827A-4C7D-41E7-BE66-2C513C6F0C4D@oracle.com> Message-ID: <249e717b-699c-2500-dd39-84967160965c@oracle.com> FTR: This trivial fix caused test carnage and was backed out under JDK-8213487. David On 7/11/2018 4:28 AM, Kim Barrett wrote: >> On Nov 6, 2018, at 6:55 AM, Kim Barrett wrote: >> >>> On Nov 6, 2018, at 6:16 AM, Magnus Ihse Bursie wrote: >>> >>> In JDK-8211073, an attempt was made to add -Wextra to hotspot. That patch was not accepted at the time being, but it did dig out some bugs in the code. >>> >>> While we await JDK-8211073 to be included in the code base (so -Wextra is always on), we can at least fix the bugs it found in some copy constructors. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213414 >>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8213414-fix-hotspot-copy-constructors/webrev.01 >>> >>> /Magnus >> >> Looks good. > > ? and trivial. > From jiangli.zhou at oracle.com Thu Nov 8 01:46:39 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 7 Nov 2018 17:46:39 -0800 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> <47520602-3945-9691-9c3b-217153a362a6@oracle.com> <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> <11b40c83-f15b-d1f0-2268-d6d05275f89e@oracle.com> Message-ID: <8ba1133b-30dc-5594-e081-8da1cb2d5fcc@oracle.com> Hi Ioi, This approach looks cleaner to me. Thanks! Jiangli On 11/7/18 2:34 PM, Ioi Lam wrote: > Hi Jiangli, > > To address your concern about changing the verifier, I have updated > the patch to remove the change in verifier.cpp. Instead, I just moved > the finalize_verification_constraints() call slightly ahead (to > immediately before we enter the VM thread). > > http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v03/ > > > Note that I have to use a different way of iterating over all classes > in SystemDictionaryShared::finalize_verification_constraints_for, > because at this point we haven't combined all the shared classes into > a single directory yet. > > Thanks > > -Ioi > > > > On 11/4/2018 2:29 PM, Ioi Lam wrote: >> >> >> On 11/4/18 12:12 PM, Jiangli Zhou wrote: >>> On 11/3/18 3:07 PM, Ioi Lam wrote: >>> >>>> >>>> >>>> On 11/2/18 10:42 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> >>>>> On 11/2/18 5:27 PM, Ioi Lam wrote: >>>>>> Hi Jiangli, >>>>>> >>>>>> Thanks for the review >>>>>> >>>>>> >>>>>> On 11/2/18 1:24 PM, Jiangli Zhou wrote: >>>>>>> Hi Ioi, >>>>>>> >>>>>>> - src/hotspot/share/classfile/verifier.cpp >>>>>>> >>>>>>> You moved >>>>>>> SystemDictionaryShared::finalize_verification_constraints(_klass) >>>>>>> to ClassVerifier::verify_class(). We should be very careful in >>>>>>> this area. Have you tested with cases where classes fail >>>>>>> verification? >>>>>> >>>>>> void ClassVerifier::verify_class(TRAPS) { >>>>>> ? log_info(verification)("Verifying class %s with new format", >>>>>> _klass->external_name()); >>>>>> >>>>>> ? Array* methods = _klass->methods(); >>>>>> ? int num_methods = methods->length(); >>>>>> >>>>>> ? for (int index = 0; index < num_methods; index++) { >>>>>> ??? // Check for recursive re-verification before each method. >>>>>> ??? if (was_recursively_verified())? return; >>>>>> >>>>>> ??? Method* m = methods->at(index); >>>>>> ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { >>>>>> ????? // If m is native or abstract, skip it.? It is checked in >>>>>> class file >>>>>> ????? // parser that methods do not override a final method. >>>>>> Overpass methods >>>>>> ????? // are trusted since the VM generates them. >>>>>> ????? continue; >>>>>> ??? } >>>>>> ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); >>>>>> ? } >>>>>> >>>>>> ? if (DumpSharedSpaces) { >>>>>> SystemDictionaryShared::finalize_verification_constraints(_klass); >>>>>> ? } >>>>>> >>>>>> If verification failed, the CHECK_VERIFY macro will force the >>>>>> function to return. So we will call >>>>>> finalize_verification_constraints only when verification has >>>>>> succeeded. >>>>> >>>>> Ok, thanks for checking. >>>>> >>>>> An alternative is to allocate from the RO space directly after >>>>> relocation. That way you can avoid adding another CDS specific in >>>>> the verifier code. There is also a FIXME for using the symbol >>>>> offset in >>>>> SharedDictionaryEntry::finalize_verification_constraints, which >>>>> you could handle together. >>>> >>>> I'll leave this patch simple so it can easily be backported to JDK 11. >>>> >>>> I'll do what you suggested and deal with the FIXME in a separate >>>> RFE (https://bugs.openjdk.java.net/browse/JDK-8213346). >>> >>> Follow up with a separate RFE sounds good to me. For backporting to >>> JDK 11, the risk of the fix seems to be higher than the benefit >>> (considering the nature of the issue). Maybe we should have the >>> proper fix backported instead (if backporting is required)? >>> >> >> With JDK 11, if you have more than a few thousand classes, >> -Xshare:dump will randomly fail. The workaround is to explicitly set >> -XX:MetaspaceSize=128m. >> >> I think we should fix it in JDK 11. The fix in the current RFR is >> pretty harmless. It just moves the allocation of the >> _verifier_constraints earlier. >> >> Thanks >> - Ioi >> >> >>>> >>>>>> >>>>>> >>>>>>> I haven't ran your code yet, we should avoid copying the >>>>>>> constraints for classes that fails verification at dump time. >>>>>> >>>>>> If a class has failed verification, the class will not be >>>>>> archived. So whether we have called >>>>>> finalize_verification_constraints on it really doesn't matter. >>>>> >>>>> Since it doesn't do the copying when verification fails, it is >>>>> okay in this case. With the work for dynamic archiving, we want to >>>>> avoid unnecessary work at dump time, since performance/footprint >>>>> will become higher priority with archiving at runtime. >>>>>> >>>>>>> I also need to think through any potential impact on dynamic >>>>>>> archiving (possibly none). >>>>>>> >>>>>>> - src/hotspot/share/memory/metaspace.cpp >>>>>>> >>>>>>> -??? if (is_init_completed() && !(DumpSharedSpaces && >>>>>>> THREAD->is_VM_thread())) { >>>>>>> +??? if (is_init_completed()) { >>>>>>> >>>>>>> Please also add an assert to make sure we don't allocate >>>>>>> metaspace objects from the VM_thread at dump time now. >>>>>>> >>>>>> >>>>>> I think it's better to add a blanket >>>>>> assert(!THREAD->is_VM_thread()), because no one should be >>>>>> allocating metaspace objects inside VM thread. >>>>>> >>>>>> Adding this has a wider impact, so I will do it in a separate RFE >>>>>> (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we >>>>>> need to back it out. >>>>> >>>>> I'd suggest including the assert for dump time with the fix so we >>>>> are sure it is safe. Then you can follow up with the broader >>>>> assert that you are planning for JDK-8213331. >>>>> >>>> >>>> I see. I've added >>>> >>>> MetaWord* Metaspace::allocate(ClassLoaderData* loader_data, size_t >>>> word_size, >>>> ????????????????????????????? MetaspaceObj::Type type, TRAPS) { >>>> ? assert(!_frozen, "sanity"); >>>> + assert(!(DumpSharedSpaces && THREAD->is_VM_thread()), "sanity"); >>>> >>>> I'll just remove the (DumpSharedSpaces &&) in JDK-8213331. >>> >>> Thumbs up. >>> >>> Thanks, >>> Jiangli >>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Thanks, >>>>> Jiangli >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>>> Thanks, >>>>>>> >>>>>>> Jiangli >>>>>>> >>>>>>> >>>>>>> On 11/1/18 9:47 PM, Ioi Lam wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I've revised the webrev to get to the the root of the problem >>>>>>>> -- we shouldn't allocate metaspace objects from within the VM >>>>>>>> Thread! >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>>>>>>> >>>>>>>> >>>>>>>> The original bug that caused the "!(DumpSharedSpaces && >>>>>>>> THREAD->is_VM_thread())" check to be added in >>>>>>>> Metaspace::allocate() was >>>>>>>> >>>>>>>> ??? https://bugs.openjdk.java.net/browse/JDK-8196626 >>>>>>>> ??? UseAppCDS.java crashes with "VM thread using lock Heap_lock >>>>>>>> (not allowed to block on)" >>>>>>>> >>>>>>>> The fix in JDK-8196626 is just a band-aid solution -- avoiding >>>>>>>> the GC. The proper fix is not to do any allocation inside the >>>>>>>> VM thread. So instead of calling >>>>>>>> finalize_verification_constraints, which allocates >>>>>>>> MetaspaceObjs, very late in the CDS dumping process, now we >>>>>>>> call be as soon as a class has been verified. >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> When Metaspace::allocate() fails, usually we first try to GC >>>>>>>>> to see if >>>>>>>>> some MetaspaceObjs can be freed. This is an optimization to >>>>>>>>> avoid prematurely >>>>>>>>> expanding the metaspace. >>>>>>>>> >>>>>>>>> However, GC cannot run in the last phase of CDS which runs >>>>>>>>> inside the VM thread. >>>>>>>>> The old code just aborts the CDS dump. The fix is to >>>>>>>>> unconditionally expand the >>>>>>>>> metaspace without doing a GC. >>>>>>>>> >>>>>>>>> (During GC dump time, no classes are unloaded, so even if we >>>>>>>>> force a GC, it would >>>>>>>>> not free up any metaspace anyway). >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> >>>>>>>>> Before the fix, the test case (jruby) would fail to dump once >>>>>>>>> a couple of times. Now >>>>>>>>> it has dumped hundreds of times without failure. >>>>>>>>> >>>>>>>>> I am also running hs-tier 1/2 >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From vicente.romero at oracle.com Thu Nov 8 02:26:20 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Wed, 7 Nov 2018 21:26:20 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> Message-ID: <923e07c1-feba-1e96-e138-67e44d8790d7@oracle.com> Hi Igor, On 11/7/18 2:33 PM, Igor Ignatyev wrote: > Hi Vicente, > > I recall an (internal?) discussion about updating ASM somewhen in JDK > 11TF, and AFAIR it was decided not to update ASM b/c nothing in JDK > needs that, has it been changed? put somewhat differently, why are we > doing this? we need the support ASM 7 has of condy, many amber projects depend on this > > in any case, I don't like the changes in mlvm tests. I understand > that?ClassWriter has been significantly changed in ASM 7.0, > so?ClassWriterExt can't disable CP entries caching (at least not in > the way it used to), but removing setCache* calls from the tests > changed them and in some cases made them invalid as they don't test > that they supposed to. therefore I'd prefer to leave all calls > setCache* as-is, change setCache* implementation to throw an exception > (similarly to the fix in JDK-8194826 > )?and mark all tests > which throw this exception w/ '@ignore?8194951' jtreg tag. sounds good, I will do this > > Thanks, > -- Igor Thanks, Vicente > >> On Nov 7, 2018, at 7:56 AM, Vicente Romero > > wrote: >> >> Hi, >> >> Version 7.0 of ASM has been released. This version supports condy, >> yay!, and we want to include it in JDK 12. Please review [1] which >> includes: >> - the new version perse substituting the preview ASM internal version >> in the JDK >> - changes to additional files in particular some tests, mostly >> hotspot tests. >> >> Thanks, >> Vicente >> >> [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ > From david.holmes at oracle.com Thu Nov 8 03:03:40 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 8 Nov 2018 13:03:40 +1000 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> Message-ID: <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> Hi Vicente, All of the javadoc comment reformatting makes it nearly impossible to see the actual substantive changes :( ASM 7 also supports the Nestmate attributes and I was trying to see how/where that appeared but its somewhat obscure. Oh well. Is it that case that the code the uses the ASM library, like the JFR code and jlink code, and the tests, doesn't actually _have to_ change to specifying Opcodes.ASM7 unless they plan on using ASM7 features? If so then you could split out the actual update of ASM from the updates to the users of ASM (some of which may be quite fine with ASM5). Thanks, David On 8/11/2018 1:56 AM, Vicente Romero wrote: > Hi, > > Version 7.0 of ASM has been released. This version supports condy, yay!, > and we want to include it in JDK 12. Please review [1] which includes: > - the new version perse substituting the preview ASM internal version in > the JDK > - changes to additional files in particular some tests, mostly hotspot > tests. > > Thanks, > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ From ioi.lam at oracle.com Thu Nov 8 04:10:47 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 7 Nov 2018 20:10:47 -0800 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: <8ba1133b-30dc-5594-e081-8da1cb2d5fcc@oracle.com> References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> <47520602-3945-9691-9c3b-217153a362a6@oracle.com> <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> <11b40c83-f15b-d1f0-2268-d6d05275f89e@oracle.com> <8ba1133b-30dc-5594-e081-8da1cb2d5fcc@oracle.com> Message-ID: <98c59f60-a8da-c11b-4a8f-740900441540@oracle.com> Hi Jiangli, Thanks for the review. I will do more testing and push. - Ioi On 11/7/18 5:46 PM, Jiangli Zhou wrote: > Hi Ioi, > > This approach looks cleaner to me. > > Thanks! > > Jiangli > > > On 11/7/18 2:34 PM, Ioi Lam wrote: >> Hi Jiangli, >> >> To address your concern about changing the verifier, I have updated >> the patch to remove the change in verifier.cpp. Instead, I just moved >> the finalize_verification_constraints() call slightly ahead (to >> immediately before we enter the VM thread). >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v03/ >> >> >> Note that I have to use a different way of iterating over all classes >> in SystemDictionaryShared::finalize_verification_constraints_for, >> because at this point we haven't combined all the shared classes into >> a single directory yet. >> >> Thanks >> >> -Ioi >> >> >> >> On 11/4/2018 2:29 PM, Ioi Lam wrote: >>> >>> >>> On 11/4/18 12:12 PM, Jiangli Zhou wrote: >>>> On 11/3/18 3:07 PM, Ioi Lam wrote: >>>> >>>>> >>>>> >>>>> On 11/2/18 10:42 PM, Jiangli Zhou wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> >>>>>> On 11/2/18 5:27 PM, Ioi Lam wrote: >>>>>>> Hi Jiangli, >>>>>>> >>>>>>> Thanks for the review >>>>>>> >>>>>>> >>>>>>> On 11/2/18 1:24 PM, Jiangli Zhou wrote: >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> - src/hotspot/share/classfile/verifier.cpp >>>>>>>> >>>>>>>> You moved >>>>>>>> SystemDictionaryShared::finalize_verification_constraints(_klass) >>>>>>>> to ClassVerifier::verify_class(). We should be very careful in >>>>>>>> this area. Have you tested with cases where classes fail >>>>>>>> verification? >>>>>>> >>>>>>> void ClassVerifier::verify_class(TRAPS) { >>>>>>> ? log_info(verification)("Verifying class %s with new format", >>>>>>> _klass->external_name()); >>>>>>> >>>>>>> ? Array* methods = _klass->methods(); >>>>>>> ? int num_methods = methods->length(); >>>>>>> >>>>>>> ? for (int index = 0; index < num_methods; index++) { >>>>>>> ??? // Check for recursive re-verification before each method. >>>>>>> ??? if (was_recursively_verified())? return; >>>>>>> >>>>>>> ??? Method* m = methods->at(index); >>>>>>> ??? if (m->is_native() || m->is_abstract() || m->is_overpass()) { >>>>>>> ????? // If m is native or abstract, skip it.? It is checked in >>>>>>> class file >>>>>>> ????? // parser that methods do not override a final method. >>>>>>> Overpass methods >>>>>>> ????? // are trusted since the VM generates them. >>>>>>> ????? continue; >>>>>>> ??? } >>>>>>> ??? verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); >>>>>>> ? } >>>>>>> >>>>>>> ? if (DumpSharedSpaces) { >>>>>>> SystemDictionaryShared::finalize_verification_constraints(_klass); >>>>>>> ? } >>>>>>> >>>>>>> If verification failed, the CHECK_VERIFY macro will force the >>>>>>> function to return. So we will call >>>>>>> finalize_verification_constraints only when verification has >>>>>>> succeeded. >>>>>> >>>>>> Ok, thanks for checking. >>>>>> >>>>>> An alternative is to allocate from the RO space directly after >>>>>> relocation. That way you can avoid adding another CDS specific in >>>>>> the verifier code. There is also a FIXME for using the symbol >>>>>> offset in >>>>>> SharedDictionaryEntry::finalize_verification_constraints, which >>>>>> you could handle together. >>>>> >>>>> I'll leave this patch simple so it can easily be backported to JDK >>>>> 11. >>>>> >>>>> I'll do what you suggested and deal with the FIXME in a separate >>>>> RFE (https://bugs.openjdk.java.net/browse/JDK-8213346). >>>> >>>> Follow up with a separate RFE sounds good to me. For backporting to >>>> JDK 11, the risk of the fix seems to be higher than the benefit >>>> (considering the nature of the issue). Maybe we should have the >>>> proper fix backported instead (if backporting is required)? >>>> >>> >>> With JDK 11, if you have more than a few thousand classes, >>> -Xshare:dump will randomly fail. The workaround is to explicitly set >>> -XX:MetaspaceSize=128m. >>> >>> I think we should fix it in JDK 11. The fix in the current RFR is >>> pretty harmless. It just moves the allocation of the >>> _verifier_constraints earlier. >>> >>> Thanks >>> - Ioi >>> >>> >>>>> >>>>>>> >>>>>>> >>>>>>>> I haven't ran your code yet, we should avoid copying the >>>>>>>> constraints for classes that fails verification at dump time. >>>>>>> >>>>>>> If a class has failed verification, the class will not be >>>>>>> archived. So whether we have called >>>>>>> finalize_verification_constraints on it really doesn't matter. >>>>>> >>>>>> Since it doesn't do the copying when verification fails, it is >>>>>> okay in this case. With the work for dynamic archiving, we want >>>>>> to avoid unnecessary work at dump time, since >>>>>> performance/footprint will become higher priority with archiving >>>>>> at runtime. >>>>>>> >>>>>>>> I also need to think through any potential impact on dynamic >>>>>>>> archiving (possibly none). >>>>>>>> >>>>>>>> - src/hotspot/share/memory/metaspace.cpp >>>>>>>> >>>>>>>> -??? if (is_init_completed() && !(DumpSharedSpaces && >>>>>>>> THREAD->is_VM_thread())) { >>>>>>>> +??? if (is_init_completed()) { >>>>>>>> >>>>>>>> Please also add an assert to make sure we don't allocate >>>>>>>> metaspace objects from the VM_thread at dump time now. >>>>>>>> >>>>>>> >>>>>>> I think it's better to add a blanket >>>>>>> assert(!THREAD->is_VM_thread()), because no one should be >>>>>>> allocating metaspace objects inside VM thread. >>>>>>> >>>>>>> Adding this has a wider impact, so I will do it in a separate >>>>>>> RFE (https://bugs.openjdk.java.net/browse/JDK-8213331), in case >>>>>>> we need to back it out. >>>>>> >>>>>> I'd suggest including the assert for dump time with the fix so we >>>>>> are sure it is safe. Then you can follow up with the broader >>>>>> assert that you are planning for JDK-8213331. >>>>>> >>>>> >>>>> I see. I've added >>>>> >>>>> MetaWord* Metaspace::allocate(ClassLoaderData* loader_data, size_t >>>>> word_size, >>>>> ????????????????????????????? MetaspaceObj::Type type, TRAPS) { >>>>> ? assert(!_frozen, "sanity"); >>>>> + assert(!(DumpSharedSpaces && THREAD->is_VM_thread()), "sanity"); >>>>> >>>>> I'll just remove the (DumpSharedSpaces &&) in JDK-8213331. >>>> >>>> Thumbs up. >>>> >>>> Thanks, >>>> Jiangli >>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Jiangli >>>>>>>> >>>>>>>> >>>>>>>> On 11/1/18 9:47 PM, Ioi Lam wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I've revised the webrev to get to the the root of the problem >>>>>>>>> -- we shouldn't allocate metaspace objects from within the VM >>>>>>>>> Thread! >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>>>>>>>> >>>>>>>>> >>>>>>>>> The original bug that caused the "!(DumpSharedSpaces && >>>>>>>>> THREAD->is_VM_thread())" check to be added in >>>>>>>>> Metaspace::allocate() was >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196626 >>>>>>>>> ??? UseAppCDS.java crashes with "VM thread using lock >>>>>>>>> Heap_lock (not allowed to block on)" >>>>>>>>> >>>>>>>>> The fix in JDK-8196626 is just a band-aid solution -- avoiding >>>>>>>>> the GC. The proper fix is not to do any allocation inside the >>>>>>>>> VM thread. So instead of calling >>>>>>>>> finalize_verification_constraints, which allocates >>>>>>>>> MetaspaceObjs, very late in the CDS dumping process, now we >>>>>>>>> call be as soon as a class has been verified. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> When Metaspace::allocate() fails, usually we first try to GC >>>>>>>>>> to see if >>>>>>>>>> some MetaspaceObjs can be freed. This is an optimization to >>>>>>>>>> avoid prematurely >>>>>>>>>> expanding the metaspace. >>>>>>>>>> >>>>>>>>>> However, GC cannot run in the last phase of CDS which runs >>>>>>>>>> inside the VM thread. >>>>>>>>>> The old code just aborts the CDS dump. The fix is to >>>>>>>>>> unconditionally expand the >>>>>>>>>> metaspace without doing a GC. >>>>>>>>>> >>>>>>>>>> (During GC dump time, no classes are unloaded, so even if we >>>>>>>>>> force a GC, it would >>>>>>>>>> not free up any metaspace anyway). >>>>>>>>>> >>>>>>>>>> Testing: >>>>>>>>>> >>>>>>>>>> Before the fix, the test case (jruby) would fail to dump once >>>>>>>>>> a couple of times. Now >>>>>>>>>> it has dumped hundreds of times without failure. >>>>>>>>>> >>>>>>>>>> I am also running hs-tier 1/2 >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From calvin.cheung at oracle.com Thu Nov 8 06:31:33 2018 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 07 Nov 2018 22:31:33 -0800 Subject: RFR(XS) 8213250 CDS archive creation aborts due to metaspace object allocation failure In-Reply-To: References: <6764fc88-1e1f-70ec-4355-642e62d8f662@oracle.com> <54db6e08-c870-47be-aef6-9704c8b25bdc@oracle.com> <51eaacff-a6b7-9507-2026-c65e1e16b9c5@oracle.com> <47520602-3945-9691-9c3b-217153a362a6@oracle.com> <4aebcc7b-643d-35c9-3457-575e080147d8@oracle.com> <11b40c83-f15b-d1f0-2268-d6d05275f89e@oracle.com> Message-ID: <5BE3D845.5040503@oracle.com> Looks good. thanks, Calvin On 11/7/18, 2:34 PM, Ioi Lam wrote: > Hi Jiangli, > > To address your concern about changing the verifier, I have updated > the patch to remove the change in verifier.cpp. Instead, I just moved > the finalize_verification_constraints() call slightly ahead (to > immediately before we enter the VM thread). > > http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v03/ > > > Note that I have to use a different way of iterating over all classes > in SystemDictionaryShared::finalize_verification_constraints_for, > because at this point we haven't combined all the shared classes into > a single directory yet. > > Thanks > > -Ioi > > > > On 11/4/2018 2:29 PM, Ioi Lam wrote: >> >> >> On 11/4/18 12:12 PM, Jiangli Zhou wrote: >>> On 11/3/18 3:07 PM, Ioi Lam wrote: >>> >>>> >>>> >>>> On 11/2/18 10:42 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> >>>>> On 11/2/18 5:27 PM, Ioi Lam wrote: >>>>>> Hi Jiangli, >>>>>> >>>>>> Thanks for the review >>>>>> >>>>>> >>>>>> On 11/2/18 1:24 PM, Jiangli Zhou wrote: >>>>>>> Hi Ioi, >>>>>>> >>>>>>> - src/hotspot/share/classfile/verifier.cpp >>>>>>> >>>>>>> You moved >>>>>>> SystemDictionaryShared::finalize_verification_constraints(_klass) to >>>>>>> ClassVerifier::verify_class(). We should be very careful in this >>>>>>> area. Have you tested with cases where classes fail verification? >>>>>> >>>>>> void ClassVerifier::verify_class(TRAPS) { >>>>>> log_info(verification)("Verifying class %s with new format", >>>>>> _klass->external_name()); >>>>>> >>>>>> Array* methods = _klass->methods(); >>>>>> int num_methods = methods->length(); >>>>>> >>>>>> for (int index = 0; index < num_methods; index++) { >>>>>> // Check for recursive re-verification before each method. >>>>>> if (was_recursively_verified()) return; >>>>>> >>>>>> Method* m = methods->at(index); >>>>>> if (m->is_native() || m->is_abstract() || m->is_overpass()) { >>>>>> // If m is native or abstract, skip it. It is checked in >>>>>> class file >>>>>> // parser that methods do not override a final method. >>>>>> Overpass methods >>>>>> // are trusted since the VM generates them. >>>>>> continue; >>>>>> } >>>>>> verify_method(methodHandle(THREAD, m), CHECK_VERIFY(this)); >>>>>> } >>>>>> >>>>>> if (DumpSharedSpaces) { >>>>>> SystemDictionaryShared::finalize_verification_constraints(_klass); >>>>>> } >>>>>> >>>>>> If verification failed, the CHECK_VERIFY macro will force the >>>>>> function to return. So we will call >>>>>> finalize_verification_constraints only when verification has >>>>>> succeeded. >>>>> >>>>> Ok, thanks for checking. >>>>> >>>>> An alternative is to allocate from the RO space directly after >>>>> relocation. That way you can avoid adding another CDS specific in >>>>> the verifier code. There is also a FIXME for using the symbol >>>>> offset in >>>>> SharedDictionaryEntry::finalize_verification_constraints, which >>>>> you could handle together. >>>> >>>> I'll leave this patch simple so it can easily be backported to JDK 11. >>>> >>>> I'll do what you suggested and deal with the FIXME in a separate >>>> RFE (https://bugs.openjdk.java.net/browse/JDK-8213346). >>> >>> Follow up with a separate RFE sounds good to me. For backporting to >>> JDK 11, the risk of the fix seems to be higher than the benefit >>> (considering the nature of the issue). Maybe we should have the >>> proper fix backported instead (if backporting is required)? >>> >> >> With JDK 11, if you have more than a few thousand classes, >> -Xshare:dump will randomly fail. The workaround is to explicitly set >> -XX:MetaspaceSize=128m. >> >> I think we should fix it in JDK 11. The fix in the current RFR is >> pretty harmless. It just moves the allocation of the >> _verifier_constraints earlier. >> >> Thanks >> - Ioi >> >> >>>> >>>>>> >>>>>> >>>>>>> I haven't ran your code yet, we should avoid copying the >>>>>>> constraints for classes that fails verification at dump time. >>>>>> >>>>>> If a class has failed verification, the class will not be >>>>>> archived. So whether we have called >>>>>> finalize_verification_constraints on it really doesn't matter. >>>>> >>>>> Since it doesn't do the copying when verification fails, it is >>>>> okay in this case. With the work for dynamic archiving, we want to >>>>> avoid unnecessary work at dump time, since performance/footprint >>>>> will become higher priority with archiving at runtime. >>>>>> >>>>>>> I also need to think through any potential impact on dynamic >>>>>>> archiving (possibly none). >>>>>>> >>>>>>> - src/hotspot/share/memory/metaspace.cpp >>>>>>> >>>>>>> - if (is_init_completed() && !(DumpSharedSpaces && >>>>>>> THREAD->is_VM_thread())) { >>>>>>> + if (is_init_completed()) { >>>>>>> >>>>>>> Please also add an assert to make sure we don't allocate >>>>>>> metaspace objects from the VM_thread at dump time now. >>>>>>> >>>>>> >>>>>> I think it's better to add a blanket >>>>>> assert(!THREAD->is_VM_thread()), because no one should be >>>>>> allocating metaspace objects inside VM thread. >>>>>> >>>>>> Adding this has a wider impact, so I will do it in a separate RFE >>>>>> (https://bugs.openjdk.java.net/browse/JDK-8213331), in case we >>>>>> need to back it out. >>>>> >>>>> I'd suggest including the assert for dump time with the fix so we >>>>> are sure it is safe. Then you can follow up with the broader >>>>> assert that you are planning for JDK-8213331. >>>>> >>>> >>>> I see. I've added >>>> >>>> MetaWord* Metaspace::allocate(ClassLoaderData* loader_data, size_t >>>> word_size, >>>> MetaspaceObj::Type type, TRAPS) { >>>> assert(!_frozen, "sanity"); >>>> + assert(!(DumpSharedSpaces && THREAD->is_VM_thread()), "sanity"); >>>> >>>> I'll just remove the (DumpSharedSpaces &&) in JDK-8213331. >>> >>> Thumbs up. >>> >>> Thanks, >>> Jiangli >>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Thanks, >>>>> Jiangli >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>>> Thanks, >>>>>>> >>>>>>> Jiangli >>>>>>> >>>>>>> >>>>>>> On 11/1/18 9:47 PM, Ioi Lam wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I've revised the webrev to get to the the root of the problem >>>>>>>> -- we shouldn't allocate metaspace objects from within the VM >>>>>>>> Thread! >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v02/ >>>>>>>> >>>>>>>> >>>>>>>> The original bug that caused the "!(DumpSharedSpaces && >>>>>>>> THREAD->is_VM_thread())" check to be added in >>>>>>>> Metaspace::allocate() was >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196626 >>>>>>>> UseAppCDS.java crashes with "VM thread using lock Heap_lock >>>>>>>> (not allowed to block on)" >>>>>>>> >>>>>>>> The fix in JDK-8196626 is just a band-aid solution -- avoiding >>>>>>>> the GC. The proper fix is not to do any allocation inside the >>>>>>>> VM thread. So instead of calling >>>>>>>> finalize_verification_constraints, which allocates >>>>>>>> MetaspaceObjs, very late in the CDS dumping process, now we >>>>>>>> call be as soon as a class has been verified. >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 11/1/18 11:07 AM, Ioi Lam wrote: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213250 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213250-cds-dump-run-out-of-metaspace.v01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> When Metaspace::allocate() fails, usually we first try to GC >>>>>>>>> to see if >>>>>>>>> some MetaspaceObjs can be freed. This is an optimization to >>>>>>>>> avoid prematurely >>>>>>>>> expanding the metaspace. >>>>>>>>> >>>>>>>>> However, GC cannot run in the last phase of CDS which runs >>>>>>>>> inside the VM thread. >>>>>>>>> The old code just aborts the CDS dump. The fix is to >>>>>>>>> unconditionally expand the >>>>>>>>> metaspace without doing a GC. >>>>>>>>> >>>>>>>>> (During GC dump time, no classes are unloaded, so even if we >>>>>>>>> force a GC, it would >>>>>>>>> not free up any metaspace anyway). >>>>>>>>> >>>>>>>>> Testing: >>>>>>>>> >>>>>>>>> Before the fix, the test case (jruby) would fail to dump once >>>>>>>>> a couple of times. Now >>>>>>>>> it has dumped hundreds of times without failure. >>>>>>>>> >>>>>>>>> I am also running hs-tier 1/2 >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From robbin.ehn at oracle.com Thu Nov 8 09:28:48 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 8 Nov 2018 10:28:48 +0100 Subject: RFR(s): 8211403: Rename SafepointMechanism::poll(...) In-Reply-To: References: <5ca886e9-6839-9c21-10f4-22ecfc05e5a4@oracle.com> Message-ID: <3099c1c5-9266-a67b-c7e5-db4bb984d9ae@oracle.com> > Thumbs up!? Your call on whether to apply the renaming to > global_poll() and local_poll(). I understand both points > of view and I don't have a preference... Ok, thanks Dan! Updated years and comment. /Robbin > > Dan > > >> >> Sanity on tier 1. >> >> Thanks, Robbin >> > From shade at redhat.com Thu Nov 8 09:38:15 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 8 Nov 2018 10:38:15 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> Message-ID: Hi! On 11/06/2018 01:07 AM, Kim Barrett wrote: > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/bitMap.cpp > 272 if ((end_full_word < beg_full_word) || (end_full_word - beg_full_word < BitsPerWord)) { > > The use of BitsPerWord here at on line 289 seems like a units error. > What does the number of bits in a word have to do with the number of > words that is considered "small"? I suspect the old value of 32 was > just a heuristic bound (which may or may not have had any measurements > to back it up). > > Of course, the assertion error message in the old version of > set_large_range is also confused; the comparison is in words but the > message says bytes. Oh. Good catch! I thought the comparison was on bit index aligned on word boundary, and suspected 32 meant "32 bits" from the old era. On a hunch, I replaced it with the actual platform-specific constant, BitsPerWord in this case. But this is not a good thing here. Instead, we should make it cleaner by introducing "small range" constant and turning the following comment into assert: http://cr.openjdk.java.net/~shade/8211926/webrev.02/ This still passes new gtest and hotspot-tier1. Does the patch make more sense now? I'd like to keep the change small and contained, without significant rewrites, to simplify backports without introducing new bugs. If additional check is significant on some path, users can just go to non-large-hinted method versions. -Aleksey From goetz.lindenmaier at sap.com Thu Nov 8 11:35:22 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 8 Nov 2018 11:35:22 +0000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Message-ID: Hi, I also ran into this issue and designed this as fix for it: http://cr.openjdk.java.net/~goetz/wr18/8213527-32bit_tests/01/ I withdraw this part of my change :) I think handling this in VMProps is the better solution. Else every developer that implements a test with UseCompressedOops must remember about the special case of this flag. But I also think a generic solution as you propose, Boris, does not makes sense. You can not handle this generically. Imagine the flag was DontUseCompressedOops. By just setting it to false would be the wrong thing to do. Therefore I think we must special case on the very flag that causes the issue. Maybe we should add other lp64_* flags right in this change? At least UseCompressedClassPointers? Best regards, Goetz > -----Original Message----- > From: hotspot-dev On Behalf Of > Boris Ulasevich > Sent: Mittwoch, 7. November 2018 09:07 > To: David Holmes ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(XS): 8213410: UseCompressedOops @requirement check fails > fails on 32-bit system > > Hi David, > > Yes, at first glance it is weird. We have actually three states for > vm.opt.final.UseCompressedOops: true, false and null. Null means "not > applicable" - when current VM does not support the option with the given > name. Here is another approach for the issue (I am not sure it is a good > one): > http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ > > thank you, > Boris > > On 07.11.2018 1:36, David Holmes wrote: > > Hi Boris, > > > > I'm somewhat perplexed as to what the difference is between: > > > > @requires vm.opt.final.UseCompressedOops > > > > and > > > > @requires vm.opt.final.UseCompressedOops == true > > > > if they are indeed different then that seems to be a bug in the > > requirement handling in the jtreg support code. > > > > I'm also wondering whether any such requirement should always be > > proceeded by a check for 64-bits? > > > > Otherwise I would expect > > > > @requires vm.opt.final.UseCompressedOops > > > > to be false in 32-bit, so again a problem with the jtreg support code. > > > > Thanks, > > David > > > > On 7/11/2018 12:43 AM, Boris Ulasevich wrote: > >> Hi all, > >> > >> Please review this patch to fix jtreg @requires > >> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on > >> ARM32: "invalid boolean value: null for expression > >> vm.opt.final.UseCompressedOops". > >> > >> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ > >> https://bugs.openjdk.java.net/browse/JDK-8213410 > >> > >> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok by > >> default, and becomes disabled with > >> -javaoptions:"-XX:-UseCompressedOops" jtreg option). > >> > >> Thanks, > >> Boris Ulasevich From david.holmes at oracle.com Thu Nov 8 12:50:28 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 8 Nov 2018 22:50:28 +1000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Message-ID: <5d3c1cbb-cbcb-0700-36ec-412c960a3d77@oracle.com> On 8/11/2018 9:35 PM, Lindenmaier, Goetz wrote: > Hi, > > I also ran into this issue and designed this as fix for it: > http://cr.openjdk.java.net/~goetz/wr18/8213527-32bit_tests/01/ > I withdraw this part of my change :) > > I think handling this in VMProps is the better solution. > Else every developer that implements a test with UseCompressedOops > must remember about the special case of this flag. Any developer writing any test that uses VM flags has to know whether the flag is valid for all configurations/platforms etc. > But I also think a generic solution as you propose, Boris, does not > makes sense. You can not handle this generically. > Imagine the flag was DontUseCompressedOops. By just setting > it to false would be the wrong thing to do. > Therefore I think we must special case on the very flag that causes > the issue. Based on what you are testing you have to understand the role of the flag. I agree that a test that requires a flag to be disabled probably does not do as intended if the flag is not present at all. But the test writer has to understand that. Just think of GC specific flags, or C2 versus C1 flags. David > Maybe we should add other lp64_* flags right in this change? > At least UseCompressedClassPointers? > > Best regards, > Goetz > > > >> -----Original Message----- >> From: hotspot-dev On Behalf Of >> Boris Ulasevich >> Sent: Mittwoch, 7. November 2018 09:07 >> To: David Holmes ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(XS): 8213410: UseCompressedOops @requirement check fails >> fails on 32-bit system >> >> Hi David, >> >> Yes, at first glance it is weird. We have actually three states for >> vm.opt.final.UseCompressedOops: true, false and null. Null means "not >> applicable" - when current VM does not support the option with the given >> name. Here is another approach for the issue (I am not sure it is a good >> one): >> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ >> >> thank you, >> Boris >> >> On 07.11.2018 1:36, David Holmes wrote: >>> Hi Boris, >>> >>> I'm somewhat perplexed as to what the difference is between: >>> >>> @requires vm.opt.final.UseCompressedOops >>> >>> and >>> >>> @requires vm.opt.final.UseCompressedOops == true >>> >>> if they are indeed different then that seems to be a bug in the >>> requirement handling in the jtreg support code. >>> >>> I'm also wondering whether any such requirement should always be >>> proceeded by a check for 64-bits? >>> >>> Otherwise I would expect >>> >>> @requires vm.opt.final.UseCompressedOops >>> >>> to be false in 32-bit, so again a problem with the jtreg support code. >>> >>> Thanks, >>> David >>> >>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>> Hi all, >>>> >>>> Please review this patch to fix jtreg @requires >>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>>> ARM32: "invalid boolean value: null for expression >>>> vm.opt.final.UseCompressedOops". >>>> >>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>> >>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok by >>>> default, and becomes disabled with >>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>> >>>> Thanks, >>>> Boris Ulasevich From Alan.Bateman at oracle.com Thu Nov 8 13:14:57 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 8 Nov 2018 13:14:57 +0000 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> Message-ID: <163111cd-b108-3add-d20f-fcf4e5e42dc7@oracle.com> On 07/11/2018 19:33, Igor Ignatyev wrote: > Hi Vicente, > > I recall an (internal?) discussion about updating ASM somewhen in JDK 11TF, and AFAIR it was decided not to update ASM b/c nothing in JDK needs that, has it been changed? put somewhat differently, why are we doing this? > > in any case, I don't like the changes in mlvm tests. I understand that ClassWriter has been significantly changed in ASM 7.0, so ClassWriterExt can't disable CP entries caching (at least not in the way it used to), but removing setCache* calls from the tests changed them and in some cases made them invalid as they don't test that they supposed to. therefore I'd prefer to leave all calls setCache* as-is, change setCache* implementation to throw an exception (similarly to the fix in JDK-8194826 ) and mark all tests which throw this exception w/ '@ignore 8194951' jtreg tag. > > ClassWriterExt the MLVM tests have come in previous upgrades too. Has there been any discussion Remi or others on ASM to make it easier for the JDK to upgrade? -Alan From goetz.lindenmaier at sap.com Thu Nov 8 14:21:29 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 8 Nov 2018 14:21:29 +0000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <5d3c1cbb-cbcb-0700-36ec-412c960a3d77@oracle.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> <5d3c1cbb-cbcb-0700-36ec-412c960a3d77@oracle.com> Message-ID: <2a0ff46703594d9e9ca006335b93b112@sap.com> Hi David, > > must remember about the special case of this flag. > > Any developer writing any test that uses VM flags has to know whether > the flag is valid for all configurations/platforms etc. I'm not referring to the special case in the VM, but to the special case in VMProps. The developer here needs to know that for vm.opt.final.UseCompressedOops he obtains three values 'true', 'false' or 'null', while for other flags it is only two values. Overall, the whole VMProp vm.opt.final.* seems inconsistent to me. As I understand it returns how the value is set in the VM parsing the test script, not how it is set in the vm running the test, which is started for main/othervm. E.g., if I add -Xmx40g to the main/othervm command, UseCompressedOops will be off in that VM, although vm.opt.final.UseCompressedOops was true in first place. Best regards, Goetz. > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 8. November 2018 13:50 > To: Lindenmaier, Goetz ; Boris Ulasevich > ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(XS): 8213410: UseCompressedOops @requirement check fails > fails on 32-bit system > > On 8/11/2018 9:35 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > I also ran into this issue and designed this as fix for it: > > http://cr.openjdk.java.net/~goetz/wr18/8213527-32bit_tests/01/ > > I withdraw this part of my change :) > > > > I think handling this in VMProps is the better solution. > > Else every developer that implements a test with UseCompressedOops > > must remember about the special case of this flag. > > Any developer writing any test that uses VM flags has to know whether > the flag is valid for all configurations/platforms etc. > > > But I also think a generic solution as you propose, Boris, does not > > makes sense. You can not handle this generically. > > Imagine the flag was DontUseCompressedOops. By just setting > > it to false would be the wrong thing to do. > > Therefore I think we must special case on the very flag that causes > > the issue. > > Based on what you are testing you have to understand the role of the > flag. I agree that a test that requires a flag to be disabled probably > does not do as intended if the flag is not present at all. But the test > writer has to understand that. Just think of GC specific flags, or C2 > versus C1 flags. > > David > > > Maybe we should add other lp64_* flags right in this change? > > At least UseCompressedClassPointers? > > > > Best regards, > > Goetz > > > > > > > >> -----Original Message----- > >> From: hotspot-dev On Behalf Of > >> Boris Ulasevich > >> Sent: Mittwoch, 7. November 2018 09:07 > >> To: David Holmes ; hotspot- > dev at openjdk.java.net > >> Subject: Re: RFR(XS): 8213410: UseCompressedOops @requirement check > fails > >> fails on 32-bit system > >> > >> Hi David, > >> > >> Yes, at first glance it is weird. We have actually three states for > >> vm.opt.final.UseCompressedOops: true, false and null. Null means "not > >> applicable" - when current VM does not support the option with the given > >> name. Here is another approach for the issue (I am not sure it is a good > >> one): > >> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ > >> > >> thank you, > >> Boris > >> > >> On 07.11.2018 1:36, David Holmes wrote: > >>> Hi Boris, > >>> > >>> I'm somewhat perplexed as to what the difference is between: > >>> > >>> @requires vm.opt.final.UseCompressedOops > >>> > >>> and > >>> > >>> @requires vm.opt.final.UseCompressedOops == true > >>> > >>> if they are indeed different then that seems to be a bug in the > >>> requirement handling in the jtreg support code. > >>> > >>> I'm also wondering whether any such requirement should always be > >>> proceeded by a check for 64-bits? > >>> > >>> Otherwise I would expect > >>> > >>> @requires vm.opt.final.UseCompressedOops > >>> > >>> to be false in 32-bit, so again a problem with the jtreg support code. > >>> > >>> Thanks, > >>> David > >>> > >>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: > >>>> Hi all, > >>>> > >>>> Please review this patch to fix jtreg @requires > >>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on > >>>> ARM32: "invalid boolean value: null for expression > >>>> vm.opt.final.UseCompressedOops". > >>>> > >>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ > >>>> https://bugs.openjdk.java.net/browse/JDK-8213410 > >>>> > >>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok by > >>>> default, and becomes disabled with > >>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). > >>>> > >>>> Thanks, > >>>> Boris Ulasevich From rkennke at redhat.com Thu Nov 8 14:48:09 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 8 Nov 2018 15:48:09 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE301F3.9080706@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> Message-ID: <8237fe1e-f842-2d7e-a1c8-3f8667eb6574@redhat.com> I submitted a job to jdk/submit yesterday in branch JDK-8213199-2. I submitted it with 2 other jobs which succeeded a few hours later. JDK-8213199-2 did not reply yet. Any idea what's up? Is JDK-8213199-2 not a legal branch name? I thought this would be common practice. I already had JDK-8213199 for an earlier test, and did not want to bother reverting and reapplying on that branch. Roman > Hi Roman, > > The si_addr is always void*, so the cast to void* is redundant. > Otherwise, looks good. > Don't need to see another webrev for that. > > Thanks, > /Erik > > On 2018-11-07 15:51, Roman Kennke wrote: >> Hi Erik, >> >>> The approach looks great! :p >> Ha! :-) >> >>> Looks like "you" forgot to call the new function in os_solaris_sparc.cpp >>> though, he he. With that fixed, we pass tier1-3 on our platforms. >>> Oh and the cast in the following line is unnecessary: intptr_t start = >>> (intptr_t)-cell_header_size; >> Both fixed here (solaris/sparc stuff blindly though): >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03.diff >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03/ >> >> Good now? >> >> Thanks for reviewing and helping! >> >> Roman >> >>> Thanks, >>> /Erik >>> >>> On 2018-11-07 09:52, Roman Kennke wrote: >>>> Alright! thanks Erik! So here I'd like to propose your version plus my >>>> little fixes for review: >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.02/ >>>> >>>> It passes hotspot/jtreg:tier1 locally on x86 and aarch64 and works with >>>> Shenandoah too (and passes the Shenandoah testsuites) >>>> >>>> Good? >>>> >>>> Roman >>>> >>>> >>>>> Hi Roman, >>>>> >>>>> Glad you liked the idea! >>>>> >>>>> On 2018-11-06 22:37, Roman Kennke wrote: >>>>>> Hi Erik, >>>>>> >>>>>> I also like it. I needed to change a little bit to make it work with >>>>>> Shenandoah: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I will give it a spin on aarch64 as soon as I get access to the >>>>>> machine. >>>>>> >>>>>> Do you want to take the bug and finish it, or do you want me to >>>>>> take it >>>>>> from here? >>>>> You can take it from here. I just wanted to show a possible direction. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> Roman >>>>>> >>>>>>> Hi Andrew, >>>>>>> >>>>>>> On 2018-11-06 17:46, Andrew Dinn wrote: >>>>>>>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>>>>>>> New webrev: >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>>>>>>> Thanks for the updated version. >>>>>>>> >>>>>>>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>>>>>>> In that case, I hear you ask, why does the old code check for >>>>>>>>>> values in >>>>>>>>>> the range [heap_base - cell_header_size, heap_base + >>>>>>>>>> os::vm_page_size)? >>>>>>>>>> Good question. I don't know the answer. I think that test is >>>>>>>>>> bogus >>>>>>>>>> with >>>>>>>>>> no Shenandoah GC present. >>>>>>>>> Interesting. It turns out that both ranges are relevant >>>>>>>>> [-cell_header_size, os::vm_page_size) and [heap_base - >>>>>>>>> cell_header_size, >>>>>>>>> heap_base + os::vm_page_size). Looks like we both missed that >>>>>>>>> from the >>>>>>>>> original check. >>>>>>>>> I updated my webrev to explicitly reflect those two ranges, as two >>>>>>>>> different checks. >>>>>>>> Well, you say that but -- as I said before -- I am not sure why >>>>>>>> *both* >>>>>>>> are relevant to the non-Shenandoah case. And, given Roman's >>>>>>>> correction >>>>>>>> (doh, yes of course! the space for the Brooks pointer field had >>>>>>>> better >>>>>>>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>>>>>>> Shenandoah case either. >>>>>>> The reason it is needed is because there may or may not be null >>>>>>> checks >>>>>>> in both the decode() step to get the base pointer of an access, and >>>>>>> the >>>>>>> access itself, given some base pointer. This results in two >>>>>>> ranges for >>>>>>> those two cases correspondingly. If the decoding uses >>>>>>> decode_not_null, >>>>>>> with an implicit null check in the decoding itself that blindly >>>>>>> adds the >>>>>>> heap base without checking for null, we will crash into the first >>>>>>> page >>>>>>> of the heap which is protected when performing accesses into >>>>>>> that. But >>>>>>> sometimes we have a decode()ed base pointer (with null check), that >>>>>>> has >>>>>>> already been fixed up to NULL, and then perform an access at an >>>>>>> offset >>>>>>> into that with an implicit null check. This results in a trap at the >>>>>>> first page. So in summary, both ranges are needed because the >>>>>>> implicitness of the null check may be both in the access and the >>>>>>> narrowOop decoding. >>>>>>> >>>>>>> But as I pointed out, this (seemingly incorrectly) assumes there >>>>>>> may not >>>>>>> be any offset values in the (heap_base, heap_base + >>>>>>> os::vm_page_size) >>>>>>> range. If you hit such an offset, we will generate incorrect >>>>>>> code, due >>>>>>> to the lack of distinction between addresses and offsets. >>>>>>> >>>>>>>> I'd be very glad to hear from someone who knows about the >>>>>>>> history of >>>>>>>> the >>>>>>>> original function as to why the extra test for addresses above >>>>>>>> heap_base >>>>>>>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>>>>>>> might have been around when it was added?] >>>>>>>> >>>>>>>> If that check is not actually needed then getting rid of it would >>>>>>>> make >>>>>>>> the whole thing a lot simpler. >>>>>>> It would indeed. But unfortunately we need it. >>>>>>> >>>>>>>>>>> Just expressing these trivial ranges separately would have been >>>>>>>>>>> very >>>>>>>>>>> easy to understand. >>>>>>>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>>>>>>> wrinkle. >>>>>>>>>> It is possible for an object to be allocated at the first >>>>>>>>>> non-guarded >>>>>>>>>> address in the heap area i.e. heap_base + os::vm_page_size. I >>>>>>>>>> that >>>>>>>>>> case >>>>>>>>>> a dereference of its Brooks pointer will lie in the protected >>>>>>>>>> page. >>>>>>>>> Looks like Roman just said that's not possible. >>>>>>>> Yes, my dumb! >>>>>>>> >>>>>>>>> Fixed. Might need some help testing this on AArch64 though if we >>>>>>>>> decide >>>>>>>>> to go with it. >>>>>>>> I am sure Roman and Aleksey can do that but if needed I will. >>>>>>>> >>>>>>>>> Do you agree though, that the current version that merges the >>>>>>>>> address >>>>>>>>> and offset case relies on offsets being smaller than the heap >>>>>>>>> base for >>>>>>>>> its magic check to work, and that we actually do not have such a >>>>>>>>> guarantee in the VM? Apart from readability preferences, this >>>>>>>>> seems >>>>>>>>> like >>>>>>>>> it is broken in the current form, unless there are seat belts I >>>>>>>>> am not >>>>>>>>> aware of. In particular, if you have an offset value that is in >>>>>>>>> the >>>>>>>>> virtual address range (heap_base, heap_base + os::vm_page_size()), >>>>>>>>> which >>>>>>>>> I argue can happen, then it looks like we will normalize that as >>>>>>>>> if it >>>>>>>>> was an address (even though it is not), and incorrectly say that >>>>>>>>> this >>>>>>>>> offset does not need explicit null checks, despite needing >>>>>>>>> explicit >>>>>>>>> null >>>>>>>>> checks. >>>>>>>> Well, without the answer as to whether the check against >>>>>>>> heap_base is >>>>>>>> needed I am not sure. Assuming it is then then ... >>>>>>>> >>>>>>>> Yes, it is possible, in theory, that an oop offset could turn >>>>>>>> out to >>>>>>>> have value (heap_base + D) where 0 <= D < os_page_size(). In that >>>>>>>> case >>>>>>>> the offset might wrongly be classified as *not* needing a null >>>>>>>> check >>>>>>>> even though it really does. I suspect it is unlikely in practice >>>>>>>> given >>>>>>>> how mmap hands out addresses but you never know. >>>>>>>> >>>>>>>> As for readibility, I think the two functions are probably >>>>>>>> clearer -- >>>>>>>> expecially if they stay adjacent in the source code. >>>>>>> Okay, sounds like we are on the same page. I agree it is unlikely >>>>>>> that >>>>>>> this will crash the VM (as evidenced by the lack of crashes >>>>>>> observed). >>>>>>> But I don't like relying on luck for correctness. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>>> regards, >>>>>>>> >>>>>>>> >>>>>>>> Andrew Dinn >>>>>>>> ----------- >>>>>>>> Senior Principal Software Engineer >>>>>>>> Red Hat UK Ltd >>>>>>>> Registered in England and Wales under Company Registration No. >>>>>>>> 03798903 >>>>>>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric >>>>>>>> Shander >>>>>>>> > From adinn at redhat.com Thu Nov 8 16:20:13 2018 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 8 Nov 2018 16:20:13 +0000 Subject: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory In-Reply-To: <94479089-2f7b-0426-8f14-afa1c7f84869@oracle.com> References: <07d4d0fe-3a2a-414c-74ac-69e8527785d9@redhat.com> <50ed4716-b76e-6557-1146-03084776c160@redhat.com> <7ab7fb00-92aa-b952-0c63-9b1ab2479def@oracle.com> <600e8c67-80a4-fef5-b441-72c51c6ccddb@oracle.com> <3df3b5cd-dbc7-7fdd-bfc5-2a54d11127da@redhat.com> <27c9458d-7257-378a-4e3a-bd03402794be@oracle.com> <09c92b1a-c6da-e16b-bb68-553876e8a6ea@oracle.com> <2a0a385d-81ee-df66-f147-e4dd9aa5b72e@oracle.com> <8b2ab749-20f1-8dd3-3cc7-64db5d45bc7d@redhat.com> <94479089-2f7b-0426-8f14-afa1c7f84869@oracle.com> Message-ID: Hi Roger, On 08/11/18 14:51, Roger Riggs wrote: > If to achieve the performance or functional goals of the JEP hotspot > changes are needed > they should be mentioned (no details needed) in the JEP. > It helps the reader understand the scope and impact of the change. Thanks. I have added a brief overview of the required Hotspot changes. I also added a note about the target OS/CPU combinations. Vladimir, could you please take a look at the revised JEP and comment. The Hotspot-specifc details are at the end of the Description section where details of proposed new method Unsafe.writebackMemory are provided. Thanks for any feedback you can provide. regards, Andrew Dinn ----------- From vicente.romero at oracle.com Thu Nov 8 16:40:34 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 8 Nov 2018 11:40:34 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <163111cd-b108-3add-d20f-fcf4e5e42dc7@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <163111cd-b108-3add-d20f-fcf4e5e42dc7@oracle.com> Message-ID: <02269c23-e3aa-ba21-5ca1-1c2ae66b83fe@oracle.com> On 11/8/18 8:14 AM, Alan Bateman wrote: > On 07/11/2018 19:33, Igor Ignatyev wrote: >> Hi Vicente, >> >> I recall an (internal?) discussion about updating ASM somewhen in JDK >> 11TF, and AFAIR it was decided not to update ASM b/c nothing in JDK >> needs that, has it been changed? put somewhat differently, why are we >> doing this? >> >> in any case, I don't like the changes in mlvm tests. I understand >> that ClassWriter has been significantly changed in ASM 7.0, so >> ClassWriterExt can't disable CP entries caching (at least not in the >> way it used to), but removing setCache* calls from the tests changed >> them and in some cases made them invalid as they don't test that they >> supposed to. therefore I'd prefer to leave all calls setCache* as-is, >> change setCache* implementation to throw an exception (similarly to >> the fix in JDK-8194826 >> ) and mark all >> tests which throw this exception w/ '@ignore 8194951' jtreg tag. >> >> > ClassWriterExt the MLVM tests have come in previous upgrades too. Has > there been any discussion Remi or others on ASM to make it easier for > the JDK to upgrade? I'm not aware of any such discussions. > > -Alan Vicente From vicente.romero at oracle.com Thu Nov 8 16:39:39 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 8 Nov 2018 11:39:39 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> Message-ID: <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> Hi David, Igor On 11/7/18 10:03 PM, David Holmes wrote: > Hi Vicente, > > All of the javadoc comment reformatting makes it nearly impossible to > see the actual substantive changes :( > > ASM 7 also supports the Nestmate attributes and I was trying to see > how/where that appeared but its somewhat obscure. Oh well. > > Is it that case that the code the uses the ASM library, like the JFR > code and jlink code, and the tests, doesn't actually _have to_ change > to specifying Opcodes.ASM7 unless they plan on using ASM7 features? I changed only the tests for which the new ASM was complaining about a particular API available only for ASM7 > If so then you could split out the actual update of ASM from the > updates to the users of ASM (some of which may be quite fine with ASM5). I have made two webrevs to make the review easier [1], contain only the changes to the internal asm and [2] contains the changes to the clients plus make files, legal, etc. I have also made the changes to ClassWriterExt and affected test proposed by Igor in another mail, > > Thanks, > David Thanks, Vicente [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.only.00/ [2] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.additional.changes.00/ > > On 8/11/2018 1:56 AM, Vicente Romero wrote: >> Hi, >> >> Version 7.0 of ASM has been released. This version supports condy, >> yay!, and we want to include it in JDK 12. Please review [1] which >> includes: >> - the new version perse substituting the preview ASM internal version >> in the JDK >> - changes to additional files in particular some tests, mostly >> hotspot tests. >> >> Thanks, >> Vicente >> >> [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ From matthias.baesken at sap.com Thu Nov 8 17:13:42 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 8 Nov 2018 17:13:42 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk , I created another webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ The user info output is now guarded by ExtensiveErrorReports . Please review! Thanks, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Dienstag, 9. Oktober 2018 16:55 > To: 'Thomas St?fe' ; David Holmes > > Cc: Langer, Christoph ; Volker Simonis > ; HotSpot Open Source Developers dev at openjdk.java.net> > Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > > Hi Thomas , thanks for filing the CSR . > > I am fine with the name proposal "-XX:+-ExtensiveErrorReports", but will > wait until it is agreed to use this name . > Then I'll post another webrev of 8211326 using the flag . > > Best regards, Matthias > > > > > -----Original Message----- > > From: Thomas St?fe > > Sent: Montag, 8. Oktober 2018 12:21 > > To: David Holmes > > Cc: Baesken, Matthias ; Langer, Christoph > > ; Volker Simonis > ; > > HotSpot Open Source Developers > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > On Mon, Oct 8, 2018 at 11:50 AM David Holmes > > > wrote: > > > > > > Hi Thomas, > > > > > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > > > Hi all, > > > > > > > > I drafted a CSR: > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > > > > > Not sure what to do now, should I open a discussion thread on a > > > > mailing list? Or just wait for the committee to decide? (I'm doing > > > > this too rarely..) > > > > > > I filled in some of the additional info needed. The text needs to be > > > completed and the CSR needs to have at least one reviewer added to it. > > > Then it can be marked "final" and ready for the committee to evaluate. > > > > > > > Thanks! I filled out what was missing and will wait for reviewers. > > > > Thanks, Thomas > > From thomas.stuefe at gmail.com Thu Nov 8 19:59:46 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 8 Nov 2018 20:59:46 +0100 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Matthias, this looks good in general. Two remarks: 1) For convenience reasons, we resolve the user name from its id (and group name etc) with getpwuid. But I remember that we did this in our port after some deliberation - there is a risk that name resolving may hang or crash, depending on how exactly user ids are resolved on that machine. For our port we decided to go with it, since our VM runs usually in very controlled environments, but for the OpenJDK I propose to remove that part from the patch and just print out the numeric values. 2) The part where you print out the mode_t umask as "rwxrwxrwx" is kind of neat. Maybe we could factor that out as an own function in os::Posix? e.g. os::Posix::print_umask(mode_t mask) ? I leave (2) up to you. We also can do this in a follow up patch. Thanks, Thomas On Thu, Nov 8, 2018 at 6:13 PM Baesken, Matthias wrote: > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk , I created another webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > The user info output is now guarded by ExtensiveErrorReports . > > Please review! > > > Thanks, Matthias > > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Dienstag, 9. Oktober 2018 16:55 > > To: 'Thomas St?fe' ; David Holmes > > > > Cc: Langer, Christoph ; Volker Simonis > > ; HotSpot Open Source Developers > dev at openjdk.java.net> > > Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > > > > Hi Thomas , thanks for filing the CSR . > > > > I am fine with the name proposal "-XX:+-ExtensiveErrorReports", but will > > wait until it is agreed to use this name . > > Then I'll post another webrev of 8211326 using the flag . > > > > Best regards, Matthias > > > > > > > > > -----Original Message----- > > > From: Thomas St?fe > > > Sent: Montag, 8. Oktober 2018 12:21 > > > To: David Holmes > > > Cc: Baesken, Matthias ; Langer, Christoph > > > ; Volker Simonis > > ; > > > HotSpot Open Source Developers > > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > > > On Mon, Oct 8, 2018 at 11:50 AM David Holmes > > > > > wrote: > > > > > > > > Hi Thomas, > > > > > > > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > > > > Hi all, > > > > > > > > > > I drafted a CSR: > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > > > > > > > Not sure what to do now, should I open a discussion thread on a > > > > > mailing list? Or just wait for the committee to decide? (I'm doing > > > > > this too rarely..) > > > > > > > > I filled in some of the additional info needed. The text needs to be > > > > completed and the CSR needs to have at least one reviewer added to it. > > > > Then it can be marked "final" and ready for the committee to evaluate. > > > > > > > > > > Thanks! I filled out what was missing and will wait for reviewers. > > > > > > Thanks, Thomas > > > > From kim.barrett at oracle.com Thu Nov 8 21:41:06 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 8 Nov 2018 16:41:06 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> Message-ID: <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> > On Nov 8, 2018, at 4:38 AM, Aleksey Shipilev wrote: > http://cr.openjdk.java.net/~shade/8211926/webrev.02/ > > This still passes new gtest and hotspot-tier1. Does the patch make more sense now? ------------------------------------------------------------------------------ src/hotspot/share/utilities/bitMap.inline.hpp 240 assert(beg <= end, "underflow"); 245 assert(beg <= end, "underflow"); These new checks should instead be using verify_range. Sorry I didn't notice this earlier. ------------------------------------------------------------------------------ src/hotspot/share/utilities/bitMap.hpp 62 static const size_t SMALL_RANGE_WORDS = 32; Good to have this value as a named constant. However, while there doesn't seem to be a consistent naming convention for constants in HotSpot, the other constants in *this* class have lower-case names. ------------------------------------------------------------------------------ src/hotspot/share/utilities/bitMap.cpp 277 assert(beg_full_word < end_full_word, 278 "Performance: The range includes at least one full word"); 295 assert(beg_full_word < end_full_word, 296 "Performance: The range includes at least one full word"); For either of these assertions to trip, the value of SMALL_RANGE_WORDS would have needed to be poorly chosen. Better would be a static assertion on the value of SMALL_RANGE_WORDS. ------------------------------------------------------------------------------ > I'd like to keep the change small and contained, without significant rewrites, to simplify backports > without introducing new bugs. If additional check is significant on some path, users can just go to > non-large-hinted method versions. My suggested revision was not about any (uninteresting) performance impact of the extra check to avoid underflow. It was about improving the readability of the code by eliminating the possibility of underflow altogether. (That it has the side-benefit of picking up some micro-optimizations along the way is really not the point.) The new initial check in the webrev change probably needs some additional commentary; none of the authors or reviewers of the original code or the JDK-8152188 modification noticed the potential underflow, and it wasn't immediately obvious (to me at least) what that new check is for (I had to go re-read the bug report when I initially started reviewing). I don't buy the backport argument here. set_large_range is untouched since the initial mercurial checkin. The only change since then to clear_large_range is JDK-8152188 (JDK 9), which changed the similar assert there to the potentially underflowing conditional call to set_range that is being addressed here. From david.holmes at oracle.com Thu Nov 8 23:12:26 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Nov 2018 09:12:26 +1000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <2a0ff46703594d9e9ca006335b93b112@sap.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> <5d3c1cbb-cbcb-0700-36ec-412c960a3d77@oracle.com> <2a0ff46703594d9e9ca006335b93b112@sap.com> Message-ID: <22ca004d-9247-73ea-e1cb-bf63a520e7a1@oracle.com> On 9/11/2018 12:21 AM, Lindenmaier, Goetz wrote: > Hi David, > >>> must remember about the special case of this flag. >> >> Any developer writing any test that uses VM flags has to know whether >> the flag is valid for all configurations/platforms etc. > I'm not referring to the special case in the VM, but to the > special case in VMProps. > The developer here needs to know that for vm.opt.final.UseCompressedOops > he obtains three values 'true', 'false' or 'null', while > for other flags it is only two values. UseCompressedOops is not the only flag that only exists in a certain kind of build. But it is this distinction between present and not-present that I'm trying to clarify. You can imagine scenarios both for when you don't care if the flag exists, and you do. > Overall, the whole VMProp vm.opt.final.* seems inconsistent to me. > As I understand it returns how the value is set in the VM parsing the > test script, not how it is set in the vm running the test, which is > started for main/othervm. > E.g., if I add -Xmx40g to the main/othervm command, UseCompressedOops > will be off in that VM, although vm.opt.final.UseCompressedOops was true in > first place. The point of the @requires and its use of VMProps is to test what flags have been passed through to the jtreg invocation. It is up to each test to understand if it has any dependencies/conflicts with the flags that might be passed and whether it impacts them via explicit use of othervm or by launching a separate VM directly in the test. Cheers, David > Best regards, > Goetz. > > > > >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 8. November 2018 13:50 >> To: Lindenmaier, Goetz ; Boris Ulasevich >> ; hotspot-dev at openjdk.java.net >> Subject: Re: RFR(XS): 8213410: UseCompressedOops @requirement check fails >> fails on 32-bit system >> >> On 8/11/2018 9:35 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> I also ran into this issue and designed this as fix for it: >>> http://cr.openjdk.java.net/~goetz/wr18/8213527-32bit_tests/01/ >>> I withdraw this part of my change :) >>> >>> I think handling this in VMProps is the better solution. >>> Else every developer that implements a test with UseCompressedOops >>> must remember about the special case of this flag. >> >> Any developer writing any test that uses VM flags has to know whether >> the flag is valid for all configurations/platforms etc. >> >>> But I also think a generic solution as you propose, Boris, does not >>> makes sense. You can not handle this generically. >>> Imagine the flag was DontUseCompressedOops. By just setting >>> it to false would be the wrong thing to do. >>> Therefore I think we must special case on the very flag that causes >>> the issue. >> >> Based on what you are testing you have to understand the role of the >> flag. I agree that a test that requires a flag to be disabled probably >> does not do as intended if the flag is not present at all. But the test >> writer has to understand that. Just think of GC specific flags, or C2 >> versus C1 flags. >> >> David >> >>> Maybe we should add other lp64_* flags right in this change? >>> At least UseCompressedClassPointers? >>> >>> Best regards, >>> Goetz >>> >>> >>> >>>> -----Original Message----- >>>> From: hotspot-dev On Behalf Of >>>> Boris Ulasevich >>>> Sent: Mittwoch, 7. November 2018 09:07 >>>> To: David Holmes ; hotspot- >> dev at openjdk.java.net >>>> Subject: Re: RFR(XS): 8213410: UseCompressedOops @requirement check >> fails >>>> fails on 32-bit system >>>> >>>> Hi David, >>>> >>>> Yes, at first glance it is weird. We have actually three states for >>>> vm.opt.final.UseCompressedOops: true, false and null. Null means "not >>>> applicable" - when current VM does not support the option with the given >>>> name. Here is another approach for the issue (I am not sure it is a good >>>> one): >>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ >>>> >>>> thank you, >>>> Boris >>>> >>>> On 07.11.2018 1:36, David Holmes wrote: >>>>> Hi Boris, >>>>> >>>>> I'm somewhat perplexed as to what the difference is between: >>>>> >>>>> @requires vm.opt.final.UseCompressedOops >>>>> >>>>> and >>>>> >>>>> @requires vm.opt.final.UseCompressedOops == true >>>>> >>>>> if they are indeed different then that seems to be a bug in the >>>>> requirement handling in the jtreg support code. >>>>> >>>>> I'm also wondering whether any such requirement should always be >>>>> proceeded by a check for 64-bits? >>>>> >>>>> Otherwise I would expect >>>>> >>>>> @requires vm.opt.final.UseCompressedOops >>>>> >>>>> to be false in 32-bit, so again a problem with the jtreg support code. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this patch to fix jtreg @requires >>>>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>>>>> ARM32: "invalid boolean value: null for expression >>>>>> vm.opt.final.UseCompressedOops". >>>>>> >>>>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>>>> >>>>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok by >>>>>> default, and becomes disabled with >>>>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>>>> >>>>>> Thanks, >>>>>> Boris Ulasevich From shade at redhat.com Thu Nov 8 23:30:21 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 9 Nov 2018 00:30:21 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> Message-ID: <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> On 11/08/2018 10:41 PM, Kim Barrett wrote: >> On Nov 8, 2018, at 4:38 AM, Aleksey Shipilev wrote: >> http://cr.openjdk.java.net/~shade/8211926/webrev.02/ >> >> This still passes new gtest and hotspot-tier1. Does the patch make more sense now? > > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/bitMap.inline.hpp > 240 assert(beg <= end, "underflow"); > 245 assert(beg <= end, "underflow"); > > These new checks should instead be using verify_range. I don't think they should: verify_range operates on bit indexes, and here beg/end are the word indexes? While this would always work (famous last words), since word indexes are always below bit indexes, this would introduce weird dependency on verify_range with implicitly reintepreted arguments. It seems to me to-the-point asserts win here. > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/bitMap.hpp > 62 static const size_t SMALL_RANGE_WORDS = 32; > > Good to have this value as a named constant. However, while there > doesn't seem to be a consistent naming convention for constants in > HotSpot, the other constants in *this* class have lower-case names. Yes! Fixed. > ------------------------------------------------------------------------------ > src/hotspot/share/utilities/bitMap.cpp > 277 assert(beg_full_word < end_full_word, > 278 "Performance: The range includes at least one full word"); > > 295 assert(beg_full_word < end_full_word, > 296 "Performance: The range includes at least one full word"); > > For either of these assertions to trip, the value of SMALL_RANGE_WORDS > would have needed to be poorly chosen. Better would be a static > assertion on the value of SMALL_RANGE_WORDS. Or, you can think about it this way: this is a precondition for calling the code below, so it should be near the code that needs it, and it should test the exact invariant that we would like to hold true. Statically asserting small_range_words runs into risk of missing the errors in conditions involving small_range_words. But, I have no energy to fight it, reverted. > My suggested revision was not about any (uninteresting) performance > impact of the extra check to avoid underflow. It was about improving > the readability of the code by eliminating the possibility of > underflow altogether. (That it has the side-benefit of picking up > some micro-optimizations along the way is really not the point.) The > new initial check in the webrev change probably needs some additional > commentary; none of the authors or reviewers of the original code or > the JDK-8152188 modification noticed the potential underflow, and it > wasn't immediately obvious (to me at least) what that new check is for > (I had to go re-read the bug report when I initially started reviewing). Noted. It looks like what you have experienced can and should be mitigated by adding the comment near the interesting code. Did so! I realized (late) that your suggestion has a much nicer check shape (a + b >= c), adopted. > I don't buy the backport argument here. set_large_range is untouched > since the initial mercurial checkin. The only change since then to > clear_large_range is JDK-8152188 (JDK 9), which changed the similar > assert there to the potentially underflowing conditional call to > set_range that is being addressed here. What JDK-8152188 teaches me is that seemingly trivial code had introduced the bug, which we have noticed years later. This is why I am resisting rewriting the code any more than necessary, and instead willing to point-fix the actual bug, with maybe a little code homogenizing, and provide the regression test to make sure the exact bug does not happen anymore. This is my backport argument: make the fix as trivial+obvious as possible to avoid introducing more bugs in backports. I believe new patch is as straightforward as it could be, see: http://cr.openjdk.java.net/~shade/8211926/webrev.03/ Thanks, -Aleksey From kim.barrett at oracle.com Fri Nov 9 01:36:13 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 8 Nov 2018 20:36:13 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> Message-ID: <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> > On Nov 8, 2018, at 6:30 PM, Aleksey Shipilev wrote: > > On 11/08/2018 10:41 PM, Kim Barrett wrote: >> >> ------------------------------------------------------------------------------ >> src/hotspot/share/utilities/bitMap.inline.hpp >> 240 assert(beg <= end, "underflow"); >> 245 assert(beg <= end, "underflow"); >> >> These new checks should instead be using verify_range. > > I don't think they should: verify_range operates on bit indexes, and here beg/end are the word > indexes? While this would always work (famous last words), since word indexes are always below bit > indexes, this would introduce weird dependency on verify_range with implicitly reintepreted > arguments. It seems to me to-the-point asserts win here. Oh, you are right! Sorry about that. >> src/hotspot/share/utilities/bitMap.cpp >> 277 assert(beg_full_word < end_full_word, >> 278 "Performance: The range includes at least one full word"); >> >> 295 assert(beg_full_word < end_full_word, >> 296 "Performance: The range includes at least one full word"); >> >> For either of these assertions to trip, the value of SMALL_RANGE_WORDS >> would have needed to be poorly chosen. Better would be a static >> assertion on the value of SMALL_RANGE_WORDS. > > Or, you can think about it this way: this is a precondition for calling the code below, so it should > be near the code that needs it, and it should test the exact invariant that we would like to hold > true. Statically asserting small_range_words runs into risk of missing the errors in conditions > involving small_range_words. But, I have no energy to fight it, reverted. I don't have a problem with the check being near the dependent code; indeed, I think it should be there. Rather, I have problem with the form of the check. The check and associated comment you added to src/hotspot/share/utilities/bitMap.hpp would be better placed with the dependent code. > I realized (late) that your suggestion has a much nicer check shape (a + b >= c), adopted. Yes, that helps; see below. > >> I don't buy the backport argument here. set_large_range is untouched >> since the initial mercurial checkin. The only change since then to >> clear_large_range is JDK-8152188 (JDK 9), which changed the similar >> assert there to the potentially underflowing conditional call to >> set_range that is being addressed here. > > What JDK-8152188 teaches me is that seemingly trivial code had introduced the bug, which we have > noticed years later. This is why I am resisting rewriting the code any more than necessary, and > instead willing to point-fix the actual bug, with maybe a little code homogenizing, and provide the > regression test to make sure the exact bug does not happen anymore. This is my backport argument: > make the fix as trivial+obvious as possible to avoid introducing more bugs in backports. The underflow problem existed from the initial mercurial checkin, in the assert condition. JDK-8152188 didn't change any behavior in that underflow case. I'm also not really a fan of ham-stringing current improvements for reasons of backporting. But using the better comparison works for me, so I think your new proposed check is fine. It still suffers from the (theoretical) potential overflow from JDK-8213415, but that's a separate issue. > I believe new patch is as straightforward as it could be, see: > http://cr.openjdk.java.net/~shade/8211926/webrev.03/ I was about ready to approve, but took another look at the new test. Unfortunately, I think it has some problems. Most importantly, BITMAP_SIZE is too small for the actual large code to be invoked. With small_range_words being 32, one needs a range size of 2K bits (on 64bit platforms) to exceed that. And just increasing it may make this test take an excessive amount of time to run. (I was already wondering how long it takes, being quadradic in BITMAP_SIZE.) I think it would be sufficient for l to range from 0 to bit_index(2), with r ranging from l to BITMAP_SIZE in power of 2 steps, e.g. something like for (idx_t r = l, step = 1; r < BITMAP_SIZE; r += step, step <<= 1) { ... } and change BITMAP_SIZE to some multiple of bit_index(small_range_words), perhaps 3 or 4. Maybe add a couple edge cases to ensure r hits some word boundaries. Also, a style nit for BITMAP_SIZE: HotSpot code generally uses "static const" rather than "const static" (though there are a small number of the latter). I'm not sure the distinction between ResourceBitMap and CHeapBitMap is really all that interesting for the purpose of this set of tests. The tests could use TEST rather than TEST_VM if a static buffer were used for the BitMap storage, rather than using dynamic allocation. (That's what test_bitMap_search.cpp does. It seems that test_bitMap_setops.cpp takes yet another tack, using ::malloc.) I don't object to the current approach, but I think gtests that don't depend on a functioning VM are preferable where possible and not too onerous. From david.holmes at oracle.com Fri Nov 9 04:00:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Nov 2018 14:00:36 +1000 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> Message-ID: <0bb51977-09ec-a2f1-cae4-ee54df1c8a89@oracle.com> Hi Vicente, On 9/11/2018 2:39 AM, Vicente Romero wrote: > Hi David, Igor > > On 11/7/18 10:03 PM, David Holmes wrote: >> Hi Vicente, >> >> All of the javadoc comment reformatting makes it nearly impossible to >> see the actual substantive changes :( >> >> ASM 7 also supports the Nestmate attributes and I was trying to see >> how/where that appeared but its somewhat obscure. Oh well. >> >> Is it that case that the code the uses the ASM library, like the JFR >> code and jlink code, and the tests, doesn't actually _have to_ change >> to specifying Opcodes.ASM7 unless they plan on using ASM7 features? > > I changed only the tests for which the new ASM was complaining about a > particular API available only for ASM7 I could not understand how this could be if the tests and other code were unchanged, so I applied the ASM-only patch and investigated the first failure running nasgen. And of course the problem is the NestHost/NestMembers attributes! We modified our internal version of ASM to add nestmate support, but of course this update removes that and replaces it with the official support. But the official support is only enabled for ASMv7 so we must update all users of ASM to request version 7. Thanks, David ----- >> If so then you could split out the actual update of ASM from the >> updates to the users of ASM (some of which may be quite fine with ASM5). > > I have made two webrevs to make the review easier [1], contain only the > changes to the internal asm and [2] contains the changes to the clients > plus make files, legal, etc. I have also made the changes to > ClassWriterExt and affected test proposed by Igor in another mail, > >> >> Thanks, >> David > > Thanks, > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.only.00/ > [2] > http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.additional.changes.00/ > >> >> On 8/11/2018 1:56 AM, Vicente Romero wrote: >>> Hi, >>> >>> Version 7.0 of ASM has been released. This version supports condy, >>> yay!, and we want to include it in JDK 12. Please review [1] which >>> includes: >>> - the new version perse substituting the preview ASM internal version >>> in the JDK >>> - changes to additional files in particular some tests, mostly >>> hotspot tests. >>> >>> Thanks, >>> Vicente >>> >>> [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ > From ioi.lam at oracle.com Fri Nov 9 06:35:22 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 8 Nov 2018 22:35:22 -0800 Subject: RFR (S) Allow ResourceHashtable size to be specified at runtime Message-ID: https://bugs.openjdk.java.net/browse/JDK-8213587 http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ TL;DR: -- add a subclass to ResourceHashtable to allow the table size to be ? ? ? ? ? dynamically specified when the table is constructed. ? ?? ?? *** C++ template guru alert *** I don't know much about C++ templates, so my attempt on doing this may be ill-advised. I *think* that with my patch, the performance of existing code, which uses a statically-defined SIZE,? should not be affected, as the C++ compiler should be able to constant-propagate and reduce the new code: ? ALWAYSINLINE unsigned size() const { ??? if (SIZE != CONFIGURABLE_SIZE) { ????? return SIZE; ??? } else { ????? return _configured_table_size; ??? } ? } ? ALWAYSINLINE Node** get_table() const { ??? if (SIZE != CONFIGURABLE_SIZE) { ????? return (Node**)(&_static_table[0]); ??? } else { ????? return _configured_table; ??? } ? } ? Node** lookup_node(unsigned hash, K const& key) { ??? unsigned index = hash % size();??? <----- ??? Node** table = get_table(); ??? Node** ptr = &table[index]; ? <----- back to the old code: ? Node** lookup_node(unsigned hash, K const& key) { ??? unsigned index = hash % SIZE;????? <----- ??? Node** ptr = &_table[index];?????? <----- If anyone has a better way of doing this, I'd love to hear it! Thanks! - Ioi From david.holmes at oracle.com Fri Nov 9 07:39:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 9 Nov 2018 17:39:05 +1000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Message-ID: Hi Boris, On 7/11/2018 6:54 PM, David Holmes wrote: > Hi Boris, > > On 7/11/2018 6:06 PM, Boris Ulasevich wrote: >> Hi David, >> >> ?? Yes, at first glance it is weird. We have actually three states for >> vm.opt.final.UseCompressedOops: true, false and null. Null means "not >> applicable" - when current VM does not support the option with the >> given name. Here is another approach for the issue (I am not sure it >> is a good one): >> ?? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ > > I like that approach! I'm just not sure everyone else will necessarily > agree :) This proposal might need a wider audience than just hotspot-dev. > > Let me make enquiries with jtreg-use at openjdk.java.net and see what the > general intent was here. That was uninformative as to intent as Jon initially thought both variants should fail the same way, but on further reflection it is likely due to the difference between trying to evaluate null as a boolean - which gives an error - and doing "true".equals(null) , which is just false. I also see now that this isn't a general facility for using @requires because we only actually support three directly: protected void vmOptFinalFlags(Map map) { vmOptFinalFlag(map, "ClassUnloading"); vmOptFinalFlag(map, "UseCompressedOops"); vmOptFinalFlag(map, "EnableJVMCI"); } but that also shows this is not specific to UseCompressedOops because EnableJVMCI only exists in a VM that was built with JVMCI support. I still think the correct way for this current case to be handled in tests is by using: @requires vm.bits == 64 & UseCompressedOops though that requires jtreg to do short-circuit boolean evaluation, which it appears not to do. :( Though: @requires vm.bits == 64 & UseCompressedOops == true should work (albeit perhaps due to implementation accident). Cheers, David ----- > Aside: if the UseCompressOops checks is preceded by a 64-bit check does > it still fail? (Just wondering whether jtreg does short-circuit > evaluation for these @require statements.) > > Thanks, > David > >> thank you, >> Boris >> >> On 07.11.2018 1:36, David Holmes wrote: >>> Hi Boris, >>> >>> I'm somewhat perplexed as to what the difference is between: >>> >>> @requires vm.opt.final.UseCompressedOops >>> >>> and >>> >>> @requires vm.opt.final.UseCompressedOops == true >>> >>> if they are indeed different then that seems to be a bug in the >>> requirement handling in the jtreg support code. >>> >>> I'm also wondering whether any such requirement should always be >>> proceeded by a check for 64-bits? >>> >>> Otherwise I would expect >>> >>> @requires vm.opt.final.UseCompressedOops >>> >>> to be false in 32-bit, so again a problem with the jtreg support code. >>> >>> Thanks, >>> David >>> >>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>> Hi all, >>>> >>>> Please review this patch to fix jtreg @requires >>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>>> ARM32: "invalid boolean value: null for expression >>>> vm.opt.final.UseCompressedOops". >>>> >>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>> >>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok >>>> by default, and becomes disabled with >>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>> >>>> Thanks, >>>> Boris Ulasevich From matthias.baesken at sap.com Fri Nov 9 08:45:45 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 9 Nov 2018 08:45:45 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Thomas, thank you for the remarks . I introduced os::Posix::print_umask > > But I remember that we did this in our > port after some deliberation - there is a risk that name resolving may > hang or crash, depending on how exactly user ids are resolved on that machine. > ..and removed the getpwuid / getgrgid calls (so we only get the numeric ids in the output and not the user names / group names) New webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/ Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Donnerstag, 8. November 2018 21:00 > To: Baesken, Matthias > Cc: David Holmes ; Langer, Christoph > ; Volker Simonis ; > HotSpot Open Source Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > Hi Matthias, > > this looks good in general. Two remarks: > > 1) For convenience reasons, we resolve the user name from its id (and > group name etc) with getpwuid. But I remember that we did this in our > port after some deliberation - there is a risk that name resolving may > hang or crash, depending on how exactly user ids are resolved on that > machine. For our port we decided to go with it, since our VM runs > usually in very controlled environments, but for the OpenJDK I propose > to remove that part from the patch and just print out the numeric > values. > > 2) The part where you print out the mode_t umask as "rwxrwxrwx" is > kind of neat. Maybe we could factor that out as an own function in > os::Posix? e.g. os::Posix::print_umask(mode_t mask) ? > > I leave (2) up to you. We also can do this in a follow up patch. > > Thanks, Thomas > > > On Thu, Nov 8, 2018 at 6:13 PM Baesken, Matthias > wrote: > > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk , > I created another webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > Please review! > > > > > > Thanks, Matthias > > > > > > > -----Original Message----- > > > From: Baesken, Matthias > > > Sent: Dienstag, 9. Oktober 2018 16:55 > > > To: 'Thomas St?fe' ; David Holmes > > > > > > Cc: Langer, Christoph ; Volker Simonis > > > ; HotSpot Open Source Developers > > > dev at openjdk.java.net> > > > Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > > > > > > Hi Thomas , thanks for filing the CSR . > > > > > > I am fine with the name proposal "-XX:+-ExtensiveErrorReports", but > will > > > wait until it is agreed to use this name . > > > Then I'll post another webrev of 8211326 using the flag . > > > > > > Best regards, Matthias > > > > > > > > > > > > > -----Original Message----- > > > > From: Thomas St?fe > > > > Sent: Montag, 8. Oktober 2018 12:21 > > > > To: David Holmes > > > > Cc: Baesken, Matthias ; Langer, > Christoph > > > > ; Volker Simonis > > > ; > > > > HotSpot Open Source Developers > > > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err > file > > > > > > > > On Mon, Oct 8, 2018 at 11:50 AM David Holmes > > > > > > > wrote: > > > > > > > > > > Hi Thomas, > > > > > > > > > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > > > > > Hi all, > > > > > > > > > > > > I drafted a CSR: > > > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > > > > > > > > > Not sure what to do now, should I open a discussion thread on a > > > > > > mailing list? Or just wait for the committee to decide? (I'm doing > > > > > > this too rarely..) > > > > > > > > > > I filled in some of the additional info needed. The text needs to be > > > > > completed and the CSR needs to have at least one reviewer added to > it. > > > > > Then it can be marked "final" and ready for the committee to > evaluate. > > > > > > > > > > > > > Thanks! I filled out what was missing and will wait for reviewers. > > > > > > > > Thanks, Thomas > > > > > > From thomas.stuefe at gmail.com Fri Nov 9 09:08:15 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 9 Nov 2018 10:08:15 +0100 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Matthias, http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/posix/os_posix.hpp.udiff.html + static void print_user_info(outputStream* st, char* buf, size_t buflen); You can get rid of buf+buflen too, not needed anymore --- http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/posix/os_posix.cpp.udiff.html In os::Posix::print_user_info(), I would probably now print it all one one like, e.g. "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: rwxr--r--" - for brevity. But I leave that up to you. --- The "removing" text does not make sense and can itself be removed :) -- I spotted a tiny issue which puts the whole print umask into question. Appearantly, there is no way for umask() to just return the current value of the file creation mask without changing it. So, that is why we restore it right afterwards. That is annoying. That leaves a tiny tiny time window where a different thread could create a file which would then have permissions all zero'd out. I am not sure how to deal with that. Could you look around whether there is another posix API to just look at the umask without changing it? Otherwise: does anyone think this is a problem? -- Sorry, turns out simple things are often more complicated than it first appears... Cheers, Thomas On Fri, Nov 9, 2018 at 9:45 AM Baesken, Matthias wrote: > > Hi Thomas, thank you for the remarks . > I introduced > > os::Posix::print_umask > > > > > But I remember that we did this in our > > port after some deliberation - there is a risk that name resolving may > > hang or crash, depending on how exactly user ids are resolved on that machine. > > > > ..and removed the getpwuid / getgrgid calls (so we only get the numeric ids in the output and not the user names / group names) > New webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/ > > > Best regards, Matthias > > > > -----Original Message----- > > From: Thomas St?fe > > Sent: Donnerstag, 8. November 2018 21:00 > > To: Baesken, Matthias > > Cc: David Holmes ; Langer, Christoph > > ; Volker Simonis ; > > HotSpot Open Source Developers > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > Hi Matthias, > > > > this looks good in general. Two remarks: > > > > 1) For convenience reasons, we resolve the user name from its id (and > > group name etc) with getpwuid. But I remember that we did this in our > > port after some deliberation - there is a risk that name resolving may > > hang or crash, depending on how exactly user ids are resolved on that > > machine. For our port we decided to go with it, since our VM runs > > usually in very controlled environments, but for the OpenJDK I propose > > to remove that part from the patch and just print out the numeric > > values. > > > > 2) The part where you print out the mode_t umask as "rwxrwxrwx" is > > kind of neat. Maybe we could factor that out as an own function in > > os::Posix? e.g. os::Posix::print_umask(mode_t mask) ? > > > > I leave (2) up to you. We also can do this in a follow up patch. > > > > Thanks, Thomas > > > > > > On Thu, Nov 8, 2018 at 6:13 PM Baesken, Matthias > > wrote: > > > > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk , > > I created another webrev : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > > > Please review! > > > > > > > > > Thanks, Matthias > > > > > > > > > > -----Original Message----- > > > > From: Baesken, Matthias > > > > Sent: Dienstag, 9. Oktober 2018 16:55 > > > > To: 'Thomas St?fe' ; David Holmes > > > > > > > > Cc: Langer, Christoph ; Volker Simonis > > > > ; HotSpot Open Source Developers > > > > > dev at openjdk.java.net> > > > > Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > > > > > > > > Hi Thomas , thanks for filing the CSR . > > > > > > > > I am fine with the name proposal "-XX:+-ExtensiveErrorReports", but > > will > > > > wait until it is agreed to use this name . > > > > Then I'll post another webrev of 8211326 using the flag . > > > > > > > > Best regards, Matthias > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Thomas St?fe > > > > > Sent: Montag, 8. Oktober 2018 12:21 > > > > > To: David Holmes > > > > > Cc: Baesken, Matthias ; Langer, > > Christoph > > > > > ; Volker Simonis > > > > ; > > > > > HotSpot Open Source Developers > > > > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err > > file > > > > > > > > > > On Mon, Oct 8, 2018 at 11:50 AM David Holmes > > > > > > > > > wrote: > > > > > > > > > > > > Hi Thomas, > > > > > > > > > > > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > > > > > > Hi all, > > > > > > > > > > > > > > I drafted a CSR: > > > > > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > > > > > > > > > > > Not sure what to do now, should I open a discussion thread on a > > > > > > > mailing list? Or just wait for the committee to decide? (I'm doing > > > > > > > this too rarely..) > > > > > > > > > > > > I filled in some of the additional info needed. The text needs to be > > > > > > completed and the CSR needs to have at least one reviewer added to > > it. > > > > > > Then it can be marked "final" and ready for the committee to > > evaluate. > > > > > > > > > > > > > > > > Thanks! I filled out what was missing and will wait for reviewers. > > > > > > > > > > Thanks, Thomas > > > > > > > > From shade at redhat.com Fri Nov 9 09:33:49 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 9 Nov 2018 10:33:49 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> Message-ID: On 11/09/2018 02:36 AM, Kim Barrett wrote: >> I believe new patch is as straightforward as it could be, see: >> http://cr.openjdk.java.net/~shade/8211926/webrev.03/ > > I was about ready to approve, but took another look at the new test. > Unfortunately, I think it has some problems. Okay, let's try again after fixing the problems you mentioned: http://cr.openjdk.java.net/~shade/8211926/webrev.04/ Moved STATIC_ASSERT and comments to the actual use site, rewritten test. Test takes 0.5s per TEST(). Looks good now? -Aleksey From matthias.baesken at sap.com Fri Nov 9 10:08:27 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 9 Nov 2018 10:08:27 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Thomas , http://man7.org/linux/man-pages/man2/umask.2.html says : It is impossible to use umask() to fetch a process's umask without at the same time changing it. A second call to umask() would then be needed to restore the umask. The nonatomicity of these two steps provides the potential for races in multithreaded programs. Since Linux 4.7, the umask of any process can be viewed via the Umask field of /proc/[pid]/status. Inspecting this field in /proc/self/status allows a process to retrieve its umask without at the same time changing it. So it seems to be possible to just get the umask-info , but only on some OS versions . Don't know how relevant the small time window is, in case of hs_err file writing probably not so relevant. Btw. there is a similar pattern here : jdk/src/java.prefs/unix/native/libprefs/FileSystemPreferences.c 94 old_umask = umask(0); 95 fd = open(fname, O_WRONLY|O_CREAT, permission); 96 result[1] = errno; 97 umask(old_umask); Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Freitag, 9. November 2018 10:08 > To: Baesken, Matthias > Cc: David Holmes ; Langer, Christoph > ; Volker Simonis ; > HotSpot Open Source Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > Hi Matthias, > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > posix/os_posix.hpp.udiff.html > > + static void print_user_info(outputStream* st, char* buf, size_t buflen); > > You can get rid of buf+buflen too, not needed anymore > --- > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > posix/os_posix.cpp.udiff.html > > In os::Posix::print_user_info(), I would probably now print it all one > one like, e.g. "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: > rwxr--r--" - for brevity. But I leave that up to you. > --- > > The "removing" text does not make sense and can itself be removed :) > > -- > > I spotted a tiny issue which puts the whole print umask into question. > Appearantly, there is no way for umask() to just return the current > value of the file creation mask without changing it. So, that is why > we restore it right afterwards. That is annoying. That leaves a tiny > tiny time window where a different thread could create a file which > would then have permissions all zero'd out. > > I am not sure how to deal with that. Could you look around whether > there is another posix API to just look at the umask without changing > it? Otherwise: does anyone think this is a problem? > > -- > > Sorry, turns out simple things are often more complicated than it > first appears... > > Cheers, Thomas > > > > > > > On Fri, Nov 9, 2018 at 9:45 AM Baesken, Matthias > wrote: > > > > Hi Thomas, thank you for the remarks . > > I introduced > > > > os::Posix::print_umask > > > > > > > > But I remember that we did this in our > > > port after some deliberation - there is a risk that name resolving may > > > hang or crash, depending on how exactly user ids are resolved on that > machine. > > > > > > > ..and removed the getpwuid / getgrgid calls (so we only get the > numeric ids in the output and not the user names / group names) > > New webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/ > > > > > > Best regards, Matthias > > > > > > > -----Original Message----- > > > From: Thomas St?fe > > > Sent: Donnerstag, 8. November 2018 21:00 > > > To: Baesken, Matthias > > > Cc: David Holmes ; Langer, Christoph > > > ; Volker Simonis > ; > > > HotSpot Open Source Developers > > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > > > Hi Matthias, > > > > > > this looks good in general. Two remarks: > > > > > > 1) For convenience reasons, we resolve the user name from its id (and > > > group name etc) with getpwuid. But I remember that we did this in our > > > port after some deliberation - there is a risk that name resolving may > > > hang or crash, depending on how exactly user ids are resolved on that > > > machine. For our port we decided to go with it, since our VM runs > > > usually in very controlled environments, but for the OpenJDK I propose > > > to remove that part from the patch and just print out the numeric > > > values. > > > > > > 2) The part where you print out the mode_t umask as "rwxrwxrwx" is > > > kind of neat. Maybe we could factor that out as an own function in > > > os::Posix? e.g. os::Posix::print_umask(mode_t mask) ? > > > > > > I leave (2) up to you. We also can do this in a follow up patch. > > > > > > Thanks, Thomas > > > > > > > > > On Thu, Nov 8, 2018 at 6:13 PM Baesken, Matthias > > > wrote: > > > > > > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into > jdk/jdk , > > > I created another webrev : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > > > > > Please review! > > > > > > > > > > > > Thanks, Matthias > > > > > > > > > > > > > -----Original Message----- > > > > > From: Baesken, Matthias > > > > > Sent: Dienstag, 9. Oktober 2018 16:55 > > > > > To: 'Thomas St?fe' ; David Holmes > > > > > > > > > > Cc: Langer, Christoph ; Volker Simonis > > > > > ; HotSpot Open Source Developers > > > > > > > dev at openjdk.java.net> > > > > > Subject: RE: RFR : 8211326 : add OS user related information to hs_err > file > > > > > > > > > > Hi Thomas , thanks for filing the CSR . > > > > > > > > > > I am fine with the name proposal "-XX:+-ExtensiveErrorReports", > but > > > will > > > > > wait until it is agreed to use this name . > > > > > Then I'll post another webrev of 8211326 using the flag . > > > > > > > > > > Best regards, Matthias > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Thomas St?fe > > > > > > Sent: Montag, 8. Oktober 2018 12:21 > > > > > > To: David Holmes > > > > > > Cc: Baesken, Matthias ; Langer, > > > Christoph > > > > > > ; Volker Simonis > > > > > ; > > > > > > HotSpot Open Source Developers dev at openjdk.java.net> > > > > > > Subject: Re: RFR : 8211326 : add OS user related information to > hs_err > > > file > > > > > > > > > > > > On Mon, Oct 8, 2018 at 11:50 AM David Holmes > > > > > > > > > > > wrote: > > > > > > > > > > > > > > Hi Thomas, > > > > > > > > > > > > > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > > > > > > > Hi all, > > > > > > > > > > > > > > > > I drafted a CSR: > > > > > > > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > > > > > > > > > > > > > Not sure what to do now, should I open a discussion thread on a > > > > > > > > mailing list? Or just wait for the committee to decide? (I'm doing > > > > > > > > this too rarely..) > > > > > > > > > > > > > > I filled in some of the additional info needed. The text needs to be > > > > > > > completed and the CSR needs to have at least one reviewer > added to > > > it. > > > > > > > Then it can be marked "final" and ready for the committee to > > > evaluate. > > > > > > > > > > > > > > > > > > > Thanks! I filled out what was missing and will wait for reviewers. > > > > > > > > > > > > Thanks, Thomas > > > > > > > > > > From boris.ulasevich at bell-sw.com Fri Nov 9 11:03:47 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Fri, 9 Nov 2018 14:03:47 +0300 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Message-ID: Hi David, Thank you. Don't you think it is better to learn jtreg to perform short-circuit boolean evaluation? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02 http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02_jtreg By the way, with the given change in jtreg two failed tests from the original list works Ok because they already have 64-bit check on previous lines: * @requires (sun.arch.data.model == "64") * @requires vm.opt.final.UseCompressedOops regards, Boris On 09.11.2018 10:39, David Holmes wrote: > Hi Boris, > > On 7/11/2018 6:54 PM, David Holmes wrote: >> Hi Boris, >> >> On 7/11/2018 6:06 PM, Boris Ulasevich wrote: >>> Hi David, >>> >>> ?? Yes, at first glance it is weird. We have actually three states >>> for vm.opt.final.UseCompressedOops: true, false and null. Null means >>> "not applicable" - when current VM does not support the option with >>> the given name. Here is another approach for the issue (I am not sure >>> it is a good one): >>> ?? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ >> >> I like that approach! I'm just not sure everyone else will necessarily >> agree :) This proposal might need a wider audience than just hotspot-dev. >> >> Let me make enquiries with jtreg-use at openjdk.java.net and see what the >> general intent was here. > > That was uninformative as to intent as Jon initially thought both > variants should fail the same way, but on further reflection it is > likely due to the difference between trying to evaluate null as a > boolean - which gives an error - and doing "true".equals(null) , which > is just false. > > I also see now that this isn't a general facility for using > > @requires > > because we only actually support three directly: > > ?protected void vmOptFinalFlags(Map map) { > ??????? vmOptFinalFlag(map, "ClassUnloading"); > ??????? vmOptFinalFlag(map, "UseCompressedOops"); > ??????? vmOptFinalFlag(map, "EnableJVMCI"); > ? } > > but that also shows this is not specific to UseCompressedOops because > EnableJVMCI only exists in a VM that was built with JVMCI support. > > I still think the correct way for this current case to be handled in > tests is by using: > > @requires vm.bits == 64 & UseCompressedOops > > though that requires jtreg to do short-circuit boolean evaluation, which > it appears not to do. :( Though: > > @requires vm.bits == 64 & UseCompressedOops == true > > should work (albeit perhaps due to implementation accident). > > Cheers, > David > ----- > >> Aside: if the UseCompressOops checks is preceded by a 64-bit check >> does it still fail? (Just wondering whether jtreg does short-circuit >> evaluation for these @require statements.) >> >> Thanks, >> David >> >>> thank you, >>> Boris >>> >>> On 07.11.2018 1:36, David Holmes wrote: >>>> Hi Boris, >>>> >>>> I'm somewhat perplexed as to what the difference is between: >>>> >>>> @requires vm.opt.final.UseCompressedOops >>>> >>>> and >>>> >>>> @requires vm.opt.final.UseCompressedOops == true >>>> >>>> if they are indeed different then that seems to be a bug in the >>>> requirement handling in the jtreg support code. >>>> >>>> I'm also wondering whether any such requirement should always be >>>> proceeded by a check for 64-bits? >>>> >>>> Otherwise I would expect >>>> >>>> @requires vm.opt.final.UseCompressedOops >>>> >>>> to be false in 32-bit, so again a problem with the jtreg support code. >>>> >>>> Thanks, >>>> David >>>> >>>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch to fix jtreg @requires >>>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>>>> ARM32: "invalid boolean value: null for expression >>>>> vm.opt.final.UseCompressedOops". >>>>> >>>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>>> >>>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works Ok >>>>> by default, and becomes disabled with >>>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>>> >>>>> Thanks, >>>>> Boris Ulasevich From amith.pawar at gmail.com Fri Nov 9 12:23:32 2018 From: amith.pawar at gmail.com (amith pawar) Date: Fri, 9 Nov 2018 17:53:32 +0530 Subject: [PATCH] JDK NUMA Interleaving issue Message-ID: Hi all, The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, S1 and S2 region (if any other ) memory areas on requested Numa nodes and it should not configure itself to access other Numa nodes. This issue is observed only when Java is allowed to run on fewer NUMA nodes than available on the system with numactl membind and interleave options. Running on all the nodes does not have any effect. This will cause some applications (objects residing in old gen and survivor region) to run slower on system with large Numa nodes. I have described below in more detail for both numactl MEMBIND and INTERLEAVE options. Addresses from both GC log and process numa maps clearly shows that the JAVA process is configured to access other memory nodes even though it is not allowed. Two scenarios are: 1. Numactl membind case : numactl -m 0-1, -N 0-1 Numa map shows these regions are INTERLEAVED ON ALL NODES instead of specified Numa memory nodes 0 and 1. ----------------------------------- GC HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- GC Options: -Xlog:gc*=info,gc+heap=debug -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [602.180s][debug][gc,heap ] GC(20) Heap before GC invocations=21 (full 4): PSYoungGen total 120604672K, used 11907587K [0x00002afc4b200000, 0x00002b198b200000, 0x00002b198b200000) [602.180s][debug][gc,heap ] GC(20) eden space 118525952K, 8% used [0x00002afc4b200000,0x00002b0bb1b376e0,0x00002b188d600000) [602.180s][debug][gc,heap ] GC(20) lgrp 0 space 59262976K, 8% used [0x00002afc4b200000,0x00002afd89bef450,0x00002b0a6c400000) [602.180s][debug][gc,heap ] GC(20) lgrp 1 space 59262976K, 8% used [0x00002b0a6c400000,0x00002b0bb1b376e0,0x00002b188d600000) [602.180s][debug][gc,heap ] GC(20) from space 2078720K, 65% used [0x00002b190c400000,0x00002b195ef5a0d0,0x00002b198b200000) [602.180s][debug][gc,heap ] GC(20) to space 2078720K, 0% used [0x00002b188d600000,0x00002b188d600000,0x00002b190c400000) [602.180s][debug][gc,heap ] GC(20) ParOldGen total 2097152K, used 226685K [0x00002afbcb200000, 0x00002afc4b200000, 0x00002afc4b200000) [602.180s][debug][gc,heap ] GC(20) object space 2097152K, 10% used [0x00002afbcb200000,0x00002afbd8f5f460,0x00002afc4b200000) [602.180s][debug][gc,heap ] GC(20) Metaspace used 28462K, capacity 29008K, committed 29184K, reserved 30720K ----------------------------------- GC HEAP END-------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------- PROCESS NUMA Maps --------------------------------------------------------------------------------------------------------------------------------------------------- Command : cat /proc/4947/numa_maps -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- =======> Following addresses are interleaved on all nodes. 2afbb4f4c000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 N4=2 N5=2 N6=2 N7=2 kernelpagesize_kB=4 2afbb4f6c000 interleave:0-7 2afbb7e88000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 N4=6 N5=6 N6=6 N7=6 kernelpagesize_kB=4 2afbbc000000 interleave:0-7 anon=8704 dirty=8704 N0=1600 N1=1088 N2=1088 N3=576 N4=1088 N5=1088 N6=1088 N7=1088 kernelpagesize_kB=4 2afbc3be6000 interleave:0-7 anon=6682 dirty=6682 N0=1027 N1=1027 N2=515 N3=515 N4=515 N5=1027 N6=1028 N7=1028 kernelpagesize_kB=4 2afbcb000000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 N4=6 N5=6 N6=6 N7=6 kernelpagesize_kB=4 2afbcb200000 interleave:0-7 anon=524288 dirty=524288 N0=65536 N1=65536 N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 kernelpagesize_kB=4 ==> OLD GEN Address 2afc4b200000 prefer:0 anon=1536 dirty=1536 N0=1536 kernelpagesize_kB=4 2b0a6c400000 prefer:1 anon=512 dirty=512 N1=512 kernelpagesize_kB=4 2b188d600000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 N1=130048 N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 kernelpagesize_kB=4 ==> Survivor Region 2b198b600000 interleave:0-7 anon=60929 dirty=60929 N0=7233 N1=7744 N2=7232 N3=7744 N4=7744 N5=7744 N6=7744 N7=7744 kernelpagesize_kB=4 ---------------------------------- PROCESS NUMA Maps ----------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------- PROCESS status ----------------------------------------------------------------------------------------------------------------------------------------------------------- Command : cat /proc/4947/status -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Cpus_allowed: 00000000,0000ffff,00000000,0000ffff Cpus_allowed_list: 0-15,64-79 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff Mems_allowed_list: 0-7 --------------------------------- PROCESS status ------------------------------------------------------------------------------------------------------------------------------------------------------------- 2. NUMACTL Interleave case: numactl -i 0-1, -N 0-1 NUMAmaps below shows interleaved on all nodes instead of specified Numa memory nodes 0 and 1. ----------------------------------- GC HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- GC Options: -Xlog:gc*=info,gc+heap=debug -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [2216.439s][debug][gc,heap ] GC(159) Heap before GC invocations=160 (full 9): PSYoungGen total 120604672K, used 30143454K [0x00002b9d47c00000, 0x00002bba87c00000, 0x00002bba87c00000) [2216.439s][debug][gc,heap ] GC(159) eden space 118525952K, 24% used [0x00002b9d47c00000,0x00002ba458400000,0x00002bb98a000000) [2216.439s][debug][gc,heap ] GC(159) lgrp 0 space 14815232K, 98% used [0x00002b9d47c00000,0x00002ba0beb87c90,0x00002ba0d0000000) [2216.439s][debug][gc,heap ] GC(159) lgrp 1 space 14815232K, 100% used [0x00002ba0d0000000,0x00002ba458400000,0x00002ba458400000) ==> Memory allocated on following nodes are unused. [2216.439s][debug][gc,heap ] GC(159) lgrp 2 space 14815232K, 0% used [0x00002ba458400000,0x00002ba458400000,0x00002ba7e0800000) [2216.439s][debug][gc,heap ] GC(159) lgrp 3 space 14815232K, 0% used [0x00002ba7e0800000,0x00002ba7e0800000,0x00002bab68c00000) [2216.439s][debug][gc,heap ] GC(159) lgrp 4 space 14815232K, 0% used [0x00002bab68c00000,0x00002bab68c00000,0x00002baef1000000) [2216.439s][debug][gc,heap ] GC(159) lgrp 5 space 14815232K, 0% used [0x00002baef1000000,0x00002baef1000000,0x00002bb279400000) [2216.439s][debug][gc,heap ] GC(159) lgrp 6 space 14815232K, 0% used [0x00002bb279400000,0x00002bb279400000,0x00002bb601800000) [2216.439s][debug][gc,heap ] GC(159) lgrp 7 space 14819328K, 0% used [0x00002bb601800000,0x00002bb601800000,0x00002bb98a000000) [2216.439s][debug][gc,heap ] GC(159) from space 2078720K, 38% used [0x00002bba08e00000,0x00002bba3976fb70,0x00002bba87c00000) [2216.439s][debug][gc,heap ] GC(159) to space 2078720K, 0% used [0x00002bb98a000000,0x00002bb98a000000,0x00002bba08e00000) [2216.439s][debug][gc,heap ] GC(159) ParOldGen total 2097152K, used 685229K [0x00002b9cc7c00000, 0x00002b9d47c00000, 0x00002b9d47c00000) [2216.439s][debug][gc,heap ] GC(159) object space 2097152K, 32% used [0x00002b9cc7c00000,0x00002b9cf192b6e8,0x00002b9d47c00000) [2216.439s][debug][gc,heap ] GC(159) Metaspace used 28753K, capacity 29257K, committed 29440K, reserved 30720K ----------------------------------- GC HEAP END------------------------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------- PROCESS NUMA Maps -------------------------------------------------------------------------------------------------------------------------------------------------- Command : cat /proc/pid/numa_maps -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ==> Following addresses are interleaved on all the nodes. 2b9cb1992000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 N4=2 N5=2 N6=2 N7=2 kernelpagesize_kB=4 2b9cb19b2000 interleave:0-7 2b9cb3e65000 interleave:0-7 anon=50 dirty=50 N0=6 N1=6 N2=6 N3=6 N4=6 N5=7 N6=7 N7=6 kernelpagesize_kB=4 2b9cb8a69000 interleave:0-7 anon=8599 dirty=8599 N0=626 N1=1139 N2=1139 N3=1139 N4=1139 N5=1139 N6=1139 N7=1139 kernelpagesize_kB=4 2b9cc064f000 interleave:0-7 anon=6577 dirty=6577 N0=566 N1=566 N2=566 N3=1078 N4=1078 N5=1078 N6=1078 N7=567 kernelpagesize_kB=4 2b9cc7a69000 interleave:0-7 anon=50 dirty=50 N0=6 N1=7 N2=7 N3=6 N4=6 N5=6 N6=6 N7=6 kernelpagesize_kB=4 2b9cc7c00000 interleave:0-7 anon=524288 dirty=524288 N0=65536 N1=65536 N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 kernelpagesize_kB=4 2b9d47c00000 prefer:0 anon=2560 dirty=2560 N0=2560 kernelpagesize_kB=4 ==> Logical group 1 2ba0d0000000 prefer:1 ==> Logical group 2 2ba458400000 prefer:2 ==> This one and below all are unnecessary and leaving memory unused. 2ba7e0800000 prefer:3 2bab68c00000 prefer:4 2baef1000000 prefer:5 2bb279400000 prefer:6 2bb601800000 prefer:7 2bb98a000000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 N1=130048 N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 kernelpagesize_kB=4 2bba88000000 interleave:0-7 anon=60929 dirty=60929 N0=7745 N1=7744 N2=7744 N3=7744 N4=7744 N5=7232 N6=7744 N7=7232 kernelpagesize_kB=4 ---------------------------------- PROCESS NUMA Maps -------------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------- PROCESS status ----------------------------------------------------------------------------------------------------------------------------------------------------------- Command : cat /proc/pid/status ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Cpus_allowed: 00000000,0000ffff,00000000,0000ffff Cpus_allowed_list: 0-15,64-79 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff Mems_allowed_list: 0-7 --------------------------------- PROCESS status ----------------------------------------------------------------------------------------------------------------------------------------------------------- Please note ?Mems_allowed? and ?Mems_allowed_list? list also shows incorrect range. This could be LIBNUMA issue in case of membind. When application is externally bound to fewer nodes then calls to libNUMA API?s SHOULD FAIL to interleave on all nodes but right now it does not with currently used/called API?s in JDK. How patch works ? The patch gets bitmask structure by calling following API's (Man page definition for these functions are also given below). 1. For Membind : Use numa_get_membind to get membind bitmask (already used in the code) numa_get_membind() returns the mask of nodes from which memory can currently be allocated. If the returned mask is equal to numa_all_nodes, then memory allocation is allowed from all nodes. 2. For Interleave: use numa_get_interleave_mask to get interleave mask (currently not used/called in JDK) numa_get_interleave_mask() returns the current interleave mask if the task's memory allocation policy is page interleaved. Otherwise, this function returns an empty mask. Check node counts from both the Bitmasks to identify current running mode. When Interleave mask is EMPTY then it is running in membind else it is in interleave mode. Call ?numa_interleave_memory? (called indirectly through ?numa_make_global? funcition) function with right bitmask identified above helps to fix this issue. Improvement: This patch is tested on EPYC with SPECJBB benchmark and score improvements are given below. 1. For NUMACTL membind Max-jOPS improved by 5-12 % and Critical-jOPS by 2-6 % 2. For NUMACTL interleave (This patch fixes memory usage when invoked with numactl -i) Max-jOPS by 5-15% and Critical-jOPS by 11-100%. 3. With this fix, flag UseNUMAInterleaving turning on/off has no effect when externally interleaved through NUMACTL. 4. Flag UseNUMA score improved by ~14 % when enabled for single NUMA node. Currently it gets disabled when externally bound to single node. ---------------------- PATCH -------------------------- diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp +++ b/src/hotspot/os/linux/os_linux.cpp @@ -2720,6 +2720,8 @@ } void os::numa_make_global(char *addr, size_t bytes) { + if (!UseNUMAInterleaving) + return ; Linux::numa_interleave_memory(addr, bytes); } @@ -2785,6 +2787,15 @@ ids[i++] = node; } } + + if (Linux::_numa_interleave_ptr != NULL ) { + i = 0; + for (int node = 0; node <= highest_node_number; node++) { + if (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, node)) { + ids[i++] = node; + } + } + } return i; } @@ -2884,11 +2895,18 @@ libnuma_dlsym(handle, "numa_distance"))); set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, libnuma_v2_dlsym(handle, "numa_get_membind"))); + set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, + libnuma_v2_dlsym(handle, "numa_get_interleave_mask"))); if (numa_available() != -1) { + struct bitmask *bmp; set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); set_numa_all_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_all_nodes_ptr")); set_numa_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_nodes_ptr")); + bmp = _numa_get_interleave_mask(); + set_numa_interleave_ptr(&bmp); + bmp = _numa_get_membind(); + set_numa_membind_ptr(&bmp); // Create an index -> node mapping, since nodes are not always consecutive _nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) GrowableArray(0, true); rebuild_nindex_to_node_map(); @@ -3015,9 +3033,12 @@ os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; os::Linux::numa_distance_func_t os::Linux::_numa_distance; os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; +os::Linux::numa_get_interleave_mask_func_t os::Linux::_numa_get_interleave_mask; unsigned long* os::Linux::_numa_all_nodes; struct bitmask* os::Linux::_numa_all_nodes_ptr; struct bitmask* os::Linux::_numa_nodes_ptr; +struct bitmask* os::Linux::_numa_interleave_ptr; +struct bitmask* os::Linux::_numa_membind_ptr; bool os::pd_uncommit_memory(char* addr, size_t size) { uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, @@ -4997,13 +5018,38 @@ if (!Linux::libnuma_init()) { UseNUMA = false; } else { - if ((Linux::numa_max_node() < 1) || Linux::isbound_to_single_node()) { - // If there's only one node (they start from 0) or if the process - // is bound explicitly to a single node using membind, disable NUMA. - UseNUMA = false; + + // Identify whether running in membind or interleave mode. + struct bitmask *bmp; + bool _is_membind = false; + bool _is_interleaved = false; + + // Check for membind mode. + bmp = Linux::_numa_membind_ptr; + for (int node = 0; node <= Linux::numa_max_node() ; node++) { + if (Linux::_numa_bitmask_isbitset(bmp, node)) { + _is_membind = true; } } + // Check for interleave mode. + bmp = Linux::_numa_interleave_ptr; + for (int node = 0; node <= Linux::numa_max_node() ; node++) { + if (Linux::_numa_bitmask_isbitset(bmp, node)) { + _is_interleaved = true; + // Set membind to false as interleave mode allows all nodes to be used. + _is_membind = false; + } + } + + if (_is_membind) + Linux::set_numa_interleave_ptr (NULL); + + if (_is_interleaved) + Linux::set_numa_membind_ptr (NULL); + + } + if (UseParallelGC && UseNUMA && UseLargePages && !can_commit_large_page_memory()) { // With SHM and HugeTLBFS large pages we cannot uncommit a page, so there's no way // we can make the adaptive lgrp chunk resizing work. If the user specified both diff --git a/src/hotspot/os/linux/os_linux.hpp b/src/hotspot/os/linux/os_linux.hpp --- a/src/hotspot/os/linux/os_linux.hpp +++ b/src/hotspot/os/linux/os_linux.hpp @@ -222,6 +222,7 @@ typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); typedef struct bitmask* (*numa_get_membind_func_t)(void); + typedef struct bitmask* (*numa_get_interleave_mask_func_t)(void); typedef void (*numa_set_bind_policy_func_t)(int policy); typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); @@ -239,9 +240,12 @@ static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; static numa_distance_func_t _numa_distance; static numa_get_membind_func_t _numa_get_membind; + static numa_get_interleave_mask_func_t _numa_get_interleave_mask; static unsigned long* _numa_all_nodes; static struct bitmask* _numa_all_nodes_ptr; static struct bitmask* _numa_nodes_ptr; + static struct bitmask* _numa_interleave_ptr; + static struct bitmask* _numa_membind_ptr; static void set_sched_getcpu(sched_getcpu_func_t func) { _sched_getcpu = func; } static void set_numa_node_to_cpus(numa_node_to_cpus_func_t func) { _numa_node_to_cpus = func; } @@ -255,9 +259,12 @@ static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } + static void set_numa_get_interleave_mask(numa_get_interleave_mask_func_t func) { _numa_get_interleave_mask = func; } static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } + static void set_numa_interleave_ptr(struct bitmask **ptr) { _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } + static void set_numa_membind_ptr(struct bitmask **ptr) { _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } static int sched_getcpu_syscall(void); public: static int sched_getcpu() { return _sched_getcpu != NULL ? _sched_getcpu() : -1; } @@ -275,7 +282,10 @@ static void numa_interleave_memory(void *start, size_t size) { // Use v2 api if available if (_numa_interleave_memory_v2 != NULL && _numa_all_nodes_ptr != NULL) { - _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr); + if (_numa_interleave_ptr != NULL) + _numa_interleave_memory_v2(start, size, _numa_interleave_ptr); + else + _numa_interleave_memory_v2(start, size, _numa_membind_ptr); } else if (_numa_interleave_memory != NULL && _numa_all_nodes != NULL) { _numa_interleave_memory(start, size, _numa_all_nodes); } ----------------------- PATCH ----------------------------------- Thanks Amit Pawar From thomas.stuefe at gmail.com Fri Nov 9 14:19:36 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 9 Nov 2018 15:19:36 +0100 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Matthias, On Fri, Nov 9, 2018 at 11:08 AM Baesken, Matthias wrote: > > Hi Thomas , > > http://man7.org/linux/man-pages/man2/umask.2.html > > says : > > It is impossible to use umask() to fetch a process's umask without at the same time changing it. A second call to umask() would then be > needed to restore the umask. The nonatomicity of these two steps provides the potential for races in multithreaded programs. > > Since Linux 4.7, the umask of any process can be viewed via the Umask field of /proc/[pid]/status. Inspecting this field in > /proc/self/status allows a process to retrieve its umask without at the same time changing it. > > So it seems to be possible to just get the umask-info , but only on some OS versions . > > Don't know how relevant the small time window is, in case of hs_err file writing probably not so relevant. > Btw. there is a similar pattern here : > > jdk/src/java.prefs/unix/native/libprefs/FileSystemPreferences.c > > 94 old_umask = umask(0); > 95 fd = open(fname, O_WRONLY|O_CREAT, permission); > 96 result[1] = errno; > 97 umask(old_umask); > > thanks for checking up on that. I'm fine with that if we only do it in error reporting. BestRegards, Thomas > > Best regards, Matthias > > > > -----Original Message----- > > From: Thomas St?fe > > Sent: Freitag, 9. November 2018 10:08 > > To: Baesken, Matthias > > Cc: David Holmes ; Langer, Christoph > > ; Volker Simonis ; > > HotSpot Open Source Developers > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > Hi Matthias, > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > > posix/os_posix.hpp.udiff.html > > > > + static void print_user_info(outputStream* st, char* buf, size_t buflen); > > > > You can get rid of buf+buflen too, not needed anymore > > --- > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > > posix/os_posix.cpp.udiff.html > > > > In os::Posix::print_user_info(), I would probably now print it all one > > one like, e.g. "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: > > rwxr--r--" - for brevity. But I leave that up to you. > > --- > > > > The "removing" text does not make sense and can itself be removed :) > > > > -- > > > > I spotted a tiny issue which puts the whole print umask into question. > > Appearantly, there is no way for umask() to just return the current > > value of the file creation mask without changing it. So, that is why > > we restore it right afterwards. That is annoying. That leaves a tiny > > tiny time window where a different thread could create a file which > > would then have permissions all zero'd out. > > > > I am not sure how to deal with that. Could you look around whether > > there is another posix API to just look at the umask without changing > > it? Otherwise: does anyone think this is a problem? > > > > -- > > > > Sorry, turns out simple things are often more complicated than it > > first appears... > > > > Cheers, Thomas > > > > > > > > > > > > > > On Fri, Nov 9, 2018 at 9:45 AM Baesken, Matthias > > wrote: > > > > > > Hi Thomas, thank you for the remarks . > > > I introduced > > > > > > os::Posix::print_umask > > > > > > > > > > > But I remember that we did this in our > > > > port after some deliberation - there is a risk that name resolving may > > > > hang or crash, depending on how exactly user ids are resolved on that > > machine. > > > > > > > > > > ..and removed the getpwuid / getgrgid calls (so we only get the > > numeric ids in the output and not the user names / group names) > > > New webrev : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/ > > > > > > > > > Best regards, Matthias > > > > > > > > > > -----Original Message----- > > > > From: Thomas St?fe > > > > Sent: Donnerstag, 8. November 2018 21:00 > > > > To: Baesken, Matthias > > > > Cc: David Holmes ; Langer, Christoph > > > > ; Volker Simonis > > ; > > > > HotSpot Open Source Developers > > > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > > > > > Hi Matthias, > > > > > > > > this looks good in general. Two remarks: > > > > > > > > 1) For convenience reasons, we resolve the user name from its id (and > > > > group name etc) with getpwuid. But I remember that we did this in our > > > > port after some deliberation - there is a risk that name resolving may > > > > hang or crash, depending on how exactly user ids are resolved on that > > > > machine. For our port we decided to go with it, since our VM runs > > > > usually in very controlled environments, but for the OpenJDK I propose > > > > to remove that part from the patch and just print out the numeric > > > > values. > > > > > > > > 2) The part where you print out the mode_t umask as "rwxrwxrwx" is > > > > kind of neat. Maybe we could factor that out as an own function in > > > > os::Posix? e.g. os::Posix::print_umask(mode_t mask) ? > > > > > > > > I leave (2) up to you. We also can do this in a follow up patch. > > > > > > > > Thanks, Thomas > > > > > > > > > > > > On Thu, Nov 8, 2018 at 6:13 PM Baesken, Matthias > > > > wrote: > > > > > > > > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into > > jdk/jdk , > > > > I created another webrev : > > > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > > > > > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > > > > > > > Please review! > > > > > > > > > > > > > > > Thanks, Matthias > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Baesken, Matthias > > > > > > Sent: Dienstag, 9. Oktober 2018 16:55 > > > > > > To: 'Thomas St?fe' ; David Holmes > > > > > > > > > > > > Cc: Langer, Christoph ; Volker Simonis > > > > > > ; HotSpot Open Source Developers > > > > > > > > > dev at openjdk.java.net> > > > > > > Subject: RE: RFR : 8211326 : add OS user related information to hs_err > > file > > > > > > > > > > > > Hi Thomas , thanks for filing the CSR . > > > > > > > > > > > > I am fine with the name proposal "-XX:+-ExtensiveErrorReports", > > but > > > > will > > > > > > wait until it is agreed to use this name . > > > > > > Then I'll post another webrev of 8211326 using the flag . > > > > > > > > > > > > Best regards, Matthias > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Thomas St?fe > > > > > > > Sent: Montag, 8. Oktober 2018 12:21 > > > > > > > To: David Holmes > > > > > > > Cc: Baesken, Matthias ; Langer, > > > > Christoph > > > > > > > ; Volker Simonis > > > > > > ; > > > > > > > HotSpot Open Source Developers > dev at openjdk.java.net> > > > > > > > Subject: Re: RFR : 8211326 : add OS user related information to > > hs_err > > > > file > > > > > > > > > > > > > > On Mon, Oct 8, 2018 at 11:50 AM David Holmes > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > Hi Thomas, > > > > > > > > > > > > > > > > On 8/10/2018 7:43 PM, Thomas St?fe wrote: > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > I drafted a CSR: > > > > > > > > > > > > > > > > > > https://bugs.openjdk.java.net/browse/JDK-8211846 > > > > > > > > > > > > > > > > > > Not sure what to do now, should I open a discussion thread on a > > > > > > > > > mailing list? Or just wait for the committee to decide? (I'm doing > > > > > > > > > this too rarely..) > > > > > > > > > > > > > > > > I filled in some of the additional info needed. The text needs to be > > > > > > > > completed and the CSR needs to have at least one reviewer > > added to > > > > it. > > > > > > > > Then it can be marked "final" and ready for the committee to > > > > evaluate. > > > > > > > > > > > > > > > > > > > > > > Thanks! I filled out what was missing and will wait for reviewers. > > > > > > > > > > > > > > Thanks, Thomas > > > > > > > > > > > > From igor.ignatyev at oracle.com Fri Nov 9 16:25:49 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 9 Nov 2018 08:25:49 -0800 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <02269c23-e3aa-ba21-5ca1-1c2ae66b83fe@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <163111cd-b108-3add-d20f-fcf4e5e42dc7@oracle.com> <02269c23-e3aa-ba21-5ca1-1c2ae66b83fe@oracle.com> Message-ID: <7C44E025-1548-4F29-8381-6E1746435398@oracle.com> Vicente, Alan, back when we 1st bumped into this problem w/ ClassWriterExt (about 1y ago), it was my AI to discuss it w/ Remi. I sent him an email, didn't get a replay, and as it usually goes, had to switch to something else, completely forgot about that and didn't follow up. Thanks, -- Igor > On Nov 8, 2018, at 8:40 AM, Vicente Romero wrote: > > > > On 11/8/18 8:14 AM, Alan Bateman wrote: >> On 07/11/2018 19:33, Igor Ignatyev wrote: >>> Hi Vicente, >>> >>> I recall an (internal?) discussion about updating ASM somewhen in JDK 11TF, and AFAIR it was decided not to update ASM b/c nothing in JDK needs that, has it been changed? put somewhat differently, why are we doing this? >>> >>> in any case, I don't like the changes in mlvm tests. I understand that ClassWriter has been significantly changed in ASM 7.0, so ClassWriterExt can't disable CP entries caching (at least not in the way it used to), but removing setCache* calls from the tests changed them and in some cases made them invalid as they don't test that they supposed to. therefore I'd prefer to leave all calls setCache* as-is, change setCache* implementation to throw an exception (similarly to the fix in JDK-8194826 ) and mark all tests which throw this exception w/ '@ignore 8194951' jtreg tag. >>> >>> >> ClassWriterExt the MLVM tests have come in previous upgrades too. Has there been any discussion Remi or others on ASM to make it easier for the JDK to upgrade? > > I'm not aware of any such discussions. > >> >> -Alan > > Vicente From mvala at redhat.com Fri Nov 9 16:43:25 2018 From: mvala at redhat.com (Michal Vala) Date: Fri, 9 Nov 2018 17:43:25 +0100 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" Message-ID: Hi, please review following patch fixing build failure on Windows with VS2013 toolchain. http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ -- Michal Vala OpenJDK QE Red Hat Czech From kim.barrett at oracle.com Fri Nov 9 18:35:02 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 9 Nov 2018 13:35:02 -0500 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: Message-ID: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> > On Nov 9, 2018, at 11:43 AM, Michal Vala wrote: > > Hi, > > please review following patch fixing build failure on Windows with VS2013 toolchain. > > http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ The failure is in a hotspot test. It should be using os::snprintf. From forax at univ-mlv.fr Fri Nov 9 19:33:03 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 9 Nov 2018 20:33:03 +0100 (CET) Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <7C44E025-1548-4F29-8381-6E1746435398@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <163111cd-b108-3add-d20f-fcf4e5e42dc7@oracle.com> <02269c23-e3aa-ba21-5ca1-1c2ae66b83fe@oracle.com> <7C44E025-1548-4F29-8381-6E1746435398@oracle.com> Message-ID: <1300223702.1513924.1541791983615.JavaMail.zimbra@u-pem.fr> Hi Igor, sorry to have not answered to your mail :( I believe that the easiest way to disable the caching of the entries of the constant pool in ASM is to change SymbolTable.get() [1] to always return null. regards, R?mi [1] https://gitlab.ow2.org/asm/asm/blob/master/asm/src/main/java/org/objectweb/asm/SymbolTable.java#L393 ----- Mail original ----- > De: "Igor Ignatyev" > ?: "Vicente Romero" , "Alan Bateman" > Cc: "hotspot-dev" , "core-libs-dev" > Envoy?: Vendredi 9 Novembre 2018 17:25:49 > Objet: Re: RFR: JDK-8213480: update internal ASM version to 7.0 > Vicente, Alan, > > back when we 1st bumped into this problem w/ ClassWriterExt (about 1y ago), it > was my AI to discuss it w/ Remi. I sent him an email, didn't get a replay, and > as it usually goes, had to switch to something else, completely forgot about > that and didn't follow up. > > Thanks, > -- Igor > >> On Nov 8, 2018, at 8:40 AM, Vicente Romero wrote: >> >> >> >> On 11/8/18 8:14 AM, Alan Bateman wrote: >>> On 07/11/2018 19:33, Igor Ignatyev wrote: >>>> Hi Vicente, >>>> >>>> I recall an (internal?) discussion about updating ASM somewhen in JDK 11TF, and >>>> AFAIR it was decided not to update ASM b/c nothing in JDK needs that, has it >>>> been changed? put somewhat differently, why are we doing this? >>>> >>>> in any case, I don't like the changes in mlvm tests. I understand that >>>> ClassWriter has been significantly changed in ASM 7.0, so ClassWriterExt can't >>>> disable CP entries caching (at least not in the way it used to), but removing >>>> setCache* calls from the tests changed them and in some cases made them invalid >>>> as they don't test that they supposed to. therefore I'd prefer to leave all >>>> calls setCache* as-is, change setCache* implementation to throw an exception >>>> (similarly to the fix in JDK-8194826 >>>> ) and mark all tests which >>>> throw this exception w/ '@ignore 8194951' jtreg tag. >>>> >>>> >>> ClassWriterExt the MLVM tests have come in previous upgrades too. Has there been >>> any discussion Remi or others on ASM to make it easier for the JDK to upgrade? >> >> I'm not aware of any such discussions. >> >>> >>> -Alan >> > > Vicente From kim.barrett at oracle.com Fri Nov 9 20:02:22 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 9 Nov 2018 15:02:22 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> Message-ID: > On Nov 9, 2018, at 4:33 AM, Aleksey Shipilev wrote: > > On 11/09/2018 02:36 AM, Kim Barrett wrote: >>> I believe new patch is as straightforward as it could be, see: >>> http://cr.openjdk.java.net/~shade/8211926/webrev.03/ >> >> I was about ready to approve, but took another look at the new test. >> Unfortunately, I think it has some problems. > > Okay, let's try again after fixing the problems you mentioned: > http://cr.openjdk.java.net/~shade/8211926/webrev.04/ > > Moved STATIC_ASSERT and comments to the actual use site, rewritten test. Test takes 0.5s per TEST(). > Looks good now? > > -Aleksey Almost, but not quite, I'm afraid. All but the first comment below are optional, though I think make things better. ------------------------------------------------------------------------------ test/hotspot/gtest/utilities/test_bitMap_large.cpp 54 for (size_t size_class = 1; size_class <= BITMAP_SIZE; size_class *= 2) { 55 for (BitMap::idx_t l = 0; l < FUZZ_WINDOW; l++) { 56 for (BitMap::idx_t tr = l; tr < FUZZ_WINDOW; tr++) { 57 BitMap::idx_t r = MIN2(BITMAP_SIZE, size_class + tr); // avoid overflow I generally like the new test structure. Except there's a bug; this never tests the case of l == r. Maybe hoist the inner loop body into a helper function, and add an empty range call before the loop on tr. ------------------------------------------------------------------------------ test/hotspot/gtest/utilities/test_bitMap_large.cpp 29 static const BitMap::idx_t BITMAP_SIZE = 8192; Consider instead const BitMap::idx_t BITMAP_SIZE = 4 * BitMap::bit_index(BitMap::small_range_words); ------------------------------------------------------------------------------ test/hotspot/gtest/utilities/test_bitMap_large.cpp 34 static const BitMap::idx_t FUZZ_WINDOW = sizeof(BitMap::bm_word_t) * 8; Consider instead const BitMap::idx_t FUZZ_WINDOW = BitMap::bit_index(1); ------------------------------------------------------------------------------ test/hotspot/gtest/utilities/test_bitMap_large.cpp 54 for (size_t size_class = 1; size_class <= BITMAP_SIZE; size_class *= 2) { Using (size_class <= BITMAP_SIZE) rather than < ensures we hit the case of r == BITMAP_SIZE, which is good. I thought about other ways of doing that, but they all make the code more complicated in order to reduce the number of times we check that case, so probably not worth the trouble. But it's a somewhat subtle point, that might be worth a comment on the use of <= there. ------------------------------------------------------------------------------ src/hotspot/share/utilities/bitMap.cpp 272 // There is little point to call large version on small ranges. 273 // Need to check carefully, keeping potential idx_t underflow in mind. 274 // The threshold should be at least one word. 275 STATIC_ASSERT(small_range_words >= 1); 276 if (beg_full_word + small_range_words >= end_full_word) { Consider hoisting this out into a helper function for the two identical uses. ------------------------------------------------------------------------------ From vicente.romero at oracle.com Fri Nov 9 21:01:48 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 9 Nov 2018 16:01:48 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <0bb51977-09ec-a2f1-cae4-ee54df1c8a89@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> <0bb51977-09ec-a2f1-cae4-ee54df1c8a89@oracle.com> Message-ID: <3a7c3d41-a760-155a-9fd9-4759481a3bc5@oracle.com> Hi David, On 11/8/18 11:00 PM, David Holmes wrote: > Hi Vicente, > > On 9/11/2018 2:39 AM, Vicente Romero wrote: >> Hi David, Igor >> >> On 11/7/18 10:03 PM, David Holmes wrote: >>> Hi Vicente, >>> >>> All of the javadoc comment reformatting makes it nearly impossible >>> to see the actual substantive changes :( >>> >>> ASM 7 also supports the Nestmate attributes and I was trying to see >>> how/where that appeared but its somewhat obscure. Oh well. >>> >>> Is it that case that the code the uses the ASM library, like the JFR >>> code and jlink code, and the tests, doesn't actually _have to_ >>> change to specifying Opcodes.ASM7 unless they plan on using ASM7 >>> features? >> >> I changed only the tests for which the new ASM was complaining about >> a particular API available only for ASM7 > > I could not understand how this could be if the tests and other code > were unchanged, so I applied the ASM-only patch and investigated the > first failure running nasgen. And of course the problem is the > NestHost/NestMembers attributes! We modified our internal version of > ASM to add nestmate support, but of course this update removes that > and replaces it with the official support. But the official support is > only enabled for ASMv7 so we must update all users of ASM to request > version 7. thanks for double checking this > > Thanks, > David > ----- Vicente > >>> If so then you could split out the actual update of ASM from the >>> updates to the users of ASM (some of which may be quite fine with >>> ASM5). >> >> I have made two webrevs to make the review easier [1], contain only >> the changes to the internal asm and [2] contains the changes to the >> clients plus make files, legal, etc. I have also made the changes to >> ClassWriterExt and affected test proposed by Igor in another mail, >> >>> >>> Thanks, >>> David >> >> Thanks, >> Vicente >> >> [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.only.00/ >> [2] >> http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.additional.changes.00/ >> >>> >>> On 8/11/2018 1:56 AM, Vicente Romero wrote: >>>> Hi, >>>> >>>> Version 7.0 of ASM has been released. This version supports condy, >>>> yay!, and we want to include it in JDK 12. Please review [1] which >>>> includes: >>>> - the new version perse substituting the preview ASM internal >>>> version in the JDK >>>> - changes to additional files in particular some tests, mostly >>>> hotspot tests. >>>> >>>> Thanks, >>>> Vicente >>>> >>>> [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ >> From igor.ignatyev at oracle.com Fri Nov 9 21:12:48 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 9 Nov 2018 13:12:48 -0800 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <1300223702.1513924.1541791983615.JavaMail.zimbra@u-pem.fr> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <163111cd-b108-3add-d20f-fcf4e5e42dc7@oracle.com> <02269c23-e3aa-ba21-5ca1-1c2ae66b83fe@oracle.com> <7C44E025-1548-4F29-8381-6E1746435398@oracle.com> <1300223702.1513924.1541791983615.JavaMail.zimbra@u-pem.fr> Message-ID: <2C66B63B-919D-4348-81A4-BD42ACEC9B39@oracle.com> Hi Remi, w/ SymbolTable::get being private method _and_ SymbolTable being final class, it doesn't sound like the easiest way ;) w/ that being said, I haven't looked at the latest version of ASM, so there might be ways to get that we need. Thanks, -- Igor > On Nov 9, 2018, at 11:33 AM, Remi Forax wrote: > > Hi Igor, > sorry to have not answered to your mail :( > > I believe that the easiest way to disable the caching of the entries of the constant pool in ASM is to change SymbolTable.get() [1] to always return null. > > regards, > R?mi > > [1] https://gitlab.ow2.org/asm/asm/blob/master/asm/src/main/java/org/objectweb/asm/SymbolTable.java#L393 > > ----- Mail original ----- >> De: "Igor Ignatyev" >> ?: "Vicente Romero" , "Alan Bateman" >> Cc: "hotspot-dev" , "core-libs-dev" >> Envoy?: Vendredi 9 Novembre 2018 17:25:49 >> Objet: Re: RFR: JDK-8213480: update internal ASM version to 7.0 > >> Vicente, Alan, >> >> back when we 1st bumped into this problem w/ ClassWriterExt (about 1y ago), it >> was my AI to discuss it w/ Remi. I sent him an email, didn't get a replay, and >> as it usually goes, had to switch to something else, completely forgot about >> that and didn't follow up. >> >> Thanks, >> -- Igor >> >>> On Nov 8, 2018, at 8:40 AM, Vicente Romero wrote: >>> >>> >>> >>> On 11/8/18 8:14 AM, Alan Bateman wrote: >>>> On 07/11/2018 19:33, Igor Ignatyev wrote: >>>>> Hi Vicente, >>>>> >>>>> I recall an (internal?) discussion about updating ASM somewhen in JDK 11TF, and >>>>> AFAIR it was decided not to update ASM b/c nothing in JDK needs that, has it >>>>> been changed? put somewhat differently, why are we doing this? >>>>> >>>>> in any case, I don't like the changes in mlvm tests. I understand that >>>>> ClassWriter has been significantly changed in ASM 7.0, so ClassWriterExt can't >>>>> disable CP entries caching (at least not in the way it used to), but removing >>>>> setCache* calls from the tests changed them and in some cases made them invalid >>>>> as they don't test that they supposed to. therefore I'd prefer to leave all >>>>> calls setCache* as-is, change setCache* implementation to throw an exception >>>>> (similarly to the fix in JDK-8194826 >>>>> ) and mark all tests which >>>>> throw this exception w/ '@ignore 8194951' jtreg tag. >>>>> >>>>> >>>> ClassWriterExt the MLVM tests have come in previous upgrades too. Has there been >>>> any discussion Remi or others on ASM to make it easier for the JDK to upgrade? >>> >>> I'm not aware of any such discussions. >>> >>>> >>>> -Alan >>> >>> Vicente From zgu at redhat.com Fri Nov 9 21:43:54 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 9 Nov 2018 16:43:54 -0500 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name Message-ID: Please review this trivial fix. Currently, only Shenandoah experiences the failure, since other GCs run out Java heap memory, then the rest of test is skipped. E.g. pool[125131189]=new Object(); // elapsed 10.287s pool[126306334]=new Object(); // elapsed 7.947s Heap seems exhausted - OutOfMemoryError thrown. Skipped: ExceptionInInitializerError: thrown OutOfMemoryError Test passed. Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ Thanks, -Zhengyu From forax at univ-mlv.fr Fri Nov 9 22:11:53 2018 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 9 Nov 2018 23:11:53 +0100 (CET) Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <2C66B63B-919D-4348-81A4-BD42ACEC9B39@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <163111cd-b108-3add-d20f-fcf4e5e42dc7@oracle.com> <02269c23-e3aa-ba21-5ca1-1c2ae66b83fe@oracle.com> <7C44E025-1548-4F29-8381-6E1746435398@oracle.com> <1300223702.1513924.1541791983615.JavaMail.zimbra@u-pem.fr> <2C66B63B-919D-4348-81A4-BD42ACEC9B39@oracle.com> Message-ID: <2104140556.1523903.1541801513080.JavaMail.zimbra@u-pem.fr> It's not the simplest solution but you can use a classloader or an agent to patch the bytecode of ASM with ASM at runtime :) R?mi ----- Mail original ----- > De: "Igor Ignatyev" > ?: "Remi Forax" > Cc: "Vicente Romero" , "Alan Bateman" , "hotspot-dev" > , "core-libs-dev" > Envoy?: Vendredi 9 Novembre 2018 22:12:48 > Objet: Re: RFR: JDK-8213480: update internal ASM version to 7.0 > Hi Remi, > > > w/ SymbolTable::get being private method _and_ SymbolTable being final class, it > doesn't sound like the easiest way ;) w/ that being said, I haven't looked at > the latest version of ASM, so there might be ways to get that we need. > > Thanks, > -- Igor > >> On Nov 9, 2018, at 11:33 AM, Remi Forax wrote: >> >> Hi Igor, >> sorry to have not answered to your mail :( >> >> I believe that the easiest way to disable the caching of the entries of the >> constant pool in ASM is to change SymbolTable.get() [1] to always return null. >> >> regards, >> R?mi >> >> [1] >> https://gitlab.ow2.org/asm/asm/blob/master/asm/src/main/java/org/objectweb/asm/SymbolTable.java#L393 >> >> ----- Mail original ----- >>> De: "Igor Ignatyev" >>> ?: "Vicente Romero" , "Alan Bateman" >>> >>> Cc: "hotspot-dev" , "core-libs-dev" >>> >>> Envoy?: Vendredi 9 Novembre 2018 17:25:49 >>> Objet: Re: RFR: JDK-8213480: update internal ASM version to 7.0 >> >>> Vicente, Alan, >>> >>> back when we 1st bumped into this problem w/ ClassWriterExt (about 1y ago), it >>> was my AI to discuss it w/ Remi. I sent him an email, didn't get a replay, and >>> as it usually goes, had to switch to something else, completely forgot about >>> that and didn't follow up. >>> >>> Thanks, >>> -- Igor >>> >>>> On Nov 8, 2018, at 8:40 AM, Vicente Romero wrote: >>>> >>>> >>>> >>>> On 11/8/18 8:14 AM, Alan Bateman wrote: >>>>> On 07/11/2018 19:33, Igor Ignatyev wrote: >>>>>> Hi Vicente, >>>>>> >>>>>> I recall an (internal?) discussion about updating ASM somewhen in JDK 11TF, and >>>>>> AFAIR it was decided not to update ASM b/c nothing in JDK needs that, has it >>>>>> been changed? put somewhat differently, why are we doing this? >>>>>> >>>>>> in any case, I don't like the changes in mlvm tests. I understand that >>>>>> ClassWriter has been significantly changed in ASM 7.0, so ClassWriterExt can't >>>>>> disable CP entries caching (at least not in the way it used to), but removing >>>>>> setCache* calls from the tests changed them and in some cases made them invalid >>>>>> as they don't test that they supposed to. therefore I'd prefer to leave all >>>>>> calls setCache* as-is, change setCache* implementation to throw an exception >>>>>> (similarly to the fix in JDK-8194826 >>>>>> ) and mark all tests which >>>>>> throw this exception w/ '@ignore 8194951' jtreg tag. >>>>>> >>>>>> >>>>> ClassWriterExt the MLVM tests have come in previous upgrades too. Has there been >>>>> any discussion Remi or others on ASM to make it easier for the JDK to upgrade? >>>> >>>> I'm not aware of any such discussions. >>>> >>>>> >>>>> -Alan >>>> > >>> Vicente From david.holmes at oracle.com Fri Nov 9 22:33:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 10 Nov 2018 08:33:23 +1000 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: Message-ID: <5dd9d93c-9901-8756-d6be-c8c046726204@oracle.com> Hi Amith, I do not see you listed as a Contributor on the OCA list [1], nor can I tell from your email address whether you work for a company that is a signatory. Please confirm your status. Thanks, David [1] https://www.oracle.com/technetwork/community/oca-486395.html On 9/11/2018 10:23 PM, amith pawar wrote: > Hi all, > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, S1 > and S2 region (if any other ) memory areas > > on requested Numa nodes and it should not configure itself to access other > Numa nodes. This issue is observed only when Java > > is allowed to run on fewer NUMA nodes than available on the system with > numactl membind and interleave options. Running > > on all the nodes does not have any effect. This will cause some > applications (objects residing in old gen and survivor region) to > > run slower on system with large Numa nodes. > > > > I have described below in more detail for both numactl MEMBIND and > INTERLEAVE options. Addresses from both GC log and > > process numa maps clearly shows that the JAVA process is configured to > access other memory nodes even though it is not allowed. > > > > Two scenarios are: > > 1. Numactl membind case : numactl -m 0-1, -N 0-1 arguments> > > > Numa map shows these regions are INTERLEAVED ON ALL NODES instead of > specified Numa memory nodes 0 and 1. > > > > ----------------------------------- GC > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > GC Options: -Xlog:gc*=info,gc+heap=debug > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > [602.180s][debug][gc,heap ] GC(20) Heap before GC invocations=21 > (full 4): PSYoungGen total 120604672K, used 11907587K > [0x00002afc4b200000, 0x00002b198b200000, 0x00002b198b200000) > > [602.180s][debug][gc,heap ] GC(20) eden space 118525952K, 8% used > [0x00002afc4b200000,0x00002b0bb1b376e0,0x00002b188d600000) > > [602.180s][debug][gc,heap ] GC(20) lgrp 0 space 59262976K, 8% > used [0x00002afc4b200000,0x00002afd89bef450,0x00002b0a6c400000) > > [602.180s][debug][gc,heap ] GC(20) lgrp 1 space 59262976K, 8% > used [0x00002b0a6c400000,0x00002b0bb1b376e0,0x00002b188d600000) > > [602.180s][debug][gc,heap ] GC(20) from space 2078720K, 65% used > [0x00002b190c400000,0x00002b195ef5a0d0,0x00002b198b200000) > > [602.180s][debug][gc,heap ] GC(20) to space 2078720K, 0% used > [0x00002b188d600000,0x00002b188d600000,0x00002b190c400000) > > [602.180s][debug][gc,heap ] GC(20) ParOldGen total 2097152K, > used 226685K [0x00002afbcb200000, 0x00002afc4b200000, 0x00002afc4b200000) > > [602.180s][debug][gc,heap ] GC(20) object space 2097152K, 10% used > [0x00002afbcb200000,0x00002afbd8f5f460,0x00002afc4b200000) > > [602.180s][debug][gc,heap ] GC(20) Metaspace used 28462K, > capacity 29008K, committed 29184K, reserved 30720K > > ----------------------------------- GC HEAP > END-------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > ---------------------------------- PROCESS NUMA Maps > --------------------------------------------------------------------------------------------------------------------------------------------------- > > Command : cat /proc/4947/numa_maps > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > =======> Following addresses are interleaved on all nodes. > > 2afbb4f4c000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 N4=2 N5=2 > N6=2 N7=2 kernelpagesize_kB=4 > > 2afbb4f6c000 interleave:0-7 > > 2afbb7e88000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 N4=6 N5=6 > N6=6 N7=6 kernelpagesize_kB=4 > > 2afbbc000000 interleave:0-7 anon=8704 dirty=8704 N0=1600 N1=1088 N2=1088 > N3=576 N4=1088 N5=1088 N6=1088 N7=1088 kernelpagesize_kB=4 > > 2afbc3be6000 interleave:0-7 anon=6682 dirty=6682 N0=1027 N1=1027 N2=515 > N3=515 N4=515 N5=1027 N6=1028 N7=1028 kernelpagesize_kB=4 > > 2afbcb000000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 N4=6 N5=6 > N6=6 N7=6 kernelpagesize_kB=4 > > 2afbcb200000 interleave:0-7 anon=524288 dirty=524288 N0=65536 N1=65536 > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 kernelpagesize_kB=4 > > ==> OLD GEN Address > > 2afc4b200000 prefer:0 anon=1536 dirty=1536 N0=1536 kernelpagesize_kB=4 > > 2b0a6c400000 prefer:1 anon=512 dirty=512 N1=512 kernelpagesize_kB=4 > > 2b188d600000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 N1=130048 > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > kernelpagesize_kB=4 > > ==> Survivor Region > > 2b198b600000 interleave:0-7 anon=60929 dirty=60929 N0=7233 N1=7744 N2=7232 > N3=7744 N4=7744 N5=7744 N6=7744 N7=7744 kernelpagesize_kB=4 > > ---------------------------------- PROCESS NUMA Maps > ----------------------------------------------------------------------------------------------------------------------------------------------------- > > > > ---------------------------------- PROCESS status > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > Command : cat /proc/4947/status > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > Cpus_allowed_list: 0-15,64-79 > > Mems_allowed: > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > Mems_allowed_list: 0-7 > > --------------------------------- PROCESS status > ------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > 2. NUMACTL Interleave case: numactl -i 0-1, -N 0-1 its arguments> > > > NUMAmaps below shows interleaved on all nodes instead of specified Numa > memory nodes 0 and 1. > > ----------------------------------- GC > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > [2216.439s][debug][gc,heap ] GC(159) Heap before GC invocations=160 > (full 9): PSYoungGen total 120604672K, used 30143454K > [0x00002b9d47c00000, 0x00002bba87c00000, 0x00002bba87c00000) > > [2216.439s][debug][gc,heap ] GC(159) eden space 118525952K, 24% > used [0x00002b9d47c00000,0x00002ba458400000,0x00002bb98a000000) > > [2216.439s][debug][gc,heap ] GC(159) lgrp 0 space 14815232K, 98% > used [0x00002b9d47c00000,0x00002ba0beb87c90,0x00002ba0d0000000) > > [2216.439s][debug][gc,heap ] GC(159) lgrp 1 space 14815232K, > 100% used [0x00002ba0d0000000,0x00002ba458400000,0x00002ba458400000) > > ==> Memory allocated on following nodes are unused. > > [2216.439s][debug][gc,heap ] GC(159) lgrp 2 space 14815232K, 0% > used [0x00002ba458400000,0x00002ba458400000,0x00002ba7e0800000) > > [2216.439s][debug][gc,heap ] GC(159) lgrp 3 space 14815232K, 0% > used [0x00002ba7e0800000,0x00002ba7e0800000,0x00002bab68c00000) > > [2216.439s][debug][gc,heap ] GC(159) lgrp 4 space 14815232K, 0% > used [0x00002bab68c00000,0x00002bab68c00000,0x00002baef1000000) > > [2216.439s][debug][gc,heap ] GC(159) lgrp 5 space 14815232K, 0% > used [0x00002baef1000000,0x00002baef1000000,0x00002bb279400000) > > [2216.439s][debug][gc,heap ] GC(159) lgrp 6 space 14815232K, 0% > used [0x00002bb279400000,0x00002bb279400000,0x00002bb601800000) > > [2216.439s][debug][gc,heap ] GC(159) lgrp 7 space 14819328K, 0% > used [0x00002bb601800000,0x00002bb601800000,0x00002bb98a000000) > > [2216.439s][debug][gc,heap ] GC(159) from space 2078720K, 38% used > [0x00002bba08e00000,0x00002bba3976fb70,0x00002bba87c00000) > > [2216.439s][debug][gc,heap ] GC(159) to space 2078720K, 0% used > [0x00002bb98a000000,0x00002bb98a000000,0x00002bba08e00000) > > [2216.439s][debug][gc,heap ] GC(159) ParOldGen total > 2097152K, used 685229K [0x00002b9cc7c00000, 0x00002b9d47c00000, > 0x00002b9d47c00000) > > [2216.439s][debug][gc,heap ] GC(159) object space 2097152K, 32% > used [0x00002b9cc7c00000,0x00002b9cf192b6e8,0x00002b9d47c00000) > > [2216.439s][debug][gc,heap ] GC(159) Metaspace used 28753K, > capacity 29257K, committed 29440K, reserved 30720K > > ----------------------------------- GC HEAP > END------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > ---------------------------------- PROCESS NUMA Maps > -------------------------------------------------------------------------------------------------------------------------------------------------- > > Command : cat /proc/pid/numa_maps > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > ==> Following addresses are interleaved on all the nodes. > > 2b9cb1992000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 N4=2 N5=2 > N6=2 N7=2 kernelpagesize_kB=4 > > 2b9cb19b2000 interleave:0-7 > > 2b9cb3e65000 interleave:0-7 anon=50 dirty=50 N0=6 N1=6 N2=6 N3=6 N4=6 N5=7 > N6=7 N7=6 kernelpagesize_kB=4 > > 2b9cb8a69000 interleave:0-7 anon=8599 dirty=8599 N0=626 N1=1139 N2=1139 > N3=1139 N4=1139 N5=1139 N6=1139 N7=1139 kernelpagesize_kB=4 > > 2b9cc064f000 interleave:0-7 anon=6577 dirty=6577 N0=566 N1=566 N2=566 > N3=1078 N4=1078 N5=1078 N6=1078 N7=567 kernelpagesize_kB=4 > > 2b9cc7a69000 interleave:0-7 anon=50 dirty=50 N0=6 N1=7 N2=7 N3=6 N4=6 N5=6 > N6=6 N7=6 kernelpagesize_kB=4 > > 2b9cc7c00000 interleave:0-7 anon=524288 dirty=524288 N0=65536 N1=65536 > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 kernelpagesize_kB=4 > > 2b9d47c00000 prefer:0 anon=2560 dirty=2560 N0=2560 kernelpagesize_kB=4 > ==> Logical group 1 > > 2ba0d0000000 > prefer:1 > ==> Logical group 2 > > 2ba458400000 prefer:2 > > ==> This one and below all are unnecessary and leaving memory unused. > > 2ba7e0800000 prefer:3 > > 2bab68c00000 prefer:4 > > 2baef1000000 prefer:5 > > 2bb279400000 prefer:6 > > 2bb601800000 prefer:7 > > 2bb98a000000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 N1=130048 > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > kernelpagesize_kB=4 > > 2bba88000000 interleave:0-7 anon=60929 dirty=60929 N0=7745 N1=7744 N2=7744 > N3=7744 N4=7744 N5=7232 N6=7744 N7=7232 kernelpagesize_kB=4 > > ---------------------------------- PROCESS NUMA Maps > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > --------------------------------- PROCESS status > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > Command : cat /proc/pid/status > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > Cpus_allowed_list: 0-15,64-79 > > Mems_allowed: > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > Mems_allowed_list: 0-7 > > --------------------------------- PROCESS status > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > Please note ?Mems_allowed? and ?Mems_allowed_list? list also shows > incorrect range. This could be LIBNUMA issue in case of membind. > > When application is externally bound to fewer nodes then calls to libNUMA > API?s SHOULD FAIL to interleave on all nodes but right now it > > does not with currently used/called API?s in JDK. > > > > How patch works ? > > The patch gets bitmask structure by calling following API's (Man page > definition for these functions are also given below). > > 1. For Membind : Use numa_get_membind to get membind bitmask (already used > in the code) > > > numa_get_membind() returns the mask of nodes from which memory can > currently be allocated. If the returned mask is equal to numa_all_nodes, > then memory allocation is allowed from all nodes. > > 2. For Interleave: use numa_get_interleave_mask to get interleave mask > (currently not used/called in JDK) > numa_get_interleave_mask() returns the current interleave mask if the > task's memory allocation policy is page interleaved. Otherwise, this > function returns an empty mask. > > > > Check node counts from both the Bitmasks to identify current running mode. > When Interleave mask is EMPTY then it is running in membind else it is in > interleave mode. > > Call ?numa_interleave_memory? (called indirectly through ?numa_make_global? > funcition) function with right bitmask identified above helps to fix this > issue. > > > > > > Improvement: > > This patch is tested on EPYC with SPECJBB benchmark and score improvements > are given below. > > 1. For NUMACTL membind > > Max-jOPS improved by 5-12 % and Critical-jOPS by 2-6 % > > 2. For NUMACTL interleave (This patch fixes memory usage when invoked with > numactl -i) > > Max-jOPS by 5-15% and Critical-jOPS by 11-100%. > > 3. With this fix, flag UseNUMAInterleaving turning on/off has no effect > when externally interleaved through NUMACTL. > > 4. Flag UseNUMA score improved by ~14 % when enabled for single NUMA node. > Currently it gets disabled when externally bound to single node. > > > > > > ---------------------- PATCH -------------------------- > diff --git a/src/hotspot/os/linux/os_linux.cpp > b/src/hotspot/os/linux/os_linux.cpp > --- a/src/hotspot/os/linux/os_linux.cpp > +++ b/src/hotspot/os/linux/os_linux.cpp > @@ -2720,6 +2720,8 @@ > } > > void os::numa_make_global(char *addr, size_t bytes) { > + if (!UseNUMAInterleaving) > + return ; > Linux::numa_interleave_memory(addr, bytes); > } > > @@ -2785,6 +2787,15 @@ > ids[i++] = node; > } > } > + > + if (Linux::_numa_interleave_ptr != NULL ) { > + i = 0; > + for (int node = 0; node <= highest_node_number; node++) { > + if (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, > node)) { > + ids[i++] = node; > + } > + } > + } > return i; > } > > @@ -2884,11 +2895,18 @@ > libnuma_dlsym(handle, > "numa_distance"))); > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > libnuma_v2_dlsym(handle, > "numa_get_membind"))); > + > set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, > + libnuma_v2_dlsym(handle, > "numa_get_interleave_mask"))); > > if (numa_available() != -1) { > + struct bitmask *bmp; > set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > "numa_all_nodes")); > set_numa_all_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, > "numa_all_nodes_ptr")); > set_numa_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, > "numa_nodes_ptr")); > + bmp = _numa_get_interleave_mask(); > + set_numa_interleave_ptr(&bmp); > + bmp = _numa_get_membind(); > + set_numa_membind_ptr(&bmp); > // Create an index -> node mapping, since nodes are not always > consecutive > _nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) > GrowableArray(0, true); > rebuild_nindex_to_node_map(); > @@ -3015,9 +3033,12 @@ > os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; > os::Linux::numa_distance_func_t os::Linux::_numa_distance; > os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > +os::Linux::numa_get_interleave_mask_func_t > os::Linux::_numa_get_interleave_mask; > unsigned long* os::Linux::_numa_all_nodes; > struct bitmask* os::Linux::_numa_all_nodes_ptr; > struct bitmask* os::Linux::_numa_nodes_ptr; > +struct bitmask* os::Linux::_numa_interleave_ptr; > +struct bitmask* os::Linux::_numa_membind_ptr; > > bool os::pd_uncommit_memory(char* addr, size_t size) { > uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, > @@ -4997,13 +5018,38 @@ > if (!Linux::libnuma_init()) { > UseNUMA = false; > } else { > - if ((Linux::numa_max_node() < 1) || Linux::isbound_to_single_node()) > { > - // If there's only one node (they start from 0) or if the process > - // is bound explicitly to a single node using membind, disable > NUMA. > - UseNUMA = false; > + > + // Identify whether running in membind or interleave mode. > + struct bitmask *bmp; > + bool _is_membind = false; > + bool _is_interleaved = false; > + > + // Check for membind mode. > + bmp = Linux::_numa_membind_ptr; > + for (int node = 0; node <= Linux::numa_max_node() ; node++) { > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > + _is_membind = true; > } > } > > + // Check for interleave mode. > + bmp = Linux::_numa_interleave_ptr; > + for (int node = 0; node <= Linux::numa_max_node() ; node++) { > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > + _is_interleaved = true; > + // Set membind to false as interleave mode allows all nodes to be > used. > + _is_membind = false; > + } > + } > + > + if (_is_membind) > + Linux::set_numa_interleave_ptr (NULL); > + > + if (_is_interleaved) > + Linux::set_numa_membind_ptr (NULL); > + > + } > + > if (UseParallelGC && UseNUMA && UseLargePages && > !can_commit_large_page_memory()) { > // With SHM and HugeTLBFS large pages we cannot uncommit a page, so > there's no way > // we can make the adaptive lgrp chunk resizing work. If the user > specified both > diff --git a/src/hotspot/os/linux/os_linux.hpp > b/src/hotspot/os/linux/os_linux.hpp > --- a/src/hotspot/os/linux/os_linux.hpp > +++ b/src/hotspot/os/linux/os_linux.hpp > @@ -222,6 +222,7 @@ > typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, > unsigned long *nodemask); > typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t > size, struct bitmask* mask); > typedef struct bitmask* (*numa_get_membind_func_t)(void); > + typedef struct bitmask* (*numa_get_interleave_mask_func_t)(void); > > typedef void (*numa_set_bind_policy_func_t)(int policy); > typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > unsigned int n); > @@ -239,9 +240,12 @@ > static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > static numa_distance_func_t _numa_distance; > static numa_get_membind_func_t _numa_get_membind; > + static numa_get_interleave_mask_func_t _numa_get_interleave_mask; > static unsigned long* _numa_all_nodes; > static struct bitmask* _numa_all_nodes_ptr; > static struct bitmask* _numa_nodes_ptr; > + static struct bitmask* _numa_interleave_ptr; > + static struct bitmask* _numa_membind_ptr; > > static void set_sched_getcpu(sched_getcpu_func_t func) { _sched_getcpu = > func; } > static void set_numa_node_to_cpus(numa_node_to_cpus_func_t func) { > _numa_node_to_cpus = func; } > @@ -255,9 +259,12 @@ > static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) > { _numa_bitmask_isbitset = func; } > static void set_numa_distance(numa_distance_func_t func) { > _numa_distance = func; } > static void set_numa_get_membind(numa_get_membind_func_t func) { > _numa_get_membind = func; } > + static void set_numa_get_interleave_mask(numa_get_interleave_mask_func_t > func) { _numa_get_interleave_mask = func; } > static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = > ptr; } > static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = > (ptr == NULL ? NULL : *ptr); } > + static void set_numa_interleave_ptr(struct bitmask **ptr) { > _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } > + static void set_numa_membind_ptr(struct bitmask **ptr) { > _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } > static int sched_getcpu_syscall(void); > public: > static int sched_getcpu() { return _sched_getcpu != NULL ? > _sched_getcpu() : -1; } > @@ -275,7 +282,10 @@ > static void numa_interleave_memory(void *start, size_t size) { > // Use v2 api if available > if (_numa_interleave_memory_v2 != NULL && _numa_all_nodes_ptr != NULL) > { > - _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr); > + if (_numa_interleave_ptr != NULL) > + _numa_interleave_memory_v2(start, size, _numa_interleave_ptr); > + else > + _numa_interleave_memory_v2(start, size, _numa_membind_ptr); > } else if (_numa_interleave_memory != NULL && _numa_all_nodes != NULL) > { > _numa_interleave_memory(start, size, _numa_all_nodes); > } > ----------------------- PATCH ----------------------------------- > > > Thanks > Amit Pawar > From david.holmes at oracle.com Fri Nov 9 22:58:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 10 Nov 2018 08:58:12 +1000 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: References: Message-ID: Looks good - thanks for finding and fixing. I see cases internally where we fail because of this: Failure: ExceptionInInitializerError: target class not found but no bug was ever filed. I wonder if we will now fail because of the OOME ... Thanks, David On 10/11/2018 7:43 AM, Zhengyu Gu wrote: > Please review this trivial fix. > > Currently, only Shenandoah experiences the failure, since other GCs run > out Java heap memory, then the rest of test is skipped. > > E.g. > pool[125131189]=new Object(); // elapsed 10.287s > pool[126306334]=new Object(); // elapsed 7.947s > Heap seems exhausted - OutOfMemoryError thrown. > Skipped: ExceptionInInitializerError: thrown OutOfMemoryError > Test passed. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 > Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ > > Thanks, > > -Zhengyu From david.holmes at oracle.com Fri Nov 9 23:07:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 10 Nov 2018 09:07:34 +1000 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> Message-ID: cc'ing JC Beyler as this was his code. On 10/11/2018 4:35 AM, Kim Barrett wrote: >> On Nov 9, 2018, at 11:43 AM, Michal Vala wrote: >> >> Hi, >> >> please review following patch fixing build failure on Windows with VS2013 toolchain. Not sure we're still supporting VS2013 ... ?? >> >> http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ > > The failure is in a hotspot test. It should be using os::snprintf. Tests don't have access to os::snprintf. The test is just C++ code. Cheers, David From shade at redhat.com Fri Nov 9 23:31:09 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 10 Nov 2018 00:31:09 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> Message-ID: <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> On 11/09/2018 09:02 PM, Kim Barrett wrote: > Almost, but not quite, I'm afraid. All but the first comment below > are optional, though I think make things better. Introduced helper method, moved the small_range_words to private, handled (l == r) corner case in test. See: http://cr.openjdk.java.net/~shade/8211926/webrev.05/ -Aleksey P.S. For the love of any deity of choice, can we please push this corner-case fix and move on to solve meatier bugs? I appreciate the attention to details, but the economics of spending any more time to this issue feels dubious to me. From amith.pawar at gmail.com Sat Nov 10 00:34:23 2018 From: amith.pawar at gmail.com (amith pawar) Date: Sat, 10 Nov 2018 06:04:23 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: <5dd9d93c-9901-8756-d6be-c8c046726204@oracle.com> References: <5dd9d93c-9901-8756-d6be-c8c046726204@oracle.com> Message-ID: Hi David, I have initiated for OCA processing and working as an individual contributor. Thanks, Amit Pawar On Sat, Nov 10, 2018, 4:03 AM David Holmes Hi Amith, > > I do not see you listed as a Contributor on the OCA list [1], nor can I > tell from your email address whether you work for a company that is a > signatory. Please confirm your status. > > Thanks, > David > > > [1] https://www.oracle.com/technetwork/community/oca-486395.html > > On 9/11/2018 10:23 PM, amith pawar wrote: > > Hi all, > > > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, S1 > > and S2 region (if any other ) memory areas > > > > on requested Numa nodes and it should not configure itself to access > other > > Numa nodes. This issue is observed only when Java > > > > is allowed to run on fewer NUMA nodes than available on the system with > > numactl membind and interleave options. Running > > > > on all the nodes does not have any effect. This will cause some > > applications (objects residing in old gen and survivor region) to > > > > run slower on system with large Numa nodes. > > > > > > > > I have described below in more detail for both numactl MEMBIND and > > INTERLEAVE options. Addresses from both GC log and > > > > process numa maps clearly shows that the JAVA process is configured to > > access other memory nodes even though it is not allowed. > > > > > > > > Two scenarios are: > > > > 1. Numactl membind case : numactl -m 0-1, -N 0-1 its > > arguments> > > > > > > Numa map shows these regions are INTERLEAVED ON ALL NODES instead of > > specified Numa memory nodes 0 and 1. > > > > > > > > ----------------------------------- GC > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > [602.180s][debug][gc,heap ] GC(20) Heap before GC invocations=21 > > (full 4): PSYoungGen total 120604672K, used 11907587K > > [0x00002afc4b200000, 0x00002b198b200000, 0x00002b198b200000) > > > > [602.180s][debug][gc,heap ] GC(20) eden space 118525952K, 8% > used > > [0x00002afc4b200000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > [602.180s][debug][gc,heap ] GC(20) lgrp 0 space 59262976K, 8% > > used [0x00002afc4b200000,0x00002afd89bef450,0x00002b0a6c400000) > > > > [602.180s][debug][gc,heap ] GC(20) lgrp 1 space 59262976K, 8% > > used [0x00002b0a6c400000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > [602.180s][debug][gc,heap ] GC(20) from space 2078720K, 65% used > > [0x00002b190c400000,0x00002b195ef5a0d0,0x00002b198b200000) > > > > [602.180s][debug][gc,heap ] GC(20) to space 2078720K, 0% used > > [0x00002b188d600000,0x00002b188d600000,0x00002b190c400000) > > > > [602.180s][debug][gc,heap ] GC(20) ParOldGen total > 2097152K, > > used 226685K [0x00002afbcb200000, 0x00002afc4b200000, 0x00002afc4b200000) > > > > [602.180s][debug][gc,heap ] GC(20) object space 2097152K, 10% > used > > [0x00002afbcb200000,0x00002afbd8f5f460,0x00002afc4b200000) > > > > [602.180s][debug][gc,heap ] GC(20) Metaspace used 28462K, > > capacity 29008K, committed 29184K, reserved 30720K > > > > ----------------------------------- GC HEAP > > > END-------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > ---------------------------------- PROCESS NUMA Maps > > > --------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/4947/numa_maps > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > =======> Following addresses are interleaved on all nodes. > > > > 2afbb4f4c000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 N4=2 > N5=2 > > N6=2 N7=2 kernelpagesize_kB=4 > > > > 2afbb4f6c000 interleave:0-7 > > > > 2afbb7e88000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 N4=6 > N5=6 > > N6=6 N7=6 kernelpagesize_kB=4 > > > > 2afbbc000000 interleave:0-7 anon=8704 dirty=8704 N0=1600 N1=1088 N2=1088 > > N3=576 N4=1088 N5=1088 N6=1088 N7=1088 kernelpagesize_kB=4 > > > > 2afbc3be6000 interleave:0-7 anon=6682 dirty=6682 N0=1027 N1=1027 N2=515 > > N3=515 N4=515 N5=1027 N6=1028 N7=1028 kernelpagesize_kB=4 > > > > 2afbcb000000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 N4=6 > N5=6 > > N6=6 N7=6 kernelpagesize_kB=4 > > > > 2afbcb200000 interleave:0-7 anon=524288 dirty=524288 N0=65536 N1=65536 > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 kernelpagesize_kB=4 > > > > ==> OLD GEN Address > > > > 2afc4b200000 prefer:0 anon=1536 dirty=1536 N0=1536 kernelpagesize_kB=4 > > > > 2b0a6c400000 prefer:1 anon=512 dirty=512 N1=512 kernelpagesize_kB=4 > > > > 2b188d600000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 > N1=130048 > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > kernelpagesize_kB=4 > > > > ==> Survivor Region > > > > 2b198b600000 interleave:0-7 anon=60929 dirty=60929 N0=7233 N1=7744 > N2=7232 > > N3=7744 N4=7744 N5=7744 N6=7744 N7=7744 kernelpagesize_kB=4 > > > > ---------------------------------- PROCESS NUMA Maps > > > ----------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > ---------------------------------- PROCESS status > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/4947/status > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > > > Cpus_allowed_list: 0-15,64-79 > > > > Mems_allowed: > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > Mems_allowed_list: 0-7 > > > > --------------------------------- PROCESS status > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > 2. NUMACTL Interleave case: numactl -i 0-1, -N 0-1 > its arguments> > > > > > > NUMAmaps below shows interleaved on all nodes instead of specified Numa > > memory nodes 0 and 1. > > > > ----------------------------------- GC > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > [2216.439s][debug][gc,heap ] GC(159) Heap before GC > invocations=160 > > (full 9): PSYoungGen total 120604672K, used 30143454K > > [0x00002b9d47c00000, 0x00002bba87c00000, 0x00002bba87c00000) > > > > [2216.439s][debug][gc,heap ] GC(159) eden space 118525952K, 24% > > used [0x00002b9d47c00000,0x00002ba458400000,0x00002bb98a000000) > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 0 space 14815232K, > 98% > > used [0x00002b9d47c00000,0x00002ba0beb87c90,0x00002ba0d0000000) > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 1 space 14815232K, > > 100% used [0x00002ba0d0000000,0x00002ba458400000,0x00002ba458400000) > > > > ==> Memory allocated on following nodes are unused. > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 2 space 14815232K, > 0% > > used [0x00002ba458400000,0x00002ba458400000,0x00002ba7e0800000) > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 3 space 14815232K, > 0% > > used [0x00002ba7e0800000,0x00002ba7e0800000,0x00002bab68c00000) > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 4 space 14815232K, > 0% > > used [0x00002bab68c00000,0x00002bab68c00000,0x00002baef1000000) > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 5 space 14815232K, > 0% > > used [0x00002baef1000000,0x00002baef1000000,0x00002bb279400000) > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 6 space 14815232K, > 0% > > used [0x00002bb279400000,0x00002bb279400000,0x00002bb601800000) > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 7 space 14819328K, > 0% > > used [0x00002bb601800000,0x00002bb601800000,0x00002bb98a000000) > > > > [2216.439s][debug][gc,heap ] GC(159) from space 2078720K, 38% > used > > [0x00002bba08e00000,0x00002bba3976fb70,0x00002bba87c00000) > > > > [2216.439s][debug][gc,heap ] GC(159) to space 2078720K, 0% > used > > [0x00002bb98a000000,0x00002bb98a000000,0x00002bba08e00000) > > > > [2216.439s][debug][gc,heap ] GC(159) ParOldGen total > > 2097152K, used 685229K [0x00002b9cc7c00000, 0x00002b9d47c00000, > > 0x00002b9d47c00000) > > > > [2216.439s][debug][gc,heap ] GC(159) object space 2097152K, 32% > > used [0x00002b9cc7c00000,0x00002b9cf192b6e8,0x00002b9d47c00000) > > > > [2216.439s][debug][gc,heap ] GC(159) Metaspace used 28753K, > > capacity 29257K, committed 29440K, reserved 30720K > > > > ----------------------------------- GC HEAP > > > END------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > ---------------------------------- PROCESS NUMA Maps > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/pid/numa_maps > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > ==> Following addresses are interleaved on all the nodes. > > > > 2b9cb1992000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 N4=2 > N5=2 > > N6=2 N7=2 kernelpagesize_kB=4 > > > > 2b9cb19b2000 interleave:0-7 > > > > 2b9cb3e65000 interleave:0-7 anon=50 dirty=50 N0=6 N1=6 N2=6 N3=6 N4=6 > N5=7 > > N6=7 N7=6 kernelpagesize_kB=4 > > > > 2b9cb8a69000 interleave:0-7 anon=8599 dirty=8599 N0=626 N1=1139 N2=1139 > > N3=1139 N4=1139 N5=1139 N6=1139 N7=1139 kernelpagesize_kB=4 > > > > 2b9cc064f000 interleave:0-7 anon=6577 dirty=6577 N0=566 N1=566 N2=566 > > N3=1078 N4=1078 N5=1078 N6=1078 N7=567 kernelpagesize_kB=4 > > > > 2b9cc7a69000 interleave:0-7 anon=50 dirty=50 N0=6 N1=7 N2=7 N3=6 N4=6 > N5=6 > > N6=6 N7=6 kernelpagesize_kB=4 > > > > 2b9cc7c00000 interleave:0-7 anon=524288 dirty=524288 N0=65536 N1=65536 > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 kernelpagesize_kB=4 > > > > 2b9d47c00000 prefer:0 anon=2560 dirty=2560 N0=2560 kernelpagesize_kB=4 > > ==> Logical group 1 > > > > 2ba0d0000000 > > prefer:1 > > ==> Logical group 2 > > > > 2ba458400000 prefer:2 > > > > ==> This one and below all are unnecessary and leaving memory unused. > > > > 2ba7e0800000 prefer:3 > > > > 2bab68c00000 prefer:4 > > > > 2baef1000000 prefer:5 > > > > 2bb279400000 prefer:6 > > > > 2bb601800000 prefer:7 > > > > 2bb98a000000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 > N1=130048 > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > kernelpagesize_kB=4 > > > > 2bba88000000 interleave:0-7 anon=60929 dirty=60929 N0=7745 N1=7744 > N2=7744 > > N3=7744 N4=7744 N5=7232 N6=7744 N7=7232 kernelpagesize_kB=4 > > > > ---------------------------------- PROCESS NUMA Maps > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > --------------------------------- PROCESS status > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/pid/status > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > > > Cpus_allowed_list: 0-15,64-79 > > > > Mems_allowed: > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > Mems_allowed_list: 0-7 > > > > --------------------------------- PROCESS status > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > Please note ?Mems_allowed? and ?Mems_allowed_list? list also shows > > incorrect range. This could be LIBNUMA issue in case of membind. > > > > When application is externally bound to fewer nodes then calls to libNUMA > > API?s SHOULD FAIL to interleave on all nodes but right now it > > > > does not with currently used/called API?s in JDK. > > > > > > > > How patch works ? > > > > The patch gets bitmask structure by calling following API's (Man page > > definition for these functions are also given below). > > > > 1. For Membind : Use numa_get_membind to get membind bitmask (already > used > > in the code) > > > > > > numa_get_membind() returns the mask of nodes from which memory can > > currently be allocated. If the returned mask is equal to numa_all_nodes, > > then memory allocation is allowed from all nodes. > > > > 2. For Interleave: use numa_get_interleave_mask to get interleave mask > > (currently not used/called in JDK) > > numa_get_interleave_mask() returns the current interleave mask if > the > > task's memory allocation policy is page interleaved. Otherwise, this > > function returns an empty mask. > > > > > > > > Check node counts from both the Bitmasks to identify current running > mode. > > When Interleave mask is EMPTY then it is running in membind else it is in > > interleave mode. > > > > Call ?numa_interleave_memory? (called indirectly through > ?numa_make_global? > > funcition) function with right bitmask identified above helps to fix this > > issue. > > > > > > > > > > > > Improvement: > > > > This patch is tested on EPYC with SPECJBB benchmark and score > improvements > > are given below. > > > > 1. For NUMACTL membind > > > > Max-jOPS improved by 5-12 % and Critical-jOPS by 2-6 % > > > > 2. For NUMACTL interleave (This patch fixes memory usage when invoked > with > > numactl -i) > > > > Max-jOPS by 5-15% and Critical-jOPS by 11-100%. > > > > 3. With this fix, flag UseNUMAInterleaving turning on/off has no effect > > when externally interleaved through NUMACTL. > > > > 4. Flag UseNUMA score improved by ~14 % when enabled for single NUMA > node. > > Currently it gets disabled when externally bound to single node. > > > > > > > > > > > > ---------------------- PATCH -------------------------- > > diff --git a/src/hotspot/os/linux/os_linux.cpp > > b/src/hotspot/os/linux/os_linux.cpp > > --- a/src/hotspot/os/linux/os_linux.cpp > > +++ b/src/hotspot/os/linux/os_linux.cpp > > @@ -2720,6 +2720,8 @@ > > } > > > > void os::numa_make_global(char *addr, size_t bytes) { > > + if (!UseNUMAInterleaving) > > + return ; > > Linux::numa_interleave_memory(addr, bytes); > > } > > > > @@ -2785,6 +2787,15 @@ > > ids[i++] = node; > > } > > } > > + > > + if (Linux::_numa_interleave_ptr != NULL ) { > > + i = 0; > > + for (int node = 0; node <= highest_node_number; node++) { > > + if (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, > > node)) { > > + ids[i++] = node; > > + } > > + } > > + } > > return i; > > } > > > > @@ -2884,11 +2895,18 @@ > > libnuma_dlsym(handle, > > "numa_distance"))); > > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > > libnuma_v2_dlsym(handle, > > "numa_get_membind"))); > > + > > > set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, > > + libnuma_v2_dlsym(handle, > > "numa_get_interleave_mask"))); > > > > if (numa_available() != -1) { > > + struct bitmask *bmp; > > set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > > "numa_all_nodes")); > > set_numa_all_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, > > "numa_all_nodes_ptr")); > > set_numa_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, > > "numa_nodes_ptr")); > > + bmp = _numa_get_interleave_mask(); > > + set_numa_interleave_ptr(&bmp); > > + bmp = _numa_get_membind(); > > + set_numa_membind_ptr(&bmp); > > // Create an index -> node mapping, since nodes are not always > > consecutive > > _nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) > > GrowableArray(0, true); > > rebuild_nindex_to_node_map(); > > @@ -3015,9 +3033,12 @@ > > os::Linux::numa_bitmask_isbitset_func_t > os::Linux::_numa_bitmask_isbitset; > > os::Linux::numa_distance_func_t os::Linux::_numa_distance; > > os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > > +os::Linux::numa_get_interleave_mask_func_t > > os::Linux::_numa_get_interleave_mask; > > unsigned long* os::Linux::_numa_all_nodes; > > struct bitmask* os::Linux::_numa_all_nodes_ptr; > > struct bitmask* os::Linux::_numa_nodes_ptr; > > +struct bitmask* os::Linux::_numa_interleave_ptr; > > +struct bitmask* os::Linux::_numa_membind_ptr; > > > > bool os::pd_uncommit_memory(char* addr, size_t size) { > > uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, > > @@ -4997,13 +5018,38 @@ > > if (!Linux::libnuma_init()) { > > UseNUMA = false; > > } else { > > - if ((Linux::numa_max_node() < 1) || > Linux::isbound_to_single_node()) > > { > > - // If there's only one node (they start from 0) or if the > process > > - // is bound explicitly to a single node using membind, disable > > NUMA. > > - UseNUMA = false; > > + > > + // Identify whether running in membind or interleave mode. > > + struct bitmask *bmp; > > + bool _is_membind = false; > > + bool _is_interleaved = false; > > + > > + // Check for membind mode. > > + bmp = Linux::_numa_membind_ptr; > > + for (int node = 0; node <= Linux::numa_max_node() ; node++) { > > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > + _is_membind = true; > > } > > } > > > > + // Check for interleave mode. > > + bmp = Linux::_numa_interleave_ptr; > > + for (int node = 0; node <= Linux::numa_max_node() ; node++) { > > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > + _is_interleaved = true; > > + // Set membind to false as interleave mode allows all nodes to > be > > used. > > + _is_membind = false; > > + } > > + } > > + > > + if (_is_membind) > > + Linux::set_numa_interleave_ptr (NULL); > > + > > + if (_is_interleaved) > > + Linux::set_numa_membind_ptr (NULL); > > + > > + } > > + > > if (UseParallelGC && UseNUMA && UseLargePages && > > !can_commit_large_page_memory()) { > > // With SHM and HugeTLBFS large pages we cannot uncommit a page, > so > > there's no way > > // we can make the adaptive lgrp chunk resizing work. If the user > > specified both > > diff --git a/src/hotspot/os/linux/os_linux.hpp > > b/src/hotspot/os/linux/os_linux.hpp > > --- a/src/hotspot/os/linux/os_linux.hpp > > +++ b/src/hotspot/os/linux/os_linux.hpp > > @@ -222,6 +222,7 @@ > > typedef void (*numa_interleave_memory_func_t)(void *start, size_t > size, > > unsigned long *nodemask); > > typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t > > size, struct bitmask* mask); > > typedef struct bitmask* (*numa_get_membind_func_t)(void); > > + typedef struct bitmask* (*numa_get_interleave_mask_func_t)(void); > > > > typedef void (*numa_set_bind_policy_func_t)(int policy); > > typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > > unsigned int n); > > @@ -239,9 +240,12 @@ > > static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > > static numa_distance_func_t _numa_distance; > > static numa_get_membind_func_t _numa_get_membind; > > + static numa_get_interleave_mask_func_t _numa_get_interleave_mask; > > static unsigned long* _numa_all_nodes; > > static struct bitmask* _numa_all_nodes_ptr; > > static struct bitmask* _numa_nodes_ptr; > > + static struct bitmask* _numa_interleave_ptr; > > + static struct bitmask* _numa_membind_ptr; > > > > static void set_sched_getcpu(sched_getcpu_func_t func) { > _sched_getcpu = > > func; } > > static void set_numa_node_to_cpus(numa_node_to_cpus_func_t func) { > > _numa_node_to_cpus = func; } > > @@ -255,9 +259,12 @@ > > static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > func) > > { _numa_bitmask_isbitset = func; } > > static void set_numa_distance(numa_distance_func_t func) { > > _numa_distance = func; } > > static void set_numa_get_membind(numa_get_membind_func_t func) { > > _numa_get_membind = func; } > > + static void > set_numa_get_interleave_mask(numa_get_interleave_mask_func_t > > func) { _numa_get_interleave_mask = func; } > > static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes > = > > ptr; } > > static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > > static void set_numa_nodes_ptr(struct bitmask **ptr) { > _numa_nodes_ptr = > > (ptr == NULL ? NULL : *ptr); } > > + static void set_numa_interleave_ptr(struct bitmask **ptr) { > > _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } > > + static void set_numa_membind_ptr(struct bitmask **ptr) { > > _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } > > static int sched_getcpu_syscall(void); > > public: > > static int sched_getcpu() { return _sched_getcpu != NULL ? > > _sched_getcpu() : -1; } > > @@ -275,7 +282,10 @@ > > static void numa_interleave_memory(void *start, size_t size) { > > // Use v2 api if available > > if (_numa_interleave_memory_v2 != NULL && _numa_all_nodes_ptr != > NULL) > > { > > - _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr); > > + if (_numa_interleave_ptr != NULL) > > + _numa_interleave_memory_v2(start, size, _numa_interleave_ptr); > > + else > > + _numa_interleave_memory_v2(start, size, _numa_membind_ptr); > > } else if (_numa_interleave_memory != NULL && _numa_all_nodes != > NULL) > > { > > _numa_interleave_memory(start, size, _numa_all_nodes); > > } > > ----------------------- PATCH ----------------------------------- > > > > > > Thanks > > Amit Pawar > > > From david.holmes at oracle.com Sat Nov 10 00:37:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 10 Nov 2018 10:37:30 +1000 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <5dd9d93c-9901-8756-d6be-c8c046726204@oracle.com> Message-ID: <3548029e-475f-586f-1df8-59813e4fc102@oracle.com> Hi Amit, On 10/11/2018 10:34 AM, amith pawar wrote: > Hi David, > > I have initiated for OCA processing and working as an individual > contributor. Great! Welcome to OpenJDK. Please advise once your OCA is processed and then we can properly look at your contribution. Thanks, David > Thanks, > Amit Pawar > > On Sat, Nov 10, 2018, 4:03 AM David Holmes wrote: > > Hi Amith, > > I do not see you listed as a Contributor on the OCA list [1], nor can I > tell from your email address whether you work for a company that is a > signatory. Please confirm your status. > > Thanks, > David > > > [1] https://www.oracle.com/technetwork/community/oca-486395.html > > On 9/11/2018 10:23 PM, amith pawar wrote: > > Hi all, > > > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old > gen, S1 > > and S2 region (if any other ) memory areas > > > > on requested Numa nodes and it should not configure itself to > access other > > Numa nodes. This issue is observed only when Java > > > > is allowed to run on fewer NUMA nodes than available on the > system with > > numactl membind and interleave options. Running > > > > on all the nodes does not have any effect. This will cause some > > applications (objects residing in old gen and survivor region) to > > > > run slower on system with large Numa nodes. > > > > > > > > I have described below in more detail for both numactl MEMBIND and > > INTERLEAVE options. Addresses from both GC log and > > > > process numa maps clearly shows that the JAVA process is > configured to > > access other memory nodes even though it is not allowed. > > > > > > > > Two scenarios are: > > > >? ? ? ? ? ? 1. Numactl membind case : numactl -m 0-1, -N 0-1 > > arguments> > > > > > > Numa map shows these regions are INTERLEAVED ON ALL NODES instead of > > specified Numa memory nodes 0 and 1. > > > > > > > > ----------------------------------- GC > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20) Heap before GC > invocations=21 > > (full 4): PSYoungGen? ? ? total 120604672K, used 11907587K > > [0x00002afc4b200000, 0x00002b198b200000, 0x00002b198b200000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?eden space > 118525952K, 8% used > > [0x00002afc4b200000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ? ?lgrp 0 space > 59262976K, 8% > > used [0x00002afc4b200000,0x00002afd89bef450,0x00002b0a6c400000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ? ?lgrp 1 space > 59262976K, 8% > > used [0x00002b0a6c400000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?from space 2078720K, > 65% used > > [0x00002b190c400000,0x00002b195ef5a0d0,0x00002b198b200000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?to? ?space 2078720K, > 0% used > > [0x00002b188d600000,0x00002b188d600000,0x00002b190c400000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ParOldGen? ? ? ?total > 2097152K, > > used 226685K [0x00002afbcb200000, 0x00002afc4b200000, > 0x00002afc4b200000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?object space > 2097152K, 10% used > > [0x00002afbcb200000,0x00002afbd8f5f460,0x00002afc4b200000) > > > > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? Metaspace? ? ? ?used > 28462K, > > capacity 29008K, committed 29184K, reserved 30720K > > > > ----------------------------------- GC HEAP > > > END-------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > ---------------------------------- PROCESS NUMA Maps > > > --------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/4947/numa_maps > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > >? ? ? =======> Following addresses are interleaved on all nodes. > > > > 2afbb4f4c000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 > N4=2 N5=2 > > N6=2 N7=2 kernelpagesize_kB=4 > > > > 2afbb4f6c000 interleave:0-7 > > > > 2afbb7e88000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 > N4=6 N5=6 > > N6=6 N7=6 kernelpagesize_kB=4 > > > > 2afbbc000000 interleave:0-7 anon=8704 dirty=8704 N0=1600 N1=1088 > N2=1088 > > N3=576 N4=1088 N5=1088 N6=1088 N7=1088 kernelpagesize_kB=4 > > > > 2afbc3be6000 interleave:0-7 anon=6682 dirty=6682 N0=1027 N1=1027 > N2=515 > > N3=515 N4=515 N5=1027 N6=1028 N7=1028 kernelpagesize_kB=4 > > > > 2afbcb000000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 > N4=6 N5=6 > > N6=6 N7=6 kernelpagesize_kB=4 > > > > 2afbcb200000 interleave:0-7 anon=524288 dirty=524288 N0=65536 > N1=65536 > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > kernelpagesize_kB=4 > > > >? ? ? ? ? ? ? ==> OLD GEN Address > > > > 2afc4b200000 prefer:0 anon=1536 dirty=1536 N0=1536 > kernelpagesize_kB=4 > > > > 2b0a6c400000 prefer:1 anon=512 dirty=512 N1=512 kernelpagesize_kB=4 > > > > 2b188d600000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 > N1=130048 > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > kernelpagesize_kB=4 > > > >? ? ? ? ? ? ? ==> Survivor Region > > > > 2b198b600000 interleave:0-7 anon=60929 dirty=60929 N0=7233 > N1=7744 N2=7232 > > N3=7744 N4=7744 N5=7744 N6=7744 N7=7744 kernelpagesize_kB=4 > > > > ---------------------------------- PROCESS NUMA Maps > > > ----------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > ---------------------------------- PROCESS status > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/4947/status > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Cpus_allowed:? ?00000000,0000ffff,00000000,0000ffff > > > > Cpus_allowed_list:? ? ? 0-15,64-79 > > > > Mems_allowed: > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > Mems_allowed_list:? ? ? 0-7 > > > > --------------------------------- PROCESS status > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > >? ? ? ? ? 2. NUMACTL Interleave? case: numactl -i 0-1, -N 0-1 > > its arguments> > > > > > > NUMAmaps below shows interleaved on all nodes instead of > specified Numa > > memory nodes 0 and 1. > > > > ----------------------------------- GC > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159) Heap before GC > invocations=160 > > (full 9): PSYoungGen? ? ? total 120604672K, used 30143454K > > [0x00002b9d47c00000, 0x00002bba87c00000, 0x00002bba87c00000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?eden space > 118525952K, 24% > > used [0x00002b9d47c00000,0x00002ba458400000,0x00002bb98a000000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 0 space > 14815232K, 98% > > used [0x00002b9d47c00000,0x00002ba0beb87c90,0x00002ba0d0000000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 1 space > 14815232K, > > 100% used [0x00002ba0d0000000,0x00002ba458400000,0x00002ba458400000) > > > >? ? ? ? ?==> Memory allocated on following nodes are unused. > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 2 space > 14815232K, 0% > > used [0x00002ba458400000,0x00002ba458400000,0x00002ba7e0800000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 3 space > 14815232K, 0% > > used [0x00002ba7e0800000,0x00002ba7e0800000,0x00002bab68c00000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 4 space > 14815232K, 0% > > used [0x00002bab68c00000,0x00002bab68c00000,0x00002baef1000000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 5 space > 14815232K, 0% > > used [0x00002baef1000000,0x00002baef1000000,0x00002bb279400000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 6 space > 14815232K, 0% > > used [0x00002bb279400000,0x00002bb279400000,0x00002bb601800000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 7 space > 14819328K, 0% > > used [0x00002bb601800000,0x00002bb601800000,0x00002bb98a000000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?from space > 2078720K, 38% used > > [0x00002bba08e00000,0x00002bba3976fb70,0x00002bba87c00000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?to? ?space > 2078720K, 0% used > > [0x00002bb98a000000,0x00002bb98a000000,0x00002bba08e00000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ParOldGen? ? ? ?total > > 2097152K, used 685229K [0x00002b9cc7c00000, 0x00002b9d47c00000, > > 0x00002b9d47c00000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?object space > 2097152K, 32% > > used [0x00002b9cc7c00000,0x00002b9cf192b6e8,0x00002b9d47c00000) > > > > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? Metaspace? ? ? ?used > 28753K, > > capacity 29257K, committed 29440K, reserved 30720K > > > > ----------------------------------- GC HEAP > > > END------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > ---------------------------------- PROCESS NUMA Maps > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/pid/numa_maps > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > >? ? ?==> Following addresses are interleaved on all the nodes. > > > > 2b9cb1992000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 > N4=2 N5=2 > > N6=2 N7=2 kernelpagesize_kB=4 > > > > 2b9cb19b2000 interleave:0-7 > > > > 2b9cb3e65000 interleave:0-7 anon=50 dirty=50 N0=6 N1=6 N2=6 N3=6 > N4=6 N5=7 > > N6=7 N7=6 kernelpagesize_kB=4 > > > > 2b9cb8a69000 interleave:0-7 anon=8599 dirty=8599 N0=626 N1=1139 > N2=1139 > > N3=1139 N4=1139 N5=1139 N6=1139 N7=1139 kernelpagesize_kB=4 > > > > 2b9cc064f000 interleave:0-7 anon=6577 dirty=6577 N0=566 N1=566 N2=566 > > N3=1078 N4=1078 N5=1078 N6=1078 N7=567 kernelpagesize_kB=4 > > > > 2b9cc7a69000 interleave:0-7 anon=50 dirty=50 N0=6 N1=7 N2=7 N3=6 > N4=6 N5=6 > > N6=6 N7=6 kernelpagesize_kB=4 > > > > 2b9cc7c00000 interleave:0-7 anon=524288 dirty=524288 N0=65536 > N1=65536 > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > kernelpagesize_kB=4 > > > > 2b9d47c00000 prefer:0 anon=2560 dirty=2560 N0=2560 > kernelpagesize_kB=4 > >? ? ?==> Logical group 1 > > > > 2ba0d0000000 > > prefer:1 > > ==> Logical group 2 > > > > 2ba458400000 prefer:2 > > > > ==> This one and below all are unnecessary and leaving memory unused. > > > > 2ba7e0800000 prefer:3 > > > > 2bab68c00000 prefer:4 > > > > 2baef1000000 prefer:5 > > > > 2bb279400000 prefer:6 > > > > 2bb601800000 prefer:7 > > > > 2bb98a000000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 > N1=130048 > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > kernelpagesize_kB=4 > > > > 2bba88000000 interleave:0-7 anon=60929 dirty=60929 N0=7745 > N1=7744 N2=7744 > > N3=7744 N4=7744 N5=7232 N6=7744 N7=7232 kernelpagesize_kB=4 > > > > ---------------------------------- PROCESS NUMA Maps > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > >? ?--------------------------------- PROCESS status > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Command : cat /proc/pid/status > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > Cpus_allowed:? ?00000000,0000ffff,00000000,0000ffff > > > > Cpus_allowed_list:? ? ? 0-15,64-79 > > > > Mems_allowed: > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > Mems_allowed_list:? ? ? 0-7 > > > >? ?--------------------------------- PROCESS status > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > Please note ?Mems_allowed? and ?Mems_allowed_list? list also shows > > incorrect range. This could be LIBNUMA issue in case of membind. > > > > When application is externally bound to fewer nodes then calls to > libNUMA > > API?s SHOULD FAIL to interleave on all nodes but right now it > > > > does not with currently used/called API?s in JDK. > > > > > > > > How patch works ? > > > > The patch gets bitmask structure by calling following API's (Man page > > definition for these functions are also given below). > > > >? ?1. For Membind : Use numa_get_membind to get membind bitmask > (already used > > in the code) > > > > > >? ? ? numa_get_membind() returns the mask of nodes from which > memory can > > currently be allocated.? If the returned mask is equal to > numa_all_nodes, > > then memory allocation is allowed from all nodes. > > > >? ?2. For Interleave: use numa_get_interleave_mask to get > interleave mask > > (currently not used/called in JDK) > >? ? ? ?numa_get_interleave_mask() returns the current interleave > mask if the > > task's memory allocation policy is page interleaved.? Otherwise, this > > function returns an empty mask. > > > > > > > > Check node counts from both the Bitmasks to identify current > running mode. > > When Interleave mask is EMPTY then it is running in membind else > it is in > > interleave mode. > > > > Call ?numa_interleave_memory? (called indirectly through > ?numa_make_global? > > funcition) function with right bitmask identified above helps to > fix this > > issue. > > > > > > > > > > > > Improvement: > > > > This patch is tested on EPYC with SPECJBB benchmark and score > improvements > > are given below. > > > > 1. For NUMACTL membind > > > >? ? ? Max-jOPS improved by 5-12 % and Critical-jOPS by 2-6 % > > > > 2. For NUMACTL interleave (This patch fixes memory usage when > invoked with > > numactl -i) > > > >? ? ? Max-jOPS by 5-15% and Critical-jOPS by 11-100%. > > > > 3. With this fix, flag UseNUMAInterleaving turning on/off has no > effect > > when externally interleaved through NUMACTL. > > > > 4. Flag UseNUMA score improved by ~14 % when enabled for single > NUMA node. > > Currently it gets disabled when externally bound to single node. > > > > > > > > > > > > ---------------------- PATCH -------------------------- > > diff --git a/src/hotspot/os/linux/os_linux.cpp > > b/src/hotspot/os/linux/os_linux.cpp > > --- a/src/hotspot/os/linux/os_linux.cpp > > +++ b/src/hotspot/os/linux/os_linux.cpp > > @@ -2720,6 +2720,8 @@ > >? ?} > > > >? ?void os::numa_make_global(char *addr, size_t bytes) { > > +? if (!UseNUMAInterleaving) > > +? ? return ; > >? ? ?Linux::numa_interleave_memory(addr, bytes); > >? ?} > > > > @@ -2785,6 +2787,15 @@ > >? ? ? ? ?ids[i++] = node; > >? ? ? ?} > >? ? ?} > > + > > +? if (Linux::_numa_interleave_ptr != NULL ) { > > +? ? i = 0; > > +? ? for (int node = 0; node <= highest_node_number; node++) { > > +? ? ? if (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, > > node)) { > > +? ? ? ? ids[i++] = node; > > +? ? ? } > > +? ? } > > +? } > >? ? ?return i; > >? ?} > > > > @@ -2884,11 +2895,18 @@ > >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > > "numa_distance"))); > >? ? ? ? ?set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?libnuma_v2_dlsym(handle, > > "numa_get_membind"))); > > + > > > set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, > > +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_v2_dlsym(handle, > > "numa_get_interleave_mask"))); > > > >? ? ? ? ?if (numa_available() != -1) { > > +? ? ? ? struct bitmask *bmp; > >? ? ? ? ? ?set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > > "numa_all_nodes")); > >? ? ? ? ? ?set_numa_all_nodes_ptr((struct bitmask > **)libnuma_dlsym(handle, > > "numa_all_nodes_ptr")); > >? ? ? ? ? ?set_numa_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, > > "numa_nodes_ptr")); > > +? ? ? ? bmp = _numa_get_interleave_mask(); > > +? ? ? ? set_numa_interleave_ptr(&bmp); > > +? ? ? ? bmp = _numa_get_membind(); > > +? ? ? ? set_numa_membind_ptr(&bmp); > >? ? ? ? ? ?// Create an index -> node mapping, since nodes are not > always > > consecutive > >? ? ? ? ? ?_nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) > > GrowableArray(0, true); > >? ? ? ? ? ?rebuild_nindex_to_node_map(); > > @@ -3015,9 +3033,12 @@ > >? ?os::Linux::numa_bitmask_isbitset_func_t > os::Linux::_numa_bitmask_isbitset; > >? ?os::Linux::numa_distance_func_t os::Linux::_numa_distance; > >? ?os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > > +os::Linux::numa_get_interleave_mask_func_t > > os::Linux::_numa_get_interleave_mask; > >? ?unsigned long* os::Linux::_numa_all_nodes; > >? ?struct bitmask* os::Linux::_numa_all_nodes_ptr; > >? ?struct bitmask* os::Linux::_numa_nodes_ptr; > > +struct bitmask* os::Linux::_numa_interleave_ptr; > > +struct bitmask* os::Linux::_numa_membind_ptr; > > > >? ?bool os::pd_uncommit_memory(char* addr, size_t size) { > >? ? ?uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, > > @@ -4997,13 +5018,38 @@ > >? ? ? ?if (!Linux::libnuma_init()) { > >? ? ? ? ?UseNUMA = false; > >? ? ? ?} else { > > -? ? ? if ((Linux::numa_max_node() < 1) || > Linux::isbound_to_single_node()) > > { > > -? ? ? ? // If there's only one node (they start from 0) or if > the process > > -? ? ? ? // is bound explicitly to a single node using membind, > disable > > NUMA. > > -? ? ? ? UseNUMA = false; > > + > > +? ? // Identify whether running in membind or interleave mode. > > +? ? struct bitmask *bmp; > > +? ? bool _is_membind = false; > > +? ? bool _is_interleaved = false; > > + > > +? ? // Check for membind mode. > > +? ? bmp = Linux::_numa_membind_ptr; > > +? ? for (int node = 0; node <= Linux::numa_max_node() ; node++) { > > +? ? ? if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > +? ? ? ? _is_membind = true; > >? ? ? ? ?} > >? ? ? ?} > > > > +? ? // Check for interleave mode. > > +? ? bmp = Linux::_numa_interleave_ptr; > > +? ? for (int node = 0; node <= Linux::numa_max_node() ; node++) { > > +? ? ? if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > +? ? ? ? _is_interleaved = true; > > +? ? ? ? // Set membind to false as interleave mode allows all > nodes to be > > used. > > +? ? ? ? _is_membind = false; > > +? ? ? } > > +? ? } > > + > > +? ? if (_is_membind) > > +? ? ? Linux::set_numa_interleave_ptr (NULL); > > + > > +? ? if (_is_interleaved) > > +? ? ? Linux::set_numa_membind_ptr (NULL); > > + > > +? ? } > > + > >? ? ? ?if (UseParallelGC && UseNUMA && UseLargePages && > > !can_commit_large_page_memory()) { > >? ? ? ? ?// With SHM and HugeTLBFS large pages we cannot uncommit > a page, so > > there's no way > >? ? ? ? ?// we can make the adaptive lgrp chunk resizing work. If > the user > > specified both > > diff --git a/src/hotspot/os/linux/os_linux.hpp > > b/src/hotspot/os/linux/os_linux.hpp > > --- a/src/hotspot/os/linux/os_linux.hpp > > +++ b/src/hotspot/os/linux/os_linux.hpp > > @@ -222,6 +222,7 @@ > >? ? ?typedef void (*numa_interleave_memory_func_t)(void *start, > size_t size, > > unsigned long *nodemask); > >? ? ?typedef void (*numa_interleave_memory_v2_func_t)(void *start, > size_t > > size, struct bitmask* mask); > >? ? ?typedef struct bitmask* (*numa_get_membind_func_t)(void); > > +? typedef struct bitmask* (*numa_get_interleave_mask_func_t)(void); > > > >? ? ?typedef void (*numa_set_bind_policy_func_t)(int policy); > >? ? ?typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > > unsigned int n); > > @@ -239,9 +240,12 @@ > >? ? ?static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > >? ? ?static numa_distance_func_t _numa_distance; > >? ? ?static numa_get_membind_func_t _numa_get_membind; > > +? static numa_get_interleave_mask_func_t _numa_get_interleave_mask; > >? ? ?static unsigned long* _numa_all_nodes; > >? ? ?static struct bitmask* _numa_all_nodes_ptr; > >? ? ?static struct bitmask* _numa_nodes_ptr; > > +? static struct bitmask* _numa_interleave_ptr; > > +? static struct bitmask* _numa_membind_ptr; > > > >? ? ?static void set_sched_getcpu(sched_getcpu_func_t func) { > _sched_getcpu = > > func; } > >? ? ?static void set_numa_node_to_cpus(numa_node_to_cpus_func_t > func) { > > _numa_node_to_cpus = func; } > > @@ -255,9 +259,12 @@ > >? ? ?static void > set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) > > { _numa_bitmask_isbitset = func; } > >? ? ?static void set_numa_distance(numa_distance_func_t func) { > > _numa_distance = func; } > >? ? ?static void set_numa_get_membind(numa_get_membind_func_t func) { > > _numa_get_membind = func; } > > +? static void > set_numa_get_interleave_mask(numa_get_interleave_mask_func_t > > func) { _numa_get_interleave_mask = func; } > >? ? ?static void set_numa_all_nodes(unsigned long* ptr) { > _numa_all_nodes = > > ptr; } > >? ? ?static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > >? ? ?static void set_numa_nodes_ptr(struct bitmask **ptr) { > _numa_nodes_ptr = > > (ptr == NULL ? NULL : *ptr); } > > +? static void set_numa_interleave_ptr(struct bitmask **ptr) { > > _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } > > +? static void set_numa_membind_ptr(struct bitmask **ptr) { > > _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } > >? ? ?static int sched_getcpu_syscall(void); > >? ? public: > >? ? ?static int sched_getcpu()? { return _sched_getcpu != NULL ? > > _sched_getcpu() : -1; } > > @@ -275,7 +282,10 @@ > >? ? ?static void numa_interleave_memory(void *start, size_t size) { > >? ? ? ?// Use v2 api if available > >? ? ? ?if (_numa_interleave_memory_v2 != NULL && > _numa_all_nodes_ptr != NULL) > > { > > -? ? ? _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr); > > +? ? ? if (_numa_interleave_ptr != NULL) > > +? ? ? ? _numa_interleave_memory_v2(start, size, > _numa_interleave_ptr); > > +? ? ? else > > +? ? ? ? _numa_interleave_memory_v2(start, size, _numa_membind_ptr); > >? ? ? ?} else if (_numa_interleave_memory != NULL && > _numa_all_nodes != NULL) > > { > >? ? ? ? ?_numa_interleave_memory(start, size, _numa_all_nodes); > >? ? ? ?} > > ----------------------- PATCH ----------------------------------- > > > > > > Thanks > > Amit Pawar > > > From zgu at redhat.com Sat Nov 10 01:59:12 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 9 Nov 2018 20:59:12 -0500 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: References: Message-ID: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> Thanks for reviewing, David. On 11/9/18 5:58 PM, David Holmes wrote: > Looks good - thanks for finding and fixing. > > I see cases internally where we fail because of this: > > Failure: ExceptionInInitializerError: target class not found > > but no bug was ever filed. I wonder if we will now fail because of the > OOME ... Actually, it has to avoid OOME to get to here. Several nsk/stress/except tests, e.g. except001, except002 and etc. depend on quick OOME to pass. Shenandoah has mechanism to gradually slow down allocation, so it is more resist to OOME. Unfortunately, it usually fails due to timeout. I am wondering of the usefulness of these tests. Thanks, -Zhengyu > > Thanks, > David > > On 10/11/2018 7:43 AM, Zhengyu Gu wrote: >> Please review this trivial fix. >> >> Currently, only Shenandoah experiences the failure, since other GCs >> run out Java heap memory, then the rest of test is skipped. >> >> E.g. >> pool[125131189]=new Object(); // elapsed 10.287s >> pool[126306334]=new Object(); // elapsed 7.947s >> Heap seems exhausted - OutOfMemoryError thrown. >> Skipped: ExceptionInInitializerError: thrown OutOfMemoryError >> Test passed. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 >> Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ >> >> Thanks, >> >> -Zhengyu From jcbeyler at google.com Sat Nov 10 03:24:03 2018 From: jcbeyler at google.com (JC Beyler) Date: Fri, 9 Nov 2018 19:24:03 -0800 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> Message-ID: Hi Zhengyu, The fix looks good to me (I'm not a reviewer though ;-)), I was wondering if the other forName methods don't have the same issue: except002.java: trash = Class.forName("nsk.stress.except.except002.except002$Abra$Cadabra"); // correct - should pass except002.java:// trash = Class.forName("nsk.stress.except.except002.except002.Abra.Cadabra"); // incorrect - should fail except003.java:// trash = Class.forName("nsk.stress.except.except003.except003$Abra$Cadabra"); // correct - should pass except003.java: trash = Class.forName("nsk.stress.except.except003.except003.Abra.Cadabra"); // incorrect - should fail Seems like they also have an extra exceptXXX in the name, no? Could/Should we try to fix them at the same time? Jc On Fri, Nov 9, 2018 at 6:00 PM Zhengyu Gu wrote: > Thanks for reviewing, David. > > On 11/9/18 5:58 PM, David Holmes wrote: > > Looks good - thanks for finding and fixing. > > > > I see cases internally where we fail because of this: > > > > Failure: ExceptionInInitializerError: target class not found > > > > but no bug was ever filed. I wonder if we will now fail because of the > > OOME ... > > Actually, it has to avoid OOME to get to here. > > Several nsk/stress/except tests, e.g. except001, except002 and etc. > depend on quick OOME to pass. > > Shenandoah has mechanism to gradually slow down allocation, so it is > more resist to OOME. Unfortunately, it usually fails due to timeout. > > I am wondering of the usefulness of these tests. > > Thanks, > > -Zhengyu > > > > > Thanks, > > David > > > > On 10/11/2018 7:43 AM, Zhengyu Gu wrote: > >> Please review this trivial fix. > >> > >> Currently, only Shenandoah experiences the failure, since other GCs > >> run out Java heap memory, then the rest of test is skipped. > >> > >> E.g. > >> pool[125131189]=new Object(); // elapsed 10.287s > >> pool[126306334]=new Object(); // elapsed 7.947s > >> Heap seems exhausted - OutOfMemoryError thrown. > >> Skipped: ExceptionInInitializerError: thrown OutOfMemoryError > >> Test passed. > >> > >> > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 > >> Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ > >> > >> Thanks, > >> > >> -Zhengyu > -- Thanks, Jc From david.holmes at oracle.com Sat Nov 10 11:59:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 10 Nov 2018 21:59:30 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Matthias, On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk , I created another webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > The user info output is now guarded by ExtensiveErrorReports . + STEP("printing user info") + if (ExtensiveErrorReports) { + if (_verbose) { + os::Posix::print_user_info(st, buf, sizeof(buf)); + } + } I don't understand why we explicitly need _verbose if we've asked for ExtensiveErrorreports? Also, ideally the STEP would be inside the guard as otherwise we will just print the step description followed by nothing. If the macro expansion makes that impossible then we should have an else clause that prints something like: - disabled (use -XX:+ExtensiveErrorReports to see these details) Thanks, David ----- > Please review! > > > Thanks, Matthias > > >> -----Original Message----- >> From: Baesken, Matthias >> Sent: Dienstag, 9. Oktober 2018 16:55 >> To: 'Thomas St?fe' ; David Holmes >> >> Cc: Langer, Christoph ; Volker Simonis >> ; HotSpot Open Source Developers > dev at openjdk.java.net> >> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file >> >> Hi Thomas , thanks for filing the CSR . >> >> I am fine with the name proposal "-XX:+-ExtensiveErrorReports", but will >> wait until it is agreed to use this name . >> Then I'll post another webrev of 8211326 using the flag . >> >> Best regards, Matthias >> >> >> >>> -----Original Message----- >>> From: Thomas St?fe >>> Sent: Montag, 8. Oktober 2018 12:21 >>> To: David Holmes >>> Cc: Baesken, Matthias ; Langer, Christoph >>> ; Volker Simonis >> ; >>> HotSpot Open Source Developers >>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>> >>> On Mon, Oct 8, 2018 at 11:50 AM David Holmes >> >>> wrote: >>>> >>>> Hi Thomas, >>>> >>>> On 8/10/2018 7:43 PM, Thomas St?fe wrote: >>>>> Hi all, >>>>> >>>>> I drafted a CSR: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8211846 >>>>> >>>>> Not sure what to do now, should I open a discussion thread on a >>>>> mailing list? Or just wait for the committee to decide? (I'm doing >>>>> this too rarely..) >>>> >>>> I filled in some of the additional info needed. The text needs to be >>>> completed and the CSR needs to have at least one reviewer added to it. >>>> Then it can be marked "final" and ready for the committee to evaluate. >>>> >>> >>> Thanks! I filled out what was missing and will wait for reviewers. >>> >>> Thanks, Thomas >>> > From thomas.stuefe at gmail.com Sat Nov 10 12:58:27 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 10 Nov 2018 13:58:27 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> Message-ID: Hi Aleksey, change looks fine. Some small remarks: http://cr.openjdk.java.net/~shade/8211926/webrev.05/src/hotspot/share/utilities/bitMap.inline.hpp.udiff.html + assert(beg <= end, "underflow"); I would feel slightly safer with a runtime check for product as well. -- http://cr.openjdk.java.net/~shade/8211926/webrev.05/test/hotspot/gtest/utilities/test_bitMap_large.cpp.html The tests favor the left side (lower space) of the bitmap, since l never grows beyond FUZZ_WINDOW. I wonder whether testing the right side (small ranges near the bitmap end) would be useful. Maybe with a gliding window, or just repeat the tests anchored at the right instead of the left side. -- Side notes (nothing to do with your patch): - BitMap::set_range_within_word, clear_range... : would like to see asserts here for beg <= end. - I find it confusing that we use the same type (idx_t) both for word- and bit-indices. It would make the code a bit more readable if those were separate types (even if they both were to alias to size_t, ultimately). - Generally, I find putting the decision between "large" and "non-large" APIs off to the caller a bit strange. As a user of bitmap, I cannot know at which point which variant would be better. I would prefer the class itself handling that. -- All these are idle musings, nothing more. The change is fine to me as it is. ..Thomas On Sat, Nov 10, 2018 at 12:32 AM Aleksey Shipilev wrote: > > On 11/09/2018 09:02 PM, Kim Barrett wrote: > > Almost, but not quite, I'm afraid. All but the first comment below > > are optional, though I think make things better. > > Introduced helper method, moved the small_range_words to private, handled (l == r) corner case in > test. See: > http://cr.openjdk.java.net/~shade/8211926/webrev.05/ > > -Aleksey > > P.S. For the love of any deity of choice, can we please push this corner-case fix and move on to > solve meatier bugs? I appreciate the attention to details, but the economics of spending any more > time to this issue feels dubious to me. > From thomas.stuefe at gmail.com Sat Nov 10 13:04:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 10 Nov 2018 14:04:10 +0100 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi David, On Sat, Nov 10, 2018 at 12:59 PM David Holmes wrote: > > Hi Matthias, > > On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk , I created another webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > + STEP("printing user info") > + if (ExtensiveErrorReports) { > + if (_verbose) { > + os::Posix::print_user_info(st, buf, sizeof(buf)); > + } > + } > > I don't understand why we explicitly need _verbose if we've asked for > ExtensiveErrorreports? That flag has a different purpose: _verbose distinguishes the two calls to VMError::report(): the first one to print a small report to stderr, the second print a large report to the hs-err file. We only want to print to the hs-err file, so _verbose is still needed. > > Also, ideally the STEP would be inside the guard as otherwise we will > just print the step description followed by nothing. If the macro > expansion makes that impossible then we should have an else clause that > prints something like: > > - disabled (use -XX:+ExtensiveErrorReports to see these details) > We may even go a step further (but in a separate patch) and make "ExtensiveErrorReports" a property of the STEP, to be given as parameter of the STEP macro. Thanks, Thomas > Thanks, > David > ----- > > > Please review! > > > > > > Thanks, Matthias > > > > > >> -----Original Message----- > >> From: Baesken, Matthias > >> Sent: Dienstag, 9. Oktober 2018 16:55 > >> To: 'Thomas St?fe' ; David Holmes > >> > >> Cc: Langer, Christoph ; Volker Simonis > >> ; HotSpot Open Source Developers >> dev at openjdk.java.net> > >> Subject: RE: RFR : 8211326 : add OS user related information to hs_err file > >> > >> Hi Thomas , thanks for filing the CSR . > >> > >> I am fine with the name proposal "-XX:+-ExtensiveErrorReports", but will > >> wait until it is agreed to use this name . > >> Then I'll post another webrev of 8211326 using the flag . > >> > >> Best regards, Matthias > >> > >> > >> > >>> -----Original Message----- > >>> From: Thomas St?fe > >>> Sent: Montag, 8. Oktober 2018 12:21 > >>> To: David Holmes > >>> Cc: Baesken, Matthias ; Langer, Christoph > >>> ; Volker Simonis > >> ; > >>> HotSpot Open Source Developers > >>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > >>> > >>> On Mon, Oct 8, 2018 at 11:50 AM David Holmes > >> > >>> wrote: > >>>> > >>>> Hi Thomas, > >>>> > >>>> On 8/10/2018 7:43 PM, Thomas St?fe wrote: > >>>>> Hi all, > >>>>> > >>>>> I drafted a CSR: > >>>>> > >>>>> https://bugs.openjdk.java.net/browse/JDK-8211846 > >>>>> > >>>>> Not sure what to do now, should I open a discussion thread on a > >>>>> mailing list? Or just wait for the committee to decide? (I'm doing > >>>>> this too rarely..) > >>>> > >>>> I filled in some of the additional info needed. The text needs to be > >>>> completed and the CSR needs to have at least one reviewer added to it. > >>>> Then it can be marked "final" and ready for the committee to evaluate. > >>>> > >>> > >>> Thanks! I filled out what was missing and will wait for reviewers. > >>> > >>> Thanks, Thomas > >>> > > From shade at redhat.com Sat Nov 10 13:10:16 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 10 Nov 2018 14:10:16 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> Message-ID: <1b495d0c-cab3-8773-cacf-5df828c986ed@redhat.com> On 11/10/2018 01:58 PM, Thomas St?fe wrote: > http://cr.openjdk.java.net/~shade/8211926/webrev.05/src/hotspot/share/utilities/bitMap.inline.hpp.udiff.html > > + assert(beg <= end, "underflow"); > > I would feel slightly safer with a runtime check for product as well. I think current patch never enters the bad path on any combination of incoming arguments, so we can go with just the debug sanity check. And even without the assert, you would notice the breakage in product builds as well, when it stomps over most of VM memory, reaches the end of committed memory, and SEGVs :) Asserts only make it evident where it had gone downhill. > http://cr.openjdk.java.net/~shade/8211926/webrev.05/test/hotspot/gtest/utilities/test_bitMap_large.cpp.html > > The tests favor the left side (lower space) of the bitmap, since l > never grows beyond FUZZ_WINDOW. I wonder whether testing the right > side (small ranges near the bitmap end) would be useful. Maybe with a > gliding window, or just repeat the tests anchored at the right instead > of the left side. From gray-boxing perspective, it seems to me that what matters is the position of left/right boundaries with regards to word boundaries. This keeps testing time at bay as well. I would be in favor of adding the proper stress tests for internal code that would spend tens of minutes going through every possible combination, but not for this bug. -Aleksey From zgu at redhat.com Sat Nov 10 14:03:50 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Sat, 10 Nov 2018 09:03:50 -0500 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> Message-ID: <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> Hi JC, Thanks for reviewing. On 11/9/18 10:24 PM, JC Beyler wrote: > Hi Zhengyu, > > The fix looks good to me (I'm not a reviewer though ;-)), I was > wondering if the other forName methods don't have the same issue: > except002.java:? ? ? ? ? ? trash = > Class.forName("nsk.stress.except.except002.except002$Abra$Cadabra"); // > ?correct - should pass > except002.java://? ? ? ? ? trash = > Class.forName("nsk.stress.except.except002.except002.Abra.Cadabra"); // > incorrect - should fail > except003.java://? ? ? ? ? trash = > Class.forName("nsk.stress.except.except003.except003$Abra$Cadabra"); // > ?correct - should pass > except003.java:? ? ? ? ? ? trash = > Class.forName("nsk.stress.except.except003.except003.Abra.Cadabra"); // > incorrect - should fail I got except002 timed out, has yet gotten to it. except003 passed, will check if it is due to OOM. This one happens to be low hanging fruit :-) Thanks, -Zhengyu > > Seems like they also have an extra exceptXXX in the name, no? > Could/Should we try to fix them at the same time? > Jc > > > On Fri, Nov 9, 2018 at 6:00 PM Zhengyu Gu > wrote: > > Thanks for reviewing, David. > > On 11/9/18 5:58 PM, David Holmes wrote: > > Looks good - thanks for finding and fixing. > > > > I see cases internally where we fail because of this: > > > > Failure: ExceptionInInitializerError: target class not found > > > > but no bug was ever filed. I wonder if we will now fail because > of the > > OOME ... > > Actually, it has to avoid OOME to get to here. > > Several nsk/stress/except tests, e.g. except001, except002 and etc. > depend on quick OOME to pass. > > Shenandoah has mechanism to gradually slow down allocation, so it is > more resist to OOME. Unfortunately, it usually fails due to timeout. > > I am wondering of the usefulness of these tests. > > Thanks, > > -Zhengyu > > > > > Thanks, > > David > > > > On 10/11/2018 7:43 AM, Zhengyu Gu wrote: > >> Please review this trivial fix. > >> > >> Currently, only Shenandoah experiences the failure, since other GCs > >> run out Java heap memory, then the rest of test is skipped. > >> > >> E.g. > >> pool[125131189]=new Object(); // elapsed 10.287s > >> pool[126306334]=new Object(); // elapsed 7.947s > >> Heap seems exhausted - OutOfMemoryError thrown. > >> Skipped: ExceptionInInitializerError: thrown OutOfMemoryError > >> Test passed. > >> > >> > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 > >> Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ > >> > >> Thanks, > >> > >> -Zhengyu > > > > -- > > Thanks, > Jc From shade at redhat.com Sat Nov 10 17:19:00 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 10 Nov 2018 18:19:00 +0100 Subject: RFR 8213711 (XS): Zero build broken after JDK-8213199 (GC abstraction for Assembler::needs_explicit_null_check()) Message-ID: <432c0207-e9df-4f45-f780-0e6851dd8690@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8213711 Fix is to add Zero declaration for the method added in JDK-8213199: 8213711: Zero build broken after JDK-8213199 (GC abstraction for Assembler::needs_explicit_null_check()) Reviewed-by: XXX diff -r 5643a5bff514 -r 919dde170921 src/hotspot/cpu/zero/assembler_zero.hpp --- a/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 09:06:33 2018 -0500 +++ b/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 18:15:49 2018 +0100 @@ -47,6 +47,7 @@ void align(int modulus); void bang_stack_with_offset(int offset); bool needs_explicit_null_check(intptr_t offset); + bool uses_implicit_null_check(void* address); RegisterOrConstant delayed_value_impl(intptr_t* delayed_value_addr, Register tmp, int offset); public: Testing: x86_64 {Zero,Server} builds Thanks, -Aleksey From rkennke at redhat.com Sat Nov 10 17:25:31 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 10 Nov 2018 18:25:31 +0100 Subject: RFR 8213711 (XS): Zero build broken after JDK-8213199 (GC abstraction for Assembler::needs_explicit_null_check()) In-Reply-To: <432c0207-e9df-4f45-f780-0e6851dd8690@redhat.com> References: <432c0207-e9df-4f45-f780-0e6851dd8690@redhat.com> Message-ID: <1AFB282B-9AE8-4B91-819F-A8722AA604C2@redhat.com> Good. Thanks for fixing my mistake. Am 10. November 2018 18:19:00 MEZ schrieb Aleksey Shipilev : >Bug: > https://bugs.openjdk.java.net/browse/JDK-8213711 > >Fix is to add Zero declaration for the method added in JDK-8213199: > >8213711: Zero build broken after JDK-8213199 (GC abstraction for >Assembler::needs_explicit_null_check()) >Reviewed-by: XXX > >diff -r 5643a5bff514 -r 919dde170921 >src/hotspot/cpu/zero/assembler_zero.hpp >--- a/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 09:06:33 >2018 -0500 >+++ b/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 18:15:49 >2018 +0100 >@@ -47,6 +47,7 @@ > void align(int modulus); > void bang_stack_with_offset(int offset); > bool needs_explicit_null_check(intptr_t offset); >+ bool uses_implicit_null_check(void* address); > RegisterOrConstant delayed_value_impl(intptr_t* delayed_value_addr, > Register tmp, int offset); > public: > > >Testing: x86_64 {Zero,Server} builds > >Thanks, >-Aleksey From thomas.stuefe at gmail.com Sat Nov 10 17:41:07 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 10 Nov 2018 18:41:07 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <1b495d0c-cab3-8773-cacf-5df828c986ed@redhat.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> <1b495d0c-cab3-8773-cacf-5df828c986ed@redhat.com> Message-ID: On Sat, Nov 10, 2018 at 2:10 PM Aleksey Shipilev wrote: > > On 11/10/2018 01:58 PM, Thomas St?fe wrote: > > http://cr.openjdk.java.net/~shade/8211926/webrev.05/src/hotspot/share/utilities/bitMap.inline.hpp.udiff.html > > > > + assert(beg <= end, "underflow"); > > > > I would feel slightly safer with a runtime check for product as well. > > I think current patch never enters the bad path on any combination of incoming arguments, so we can > go with just the debug sanity check. And even without the assert, you would notice the breakage in > product builds as well, when it stomps over most of VM memory, reaches the end of committed memory, > and SEGVs :) Asserts only make it evident where it had gone downhill. > Okay, yes, chance for such an overwrite to go unnoticed are slim.. > > http://cr.openjdk.java.net/~shade/8211926/webrev.05/test/hotspot/gtest/utilities/test_bitMap_large.cpp.html > > > > The tests favor the left side (lower space) of the bitmap, since l > > never grows beyond FUZZ_WINDOW. I wonder whether testing the right > > side (small ranges near the bitmap end) would be useful. Maybe with a > > gliding window, or just repeat the tests anchored at the right instead > > of the left side. > > From gray-boxing perspective, it seems to me that what matters is the position of left/right > boundaries with regards to word boundaries. This keeps testing time at bay as well. I would be in > favor of adding the proper stress tests for internal code that would spend tens of minutes going > through every possible combination, but not for this bug. In that case you could have probably done with some less variants of size_class too. But as I said I am fine with the original change. Cheers, Thomas > > -Aleksey > From kim.barrett at oracle.com Sat Nov 10 17:44:41 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 10 Nov 2018 12:44:41 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> Message-ID: <300C792B-74DB-4ACC-AF05-9173E8820C64@oracle.com> > On Nov 9, 2018, at 6:31 PM, Aleksey Shipilev wrote: > > On 11/09/2018 09:02 PM, Kim Barrett wrote: >> Almost, but not quite, I'm afraid. All but the first comment below >> are optional, though I think make things better. > > Introduced helper method, moved the small_range_words to private, handled (l == r) corner case in > test. See: > http://cr.openjdk.java.net/~shade/8211926/webrev.05/ Looks good. From kim.barrett at oracle.com Sat Nov 10 17:55:52 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 10 Nov 2018 12:55:52 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> Message-ID: > On Nov 10, 2018, at 7:58 AM, Thomas St?fe wrote: > - I find it confusing that we use the same type (idx_t) both for word- > and bit-indices. It would make the code a bit more readable if those > were separate types (even if they both were to alias to size_t, > ultimately). I dislike this too. I?d be interested in seeing a proposal to fix that, though I worry about the fanout. I *think* clients mostly operate in bits and the use of words is mostly limited to the implementation, but I haven?t done the searches needed to check that. > - Generally, I find putting the decision between "large" and > "non-large" APIs off to the caller a bit strange. As a user of bitmap, > I cannot know at which point which variant would be better. I would > prefer the class itself handling that. I was thinking much the same thing while reviewing this change. Maybe some of the public API in this area should be revisited. In particular, I was surprised that one might call ?large? functions directly, rather than calling hinted functions with a ?large? hint. Perhaps the ?large? functions should be private helpers that just do what they are told (maybe with an assert as before), and the calls to them should be based on the hint and the runtime size check they are now performing. Or maybe the whole hint thing should be reconsidered. I?d be interested in a proposal in this area too. From shade at redhat.com Sat Nov 10 18:03:00 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 10 Nov 2018 19:03:00 +0100 Subject: RFR 8213711 (XS): Zero build broken after JDK-8213199 (GC abstraction for Assembler::needs_explicit_null_check()) In-Reply-To: <1AFB282B-9AE8-4B91-819F-A8722AA604C2@redhat.com> References: <432c0207-e9df-4f45-f780-0e6851dd8690@redhat.com> <1AFB282B-9AE8-4B91-819F-A8722AA604C2@redhat.com> Message-ID: <7f778013-f3d8-abdb-9494-009049af8e85@redhat.com> Thanks! I think this is trivial, and needs a Reviewer acknowledging it :) -Aleksey On 11/10/2018 06:25 PM, Roman Kennke wrote: > Good. Thanks for fixing my mistake. > > Am 10. November 2018 18:19:00 MEZ schrieb Aleksey Shipilev : >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8213711 >> >> Fix is to add Zero declaration for the method added in JDK-8213199: >> >> 8213711: Zero build broken after JDK-8213199 (GC abstraction for >> Assembler::needs_explicit_null_check()) >> Reviewed-by: XXX >> >> diff -r 5643a5bff514 -r 919dde170921 >> src/hotspot/cpu/zero/assembler_zero.hpp >> --- a/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 09:06:33 >> 2018 -0500 >> +++ b/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 18:15:49 >> 2018 +0100 >> @@ -47,6 +47,7 @@ >> void align(int modulus); >> void bang_stack_with_offset(int offset); >> bool needs_explicit_null_check(intptr_t offset); >> + bool uses_implicit_null_check(void* address); >> RegisterOrConstant delayed_value_impl(intptr_t* delayed_value_addr, >> Register tmp, int offset); >> public: >> >> >> Testing: x86_64 {Zero,Server} builds >> >> Thanks, >> -Aleksey From thomas.stuefe at gmail.com Sat Nov 10 18:18:29 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 10 Nov 2018 19:18:29 +0100 Subject: RFR 8213711 (XS): Zero build broken after JDK-8213199 (GC abstraction for Assembler::needs_explicit_null_check()) In-Reply-To: <7f778013-f3d8-abdb-9494-009049af8e85@redhat.com> References: <432c0207-e9df-4f45-f780-0e6851dd8690@redhat.com> <1AFB282B-9AE8-4B91-819F-A8722AA604C2@redhat.com> <7f778013-f3d8-abdb-9494-009049af8e85@redhat.com> Message-ID: This is trivial. :) On Sat, Nov 10, 2018 at 7:03 PM Aleksey Shipilev wrote: > > Thanks! I think this is trivial, and needs a Reviewer acknowledging it :) > > -Aleksey > > On 11/10/2018 06:25 PM, Roman Kennke wrote: > > Good. Thanks for fixing my mistake. > > > > Am 10. November 2018 18:19:00 MEZ schrieb Aleksey Shipilev : > >> Bug: > >> https://bugs.openjdk.java.net/browse/JDK-8213711 > >> > >> Fix is to add Zero declaration for the method added in JDK-8213199: > >> > >> 8213711: Zero build broken after JDK-8213199 (GC abstraction for > >> Assembler::needs_explicit_null_check()) > >> Reviewed-by: XXX > >> > >> diff -r 5643a5bff514 -r 919dde170921 > >> src/hotspot/cpu/zero/assembler_zero.hpp > >> --- a/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 09:06:33 > >> 2018 -0500 > >> +++ b/src/hotspot/cpu/zero/assembler_zero.hpp Sat Nov 10 18:15:49 > >> 2018 +0100 > >> @@ -47,6 +47,7 @@ > >> void align(int modulus); > >> void bang_stack_with_offset(int offset); > >> bool needs_explicit_null_check(intptr_t offset); > >> + bool uses_implicit_null_check(void* address); > >> RegisterOrConstant delayed_value_impl(intptr_t* delayed_value_addr, > >> Register tmp, int offset); > >> public: > >> > >> > >> Testing: x86_64 {Zero,Server} builds > >> > >> Thanks, > >> -Aleksey > > From david.holmes at oracle.com Sun Nov 11 21:45:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Nov 2018 07:45:19 +1000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> Message-ID: <2e3be50f-b1c2-9a2f-e8c1-57df0cff3c62@oracle.com> Hi Boris, On 9/11/2018 9:03 PM, Boris Ulasevich wrote: > Hi David, > > Thank you. > > Don't you think it is better to learn jtreg to perform short-circuit > boolean evaluation? Sure but that will take a lot longer to happen. You'll need to file a bug in the CODE-TOOLS project and then discuss on code-tools-dev. Once the fix is accepted we then have to wait for the next release of jtreg to be promoted and then OpenJDK updated to start using it. Meanwhile if you want the tests to actually run they will need to be modified to work with existing jtreg behaviour. Thanks, David > http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02 > http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02_jtreg > > By the way, with the given change in jtreg two failed tests from the > original list works Ok because they already have 64-bit check on > previous lines: > * @requires (sun.arch.data.model == "64") > * @requires vm.opt.final.UseCompressedOops > > regards, > Boris > > On 09.11.2018 10:39, David Holmes wrote: >> Hi Boris, >> >> On 7/11/2018 6:54 PM, David Holmes wrote: >>> Hi Boris, >>> >>> On 7/11/2018 6:06 PM, Boris Ulasevich wrote: >>>> Hi David, >>>> >>>> ?? Yes, at first glance it is weird. We have actually three states >>>> for vm.opt.final.UseCompressedOops: true, false and null. Null means >>>> "not applicable" - when current VM does not support the option with >>>> the given name. Here is another approach for the issue (I am not >>>> sure it is a good one): >>>> ?? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ >>> >>> I like that approach! I'm just not sure everyone else will >>> necessarily agree :) This proposal might need a wider audience than >>> just hotspot-dev. >>> >>> Let me make enquiries with jtreg-use at openjdk.java.net and see what >>> the general intent was here. >> >> That was uninformative as to intent as Jon initially thought both >> variants should fail the same way, but on further reflection it is >> likely due to the difference between trying to evaluate null as a >> boolean - which gives an error - and doing "true".equals(null) , which >> is just false. >> >> I also see now that this isn't a general facility for using >> >> @requires >> >> because we only actually support three directly: >> >> ??protected void vmOptFinalFlags(Map map) { >> ???????? vmOptFinalFlag(map, "ClassUnloading"); >> ???????? vmOptFinalFlag(map, "UseCompressedOops"); >> ???????? vmOptFinalFlag(map, "EnableJVMCI"); >> ?? } >> >> but that also shows this is not specific to UseCompressedOops because >> EnableJVMCI only exists in a VM that was built with JVMCI support. >> >> I still think the correct way for this current case to be handled in >> tests is by using: >> >> @requires vm.bits == 64 & UseCompressedOops >> >> though that requires jtreg to do short-circuit boolean evaluation, >> which it appears not to do. :( Though: >> >> @requires vm.bits == 64 & UseCompressedOops == true >> >> should work (albeit perhaps due to implementation accident). >> >> Cheers, >> David >> ----- >> >>> Aside: if the UseCompressOops checks is preceded by a 64-bit check >>> does it still fail? (Just wondering whether jtreg does short-circuit >>> evaluation for these @require statements.) >>> >>> Thanks, >>> David >>> >>>> thank you, >>>> Boris >>>> >>>> On 07.11.2018 1:36, David Holmes wrote: >>>>> Hi Boris, >>>>> >>>>> I'm somewhat perplexed as to what the difference is between: >>>>> >>>>> @requires vm.opt.final.UseCompressedOops >>>>> >>>>> and >>>>> >>>>> @requires vm.opt.final.UseCompressedOops == true >>>>> >>>>> if they are indeed different then that seems to be a bug in the >>>>> requirement handling in the jtreg support code. >>>>> >>>>> I'm also wondering whether any such requirement should always be >>>>> proceeded by a check for 64-bits? >>>>> >>>>> Otherwise I would expect >>>>> >>>>> @requires vm.opt.final.UseCompressedOops >>>>> >>>>> to be false in 32-bit, so again a problem with the jtreg support code. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this patch to fix jtreg @requires >>>>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>>>>> ARM32: "invalid boolean value: null for expression >>>>>> vm.opt.final.UseCompressedOops". >>>>>> >>>>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>>>> >>>>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works >>>>>> Ok by default, and becomes disabled with >>>>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>>>> >>>>>> Thanks, >>>>>> Boris Ulasevich From david.holmes at oracle.com Sun Nov 11 21:47:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Nov 2018 07:47:01 +1000 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> Message-ID: On 11/11/2018 12:03 AM, Zhengyu Gu wrote: > Hi JC, > > Thanks for reviewing. > > On 11/9/18 10:24 PM, JC Beyler wrote: >> Hi Zhengyu, >> >> The fix looks good to me (I'm not a reviewer though ;-)), I was >> wondering if the other forName methods don't have the same issue: >> except002.java:? ? ? ? ? ? trash = >> Class.forName("nsk.stress.except.except002.except002$Abra$Cadabra"); >> // ??correct - should pass >> except002.java://? ? ? ? ? trash = >> Class.forName("nsk.stress.except.except002.except002.Abra.Cadabra"); >> // incorrect - should fail >> except003.java://? ? ? ? ? trash = >> Class.forName("nsk.stress.except.except003.except003$Abra$Cadabra"); >> // ??correct - should pass >> except003.java:? ? ? ? ? ? trash = >> Class.forName("nsk.stress.except.except003.except003.Abra.Cadabra"); >> // incorrect - should fail > > I got except002 timed out, has yet gotten to it. except003 passed, will > check if it is due to OOM. I would have fixed them anyway as they are obviously incorrect. David > This one happens to be low hanging fruit :-) > > Thanks, > > -Zhengyu > >> >> Seems like they also have an extra exceptXXX in the name, no? >> Could/Should we try to fix them at the same time? >> Jc >> >> >> On Fri, Nov 9, 2018 at 6:00 PM Zhengyu Gu > > wrote: >> >> ??? Thanks for reviewing, David. >> >> ??? On 11/9/18 5:58 PM, David Holmes wrote: >> ???? > Looks good - thanks for finding and fixing. >> ???? > >> ???? > I see cases internally where we fail because of this: >> ???? > >> ???? > Failure: ExceptionInInitializerError: target class not found >> ???? > >> ???? > but no bug was ever filed. I wonder if we will now fail because >> ??? of the >> ???? > OOME ... >> >> ??? Actually, it has to avoid OOME to get to here. >> >> ??? Several nsk/stress/except tests, e.g. except001, except002 and etc. >> ??? depend on quick OOME to pass. >> >> ??? Shenandoah has mechanism to gradually slow down allocation, so it is >> ??? more resist to OOME. Unfortunately, it usually fails due to timeout. >> >> ??? I am wondering of the usefulness of these tests. >> >> ??? Thanks, >> >> ??? -Zhengyu >> >> ???? > >> ???? > Thanks, >> ???? > David >> ???? > >> ???? > On 10/11/2018 7:43 AM, Zhengyu Gu wrote: >> ???? >> Please review this trivial fix. >> ???? >> >> ???? >> Currently, only Shenandoah experiences the failure, since >> other GCs >> ???? >> run out Java heap memory, then the rest of test is skipped. >> ???? >> >> ???? >> E.g. >> ???? >> pool[125131189]=new Object(); // elapsed 10.287s >> ???? >> pool[126306334]=new Object(); // elapsed 7.947s >> ???? >> Heap seems exhausted - OutOfMemoryError thrown. >> ???? >> Skipped: ExceptionInInitializerError: thrown OutOfMemoryError >> ???? >> Test passed. >> ???? >> >> ???? >> >> ???? >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 >> ???? >> Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ >> ???? >> >> ???? >> Thanks, >> ???? >> >> ???? >> -Zhengyu >> >> >> >> -- >> >> Thanks, >> Jc From zgu at redhat.com Mon Nov 12 00:10:27 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Sun, 11 Nov 2018 19:10:27 -0500 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> Message-ID: <6ed8cae5-9fd3-26bb-3fb9-dc529e7747f9@redhat.com> > > I would have fixed them anyway as they are obviously incorrect. > Oh, sure. I have not looked into them. Thanks, -Zhengyu > David > >> This one happens to be low hanging fruit :-) >> >> Thanks, >> >> -Zhengyu >> >>> >>> Seems like they also have an extra exceptXXX in the name, no? >>> Could/Should we try to fix them at the same time? >>> Jc >>> >>> >>> On Fri, Nov 9, 2018 at 6:00 PM Zhengyu Gu >> > wrote: >>> >>> ??? Thanks for reviewing, David. >>> >>> ??? On 11/9/18 5:58 PM, David Holmes wrote: >>> ???? > Looks good - thanks for finding and fixing. >>> ???? > >>> ???? > I see cases internally where we fail because of this: >>> ???? > >>> ???? > Failure: ExceptionInInitializerError: target class not found >>> ???? > >>> ???? > but no bug was ever filed. I wonder if we will now fail because >>> ??? of the >>> ???? > OOME ... >>> >>> ??? Actually, it has to avoid OOME to get to here. >>> >>> ??? Several nsk/stress/except tests, e.g. except001, except002 and etc. >>> ??? depend on quick OOME to pass. >>> >>> ??? Shenandoah has mechanism to gradually slow down allocation, so it is >>> ??? more resist to OOME. Unfortunately, it usually fails due to timeout. >>> >>> ??? I am wondering of the usefulness of these tests. >>> >>> ??? Thanks, >>> >>> ??? -Zhengyu >>> >>> ???? > >>> ???? > Thanks, >>> ???? > David >>> ???? > >>> ???? > On 10/11/2018 7:43 AM, Zhengyu Gu wrote: >>> ???? >> Please review this trivial fix. >>> ???? >> >>> ???? >> Currently, only Shenandoah experiences the failure, since >>> other GCs >>> ???? >> run out Java heap memory, then the rest of test is skipped. >>> ???? >> >>> ???? >> E.g. >>> ???? >> pool[125131189]=new Object(); // elapsed 10.287s >>> ???? >> pool[126306334]=new Object(); // elapsed 7.947s >>> ???? >> Heap seems exhausted - OutOfMemoryError thrown. >>> ???? >> Skipped: ExceptionInInitializerError: thrown OutOfMemoryError >>> ???? >> Test passed. >>> ???? >> >>> ???? >> >>> ???? >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 >>> ???? >> Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ >>> ???? >> >>> ???? >> Thanks, >>> ???? >> >>> ???? >> -Zhengyu >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc From zgu at redhat.com Mon Nov 12 00:17:16 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Sun, 11 Nov 2018 19:17:16 -0500 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> Message-ID: <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> On 11/11/18 4:47 PM, David Holmes wrote: > On 11/11/2018 12:03 AM, Zhengyu Gu wrote: >> Hi JC, >> >> Thanks for reviewing. >> >> On 11/9/18 10:24 PM, JC Beyler wrote: >>> Hi Zhengyu, >>> >>> The fix looks good to me (I'm not a reviewer though ;-)), I was >>> wondering if the other forName methods don't have the same issue: >>> except002.java:? ? ? ? ? ? trash = >>> Class.forName("nsk.stress.except.except002.except002$Abra$Cadabra"); >>> // ??correct - should pass >>> except002.java://? ? ? ? ? trash = >>> Class.forName("nsk.stress.except.except002.except002.Abra.Cadabra"); >>> // incorrect - should fail >>> except003.java://? ? ? ? ? trash = >>> Class.forName("nsk.stress.except.except003.except003$Abra$Cadabra"); >>> // ??correct - should pass >>> except003.java:? ? ? ? ? ? trash = >>> Class.forName("nsk.stress.except.except003.except003.Abra.Cadabra"); >>> // incorrect - should fail >> >> I got except002 timed out, has yet gotten to it. except003 passed, >> will check if it is due to OOM. > > I would have fixed them anyway as they are obviously incorrect. Bug filed: https://bugs.openjdk.java.net/browse/JDK-8213718 Thanks, -Zhengyu > > David > >> This one happens to be low hanging fruit :-) >> >> Thanks, >> >> -Zhengyu >> >>> >>> Seems like they also have an extra exceptXXX in the name, no? >>> Could/Should we try to fix them at the same time? >>> Jc >>> >>> >>> On Fri, Nov 9, 2018 at 6:00 PM Zhengyu Gu >> > wrote: >>> >>> ??? Thanks for reviewing, David. >>> >>> ??? On 11/9/18 5:58 PM, David Holmes wrote: >>> ???? > Looks good - thanks for finding and fixing. >>> ???? > >>> ???? > I see cases internally where we fail because of this: >>> ???? > >>> ???? > Failure: ExceptionInInitializerError: target class not found >>> ???? > >>> ???? > but no bug was ever filed. I wonder if we will now fail because >>> ??? of the >>> ???? > OOME ... >>> >>> ??? Actually, it has to avoid OOME to get to here. >>> >>> ??? Several nsk/stress/except tests, e.g. except001, except002 and etc. >>> ??? depend on quick OOME to pass. >>> >>> ??? Shenandoah has mechanism to gradually slow down allocation, so it is >>> ??? more resist to OOME. Unfortunately, it usually fails due to timeout. >>> >>> ??? I am wondering of the usefulness of these tests. >>> >>> ??? Thanks, >>> >>> ??? -Zhengyu >>> >>> ???? > >>> ???? > Thanks, >>> ???? > David >>> ???? > >>> ???? > On 10/11/2018 7:43 AM, Zhengyu Gu wrote: >>> ???? >> Please review this trivial fix. >>> ???? >> >>> ???? >> Currently, only Shenandoah experiences the failure, since >>> other GCs >>> ???? >> run out Java heap memory, then the rest of test is skipped. >>> ???? >> >>> ???? >> E.g. >>> ???? >> pool[125131189]=new Object(); // elapsed 10.287s >>> ???? >> pool[126306334]=new Object(); // elapsed 7.947s >>> ???? >> Heap seems exhausted - OutOfMemoryError thrown. >>> ???? >> Skipped: ExceptionInInitializerError: thrown OutOfMemoryError >>> ???? >> Test passed. >>> ???? >> >>> ???? >> >>> ???? >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 >>> ???? >> Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ >>> ???? >> >>> ???? >> Thanks, >>> ???? >> >>> ???? >> -Zhengyu >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc From david.holmes at oracle.com Mon Nov 12 00:22:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Nov 2018 10:22:05 +1000 Subject: RFR(XXS): 8213707: [TEST] vmTestbase/nsk/stress/except/except011.java failed due to wrong class name In-Reply-To: <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> Message-ID: <233f3e49-3182-5635-ad05-be66403f1db1@oracle.com> Thanks Zhengyu. I'll fix it. I want to check why we don't see them failing for this reason (or if we do why there is no bug filed for it!). David On 12/11/2018 10:17 AM, Zhengyu Gu wrote: > > > On 11/11/18 4:47 PM, David Holmes wrote: >> On 11/11/2018 12:03 AM, Zhengyu Gu wrote: >>> Hi JC, >>> >>> Thanks for reviewing. >>> >>> On 11/9/18 10:24 PM, JC Beyler wrote: >>>> Hi Zhengyu, >>>> >>>> The fix looks good to me (I'm not a reviewer though ;-)), I was >>>> wondering if the other forName methods don't have the same issue: >>>> except002.java:? ? ? ? ? ? trash = >>>> Class.forName("nsk.stress.except.except002.except002$Abra$Cadabra"); >>>> // ??correct - should pass >>>> except002.java://? ? ? ? ? trash = >>>> Class.forName("nsk.stress.except.except002.except002.Abra.Cadabra"); >>>> // incorrect - should fail >>>> except003.java://? ? ? ? ? trash = >>>> Class.forName("nsk.stress.except.except003.except003$Abra$Cadabra"); >>>> // ??correct - should pass >>>> except003.java:? ? ? ? ? ? trash = >>>> Class.forName("nsk.stress.except.except003.except003.Abra.Cadabra"); >>>> // incorrect - should fail >>> >>> I got except002 timed out, has yet gotten to it. except003 passed, >>> will check if it is due to OOM. >> >> I would have fixed them anyway as they are obviously incorrect. > > Bug filed: https://bugs.openjdk.java.net/browse/JDK-8213718 > > Thanks, > > -Zhengyu > >> >> David >> >>> This one happens to be low hanging fruit :-) >>> >>> Thanks, >>> >>> -Zhengyu >>> >>>> >>>> Seems like they also have an extra exceptXXX in the name, no? >>>> Could/Should we try to fix them at the same time? >>>> Jc >>>> >>>> >>>> On Fri, Nov 9, 2018 at 6:00 PM Zhengyu Gu >>> > wrote: >>>> >>>> ??? Thanks for reviewing, David. >>>> >>>> ??? On 11/9/18 5:58 PM, David Holmes wrote: >>>> ???? > Looks good - thanks for finding and fixing. >>>> ???? > >>>> ???? > I see cases internally where we fail because of this: >>>> ???? > >>>> ???? > Failure: ExceptionInInitializerError: target class not found >>>> ???? > >>>> ???? > but no bug was ever filed. I wonder if we will now fail because >>>> ??? of the >>>> ???? > OOME ... >>>> >>>> ??? Actually, it has to avoid OOME to get to here. >>>> >>>> ??? Several nsk/stress/except tests, e.g. except001, except002 and etc. >>>> ??? depend on quick OOME to pass. >>>> >>>> ??? Shenandoah has mechanism to gradually slow down allocation, so >>>> it is >>>> ??? more resist to OOME. Unfortunately, it usually fails due to >>>> timeout. >>>> >>>> ??? I am wondering of the usefulness of these tests. >>>> >>>> ??? Thanks, >>>> >>>> ??? -Zhengyu >>>> >>>> ???? > >>>> ???? > Thanks, >>>> ???? > David >>>> ???? > >>>> ???? > On 10/11/2018 7:43 AM, Zhengyu Gu wrote: >>>> ???? >> Please review this trivial fix. >>>> ???? >> >>>> ???? >> Currently, only Shenandoah experiences the failure, since >>>> other GCs >>>> ???? >> run out Java heap memory, then the rest of test is skipped. >>>> ???? >> >>>> ???? >> E.g. >>>> ???? >> pool[125131189]=new Object(); // elapsed 10.287s >>>> ???? >> pool[126306334]=new Object(); // elapsed 7.947s >>>> ???? >> Heap seems exhausted - OutOfMemoryError thrown. >>>> ???? >> Skipped: ExceptionInInitializerError: thrown OutOfMemoryError >>>> ???? >> Test passed. >>>> ???? >> >>>> ???? >> >>>> ???? >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213707 >>>> ???? >> Webrev: http://cr.openjdk.java.net/~zgu/8213707/webrev.00/ >>>> ???? >> >>>> ???? >> Thanks, >>>> ???? >> >>>> ???? >> -Zhengyu >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc From david.holmes at oracle.com Mon Nov 12 01:53:58 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Nov 2018 11:53:58 +1000 Subject: RFR(XXS): 8213718: [TEST] Wrong classname in vmTestbase/nsk/stress/except/except002 and except003 In-Reply-To: <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> Message-ID: <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> As a follow up to 8213707 ... Bug: https://bugs.openjdk.java.net/browse/JDK-8213718 webrev: http://cr.openjdk.java.net/~dholmes/8213718/webrev/ The extraneous package components are removed. Thanks, David From mvala at redhat.com Mon Nov 12 04:18:25 2018 From: mvala at redhat.com (Michal Vala) Date: Mon, 12 Nov 2018 05:18:25 +0100 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> Message-ID: <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> On 11/10/18 12:07 AM, David Holmes wrote: > cc'ing JC Beyler as this was his code. > > On 10/11/2018 4:35 AM, Kim Barrett wrote: >>> On Nov 9, 2018, at 11:43 AM, Michal Vala wrote: >>> >>> Hi, >>> >>> please review following patch fixing build failure on Windows with VS2013 >>> toolchain. > > Not sure we're still supporting VS2013 ... ?? Minimum accepted by configure is 2010. 2013 is 2nd in priorities after 2017. It has `VS_SUPPORTED_2013=false` though, but why not keep it buildable? > >>> >>> http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ >> >> The failure is in a hotspot test.? It should be using os::snprintf. > > Tests don't have access to os::snprintf. The test is just C++ code. > > Cheers, > David > > -- Michal Vala OpenJDK QE Red Hat Czech From kim.barrett at oracle.com Mon Nov 12 04:23:50 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 11 Nov 2018 23:23:50 -0500 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> Message-ID: > On Nov 9, 2018, at 6:07 PM, David Holmes wrote: > > cc'ing JC Beyler as this was his code. > > On 10/11/2018 4:35 AM, Kim Barrett wrote: >>> On Nov 9, 2018, at 11:43 AM, Michal Vala wrote: >>> >>> Hi, >>> >>> please review following patch fixing build failure on Windows with VS2013 toolchain. > > Not sure we're still supporting VS2013 ... ?? For now, we still claim to support VS2013; it's listed in the supported versions in make/autoconf/toolchains_windows.m4. That will change with JDK-8208089 (if not sooner). >>> http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ >> The failure is in a hotspot test. It should be using os::snprintf. > > Tests don't have access to os::snprintf. The test is just C++ code. Drat! Yeah, we don't seem to have access to VM code from these test support codes. That's too bad. However, the proposed change doesn't work as intended. There are two calls to snprintf in ExceptionCheckingJniEnv.cpp, which would be affected by the proposed change. The first is calling snprintf with an empty buffer to determine the size of the needed buffer. _snprintf returns -1 on buffer overflow, rather than the number of characters that would have been printed if the buffer size was sufficient to hold the output. So replacing that snprintf call with _snprintf will return a very different result from what is expected. I don't see a simple way to solve this while retaining the source calls to snprintf. That first snprintf call could be rewritten using several calls to strlen to calculate the needed size in some different manner. The later call could also be rewritten to use several calls to strcpy rather than snprintf. I think there currently aren't any other calls that would be affected, even though there are other #includes of the file being changed. But that's not trivial to check, and could change, possibly leading to hard to fathom bugs. There are 8 other calls to snprintf in test/hotspot/jtreg. Most of them are linux-only or solaris-only, so don't trip over the lack of snprintf on windows. However, one if them, in jvmti_tools.cpp, I think is applicable to Windows. It builds because it uses the renaming macro approach, with the macro being in jvmti_tools.h. Yuck (IMO). From kim.barrett at oracle.com Mon Nov 12 04:37:48 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 11 Nov 2018 23:37:48 -0500 Subject: RFR: 8213503: Build failure for SerialGC-only JDK Message-ID: Please review this small (trivial?) change, fixing some G1GC conditionalizations so that it is possible to configure and build a JDK that only provides SerialGC. CR: https://bugs.openjdk.java.net/browse/JDK-8213503 Webrev: http://cr.openjdk.java.net/~kbarrett/8213503/open.00/ Testing: mach5 tier1 (mostly to verify changes still build normally). Local build (linux-x64) configured with --with-jvm-features=-cmsgc,-epsilongc,-g1gc,-parallelgc,-zgc From david.holmes at oracle.com Mon Nov 12 05:00:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Nov 2018 15:00:50 +1000 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> Message-ID: Hi Michal, On 12/11/2018 2:18 PM, Michal Vala wrote: > > > On 11/10/18 12:07 AM, David Holmes wrote: >> cc'ing JC Beyler as this was his code. >> >> On 10/11/2018 4:35 AM, Kim Barrett wrote: >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala wrote: >>>> >>>> Hi, >>>> >>>> please review following patch fixing build failure on Windows with >>>> VS2013 toolchain. >> >> Not sure we're still supporting VS2013 ... ?? > > Minimum accepted by configure is 2010. 2013 is 2nd in priorities after > 2017. It has `VS_SUPPORTED_2013=false` though, but why not keep it > buildable? That depends on how painful it is to achieve that. As Kim has explained the suggested fix may allow building but it isn't correct. And we may not be able to keep this ability to build with VS2013 going forward. Thanks, David >> >>>> >>>> http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ >>> >>> The failure is in a hotspot test.? It should be using os::snprintf. >> >> Tests don't have access to os::snprintf. The test is just C++ code. >> >> Cheers, >> David >> >> > From david.holmes at oracle.com Mon Nov 12 05:41:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Nov 2018 15:41:50 +1000 Subject: RFR: 8213503: Build failure for SerialGC-only JDK In-Reply-To: References: Message-ID: Hi Kim, On 12/11/2018 2:37 PM, Kim Barrett wrote: > Please review this small (trivial?) change, fixing some G1GC > conditionalizations so that it is possible to configure and build a > JDK that only provides SerialGC. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8213503 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8213503/open.00/ That's one way to make it build, but ... I have to wonder whether the whole definition of the G1HeapRegionTypeConstant class and the registration of it: register_type(TYPE_G1HEAPREGIONTYPE, false, true, new G1HeapRegionTypeConstant()); shouldn't all be conditional on having G1? Thanks, David ----- > Testing: > mach5 tier1 (mostly to verify changes still build normally). > Local build (linux-x64) configured with > --with-jvm-features=-cmsgc,-epsilongc,-g1gc,-parallelgc,-zgc > From jcbeyler at google.com Mon Nov 12 06:39:01 2018 From: jcbeyler at google.com (JC Beyler) Date: Sun, 11 Nov 2018 22:39:01 -0800 Subject: RFR(XXS): 8213718: [TEST] Wrong classname in vmTestbase/nsk/stress/except/except002 and except003 In-Reply-To: <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> Message-ID: Hi David, Looks good to me :) Jc On Sun, Nov 11, 2018 at 5:54 PM David Holmes wrote: > As a follow up to 8213707 ... > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213718 > webrev: http://cr.openjdk.java.net/~dholmes/8213718/webrev/ > > The extraneous package components are removed. > > Thanks, > David > > -- Thanks, Jc From jcbeyler at google.com Mon Nov 12 06:38:32 2018 From: jcbeyler at google.com (JC Beyler) Date: Sun, 11 Nov 2018 22:38:32 -0800 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> Message-ID: Hi all, If I've caught up with the conversation, it sounds like: - My code breaks VS2013 build but that soon that won't be supported - We can't just do a renaming macro due to my use of sprintf with an empty buffer - So we need to either do what was suggested by Kim: "That first snprintf call could be rewritten using several calls to strlen to calculate the needed size in some different manner. The later call could also be rewritten to use several calls to strcpy rather than snprintf." Or something to that effect I can work on a fix that will handle this if we want, I'm just not sure if that's the path that is being recommended here though. It seems that in this particular case, it is not really hard to fix the code for now; the day VS2013 is no longer build-able by other pieces of code we can come back to this solution. To be honest, the reason this is like this is due to not being able to have C++ strings when I tried initially. The day we can have those, then this code can become even simpler. Let me know what you think and would prefer, Jc On Sun, Nov 11, 2018 at 9:01 PM David Holmes wrote: > Hi Michal, > > On 12/11/2018 2:18 PM, Michal Vala wrote: > > > > > > On 11/10/18 12:07 AM, David Holmes wrote: > >> cc'ing JC Beyler as this was his code. > >> > >> On 10/11/2018 4:35 AM, Kim Barrett wrote: > >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala wrote: > >>>> > >>>> Hi, > >>>> > >>>> please review following patch fixing build failure on Windows with > >>>> VS2013 toolchain. > >> > >> Not sure we're still supporting VS2013 ... ?? > > > > Minimum accepted by configure is 2010. 2013 is 2nd in priorities after > > 2017. It has `VS_SUPPORTED_2013=false` though, but why not keep it > > buildable? > > That depends on how painful it is to achieve that. As Kim has explained > the suggested fix may allow building but it isn't correct. > > And we may not be able to keep this ability to build with VS2013 going > forward. > > Thanks, > David > > > >> > >>>> > >>>> http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ > >>> > >>> The failure is in a hotspot test. It should be using os::snprintf. > >> > >> Tests don't have access to os::snprintf. The test is just C++ code. > >> > >> Cheers, > >> David > >> > >> > > > -- Thanks, Jc From thomas.stuefe at gmail.com Mon Nov 12 07:14:39 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 12 Nov 2018 08:14:39 +0100 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> Message-ID: Hi Kim, On Sat, Nov 10, 2018 at 6:55 PM Kim Barrett wrote: > > > On Nov 10, 2018, at 7:58 AM, Thomas St?fe wrote: > > - I find it confusing that we use the same type (idx_t) both for word- > > and bit-indices. It would make the code a bit more readable if those > > were separate types (even if they both were to alias to size_t, > > ultimately). > > I dislike this too. I?d be interested in seeing a proposal to fix that, though > I worry about the fanout. I *think* clients mostly operate in bits and the > use of words is mostly limited to the implementation, but I haven?t done > the searches needed to check that. Should I have the time I'll give this a try. Not sure when I will find the time though. I plan, at some point, to exchange my own hand-grown OccupancyBitmap in metaspace coding with this BitMap class. Maybe that will be a good time to look at it more closely. > > > - Generally, I find putting the decision between "large" and > > "non-large" APIs off to the caller a bit strange. As a user of bitmap, > > I cannot know at which point which variant would be better. I would > > prefer the class itself handling that. > > I was thinking much the same thing while reviewing this change. > Maybe some of the public API in this area should be revisited. > In particular, I was surprised that one might call ?large? functions > directly, rather than calling hinted functions with a ?large? hint. > Perhaps the ?large? functions should be private helpers that just > do what they are told (maybe with an assert as before), and the > calls to them should be based on the hint and the runtime size > check they are now performing. Or maybe the whole hint thing > should be reconsidered. I?d be interested in a proposal in this > area too. I would consider switching between loops and memset to be an internal tuning parameter which the caller should not have to bother with. I think hinting makes sense when I know something the code cannot know. But there are no hidden information here beside the range size. Unless I misunderstand the problem. Thanks, Thomas > From david.holmes at oracle.com Mon Nov 12 07:15:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 12 Nov 2018 17:15:12 +1000 Subject: RFR(XXS): 8213718: [TEST] Wrong classname in vmTestbase/nsk/stress/except/except002 and except003 In-Reply-To: References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> Message-ID: <1e5780b8-4e6d-2c24-ebed-367dd2ab099f@oracle.com> Thanks Jc :) David PS. I also filed 8213721 to explore possible issues these tests are exposing. On 12/11/2018 4:39 PM, JC Beyler wrote: > Hi David, > > Looks good to me :) > Jc > > On Sun, Nov 11, 2018 at 5:54 PM David Holmes > wrote: > > As a follow up to 8213707 ... > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213718 > webrev: http://cr.openjdk.java.net/~dholmes/8213718/webrev/ > > > The extraneous package components are removed. > > Thanks, > David > > > > -- > > Thanks, > Jc From kim.barrett at oracle.com Mon Nov 12 10:01:07 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 12 Nov 2018 05:01:07 -0500 Subject: RFR: 8213503: Build failure for SerialGC-only JDK In-Reply-To: References: Message-ID: <71EE3026-0703-416A-94DF-9725996CF58A@oracle.com> > On Nov 12, 2018, at 12:41 AM, David Holmes wrote: > > Hi Kim, > > On 12/11/2018 2:37 PM, Kim Barrett wrote: >> Please review this small (trivial?) change, fixing some G1GC >> conditionalizations so that it is possible to configure and build a >> JDK that only provides SerialGC. >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8213503 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8213503/open.00/ > > That's one way to make it build, but ... I have to wonder whether the whole definition of the G1HeapRegionTypeConstant class and the registration of it: > > register_type(TYPE_G1HEAPREGIONTYPE, false, true, new G1HeapRegionTypeConstant()); > > shouldn't all be conditional on having G1? I missed JDK-8209802 when I ran into this problem. Markus had a similar suggetion there. I?ll follow up on that, and probably close this one as a duplicate. > > Thanks, > David > ----- > >> Testing: >> mach5 tier1 (mostly to verify changes still build normally). >> Local build (linux-x64) configured with >> --with-jvm-features=-cmsgc,-epsilongc,-g1gc,-parallelgc,-zgc From boris.ulasevich at bell-sw.com Mon Nov 12 12:18:11 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Mon, 12 Nov 2018 15:18:11 +0300 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <2e3be50f-b1c2-9a2f-e8c1-57df0cff3c62@oracle.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> <2e3be50f-b1c2-9a2f-e8c1-57df0cff3c62@oracle.com> Message-ID: <08ce91a2-e2cc-5386-0b23-80f7dc6b7562@bell-sw.com> Hi David, Ok. Here is the solution exploiting 64-bit & UseCompressedOops == true check (works ok and ARM64 and makes tests disabled on ARM32): http://cr.openjdk.java.net/~bulasevich/8213410/webrev.03/ I will try to go forward on code-tools-dev and I will correct these tests later if we get short-circuit boolean evaluation in jtreg. thank you, Boris On 12.11.2018 0:45, David Holmes wrote: > Hi Boris, > > On 9/11/2018 9:03 PM, Boris Ulasevich wrote: >> Hi David, >> >> Thank you. >> >> Don't you think it is better to learn jtreg to perform short-circuit >> boolean evaluation? > > Sure but that will take a lot longer to happen. You'll need to file a > bug in the CODE-TOOLS project and then discuss on code-tools-dev. Once > the fix is accepted we then have to wait for the next release of jtreg > to be promoted and then OpenJDK updated to start using it. > > Meanwhile if you want the tests to actually run they will need to be > modified to work with existing jtreg behaviour. > > Thanks, > David > >> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02 >> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02_jtreg >> >> By the way, with the given change in jtreg two failed tests from the >> original list works Ok because they already have 64-bit check on >> previous lines: >> * @requires (sun.arch.data.model == "64") >> * @requires vm.opt.final.UseCompressedOops >> >> regards, >> Boris >> >> On 09.11.2018 10:39, David Holmes wrote: >>> Hi Boris, >>> >>> On 7/11/2018 6:54 PM, David Holmes wrote: >>>> Hi Boris, >>>> >>>> On 7/11/2018 6:06 PM, Boris Ulasevich wrote: >>>>> Hi David, >>>>> >>>>> ?? Yes, at first glance it is weird. We have actually three states >>>>> for vm.opt.final.UseCompressedOops: true, false and null. Null >>>>> means "not applicable" - when current VM does not support the >>>>> option with the given name. Here is another approach for the issue >>>>> (I am not sure it is a good one): >>>>> ?? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ >>>> >>>> I like that approach! I'm just not sure everyone else will >>>> necessarily agree :) This proposal might need a wider audience than >>>> just hotspot-dev. >>>> >>>> Let me make enquiries with jtreg-use at openjdk.java.net and see what >>>> the general intent was here. >>> >>> That was uninformative as to intent as Jon initially thought both >>> variants should fail the same way, but on further reflection it is >>> likely due to the difference between trying to evaluate null as a >>> boolean - which gives an error - and doing "true".equals(null) , >>> which is just false. >>> >>> I also see now that this isn't a general facility for using >>> >>> @requires >>> >>> because we only actually support three directly: >>> >>> ??protected void vmOptFinalFlags(Map map) { >>> ???????? vmOptFinalFlag(map, "ClassUnloading"); >>> ???????? vmOptFinalFlag(map, "UseCompressedOops"); >>> ???????? vmOptFinalFlag(map, "EnableJVMCI"); >>> ?? } >>> >>> but that also shows this is not specific to UseCompressedOops because >>> EnableJVMCI only exists in a VM that was built with JVMCI support. >>> >>> I still think the correct way for this current case to be handled in >>> tests is by using: >>> >>> @requires vm.bits == 64 & UseCompressedOops >>> >>> though that requires jtreg to do short-circuit boolean evaluation, >>> which it appears not to do. :( Though: >>> >>> @requires vm.bits == 64 & UseCompressedOops == true >>> >>> should work (albeit perhaps due to implementation accident). >>> >>> Cheers, >>> David >>> ----- >>> >>>> Aside: if the UseCompressOops checks is preceded by a 64-bit check >>>> does it still fail? (Just wondering whether jtreg does short-circuit >>>> evaluation for these @require statements.) >>>> >>>> Thanks, >>>> David >>>> >>>>> thank you, >>>>> Boris >>>>> >>>>> On 07.11.2018 1:36, David Holmes wrote: >>>>>> Hi Boris, >>>>>> >>>>>> I'm somewhat perplexed as to what the difference is between: >>>>>> >>>>>> @requires vm.opt.final.UseCompressedOops >>>>>> >>>>>> and >>>>>> >>>>>> @requires vm.opt.final.UseCompressedOops == true >>>>>> >>>>>> if they are indeed different then that seems to be a bug in the >>>>>> requirement handling in the jtreg support code. >>>>>> >>>>>> I'm also wondering whether any such requirement should always be >>>>>> proceeded by a check for 64-bits? >>>>>> >>>>>> Otherwise I would expect >>>>>> >>>>>> @requires vm.opt.final.UseCompressedOops >>>>>> >>>>>> to be false in 32-bit, so again a problem with the jtreg support >>>>>> code. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this patch to fix jtreg @requires >>>>>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced on >>>>>>> ARM32: "invalid boolean value: null for expression >>>>>>> vm.opt.final.UseCompressedOops". >>>>>>> >>>>>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>>>>> >>>>>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works >>>>>>> Ok by default, and becomes disabled with >>>>>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>>>>> >>>>>>> Thanks, >>>>>>> Boris Ulasevich From zgu at redhat.com Mon Nov 12 12:37:26 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 12 Nov 2018 07:37:26 -0500 Subject: RFR(XXS): 8213718: [TEST] Wrong classname in vmTestbase/nsk/stress/except/except002 and except003 In-Reply-To: <1e5780b8-4e6d-2c24-ebed-367dd2ab099f@oracle.com> References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> <1e5780b8-4e6d-2c24-ebed-367dd2ab099f@oracle.com> Message-ID: Sent from my iPad > On Nov 12, 2018, at 2:15 AM, David Holmes wrote: > > Thanks Jc :) > > David > > PS. I also filed 8213721 to explore possible issues these tests are exposing. > >> On 12/11/2018 4:39 PM, JC Beyler wrote: >> Hi David, >> Looks good to me :) >> Jc >> On Sun, Nov 11, 2018 at 5:54 PM David Holmes > wrote: >> As a follow up to 8213707 ... >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213718 >> webrev: http://cr.openjdk.java.net/~dholmes/8213718/webrev/ >> >> The extraneous package components are removed. >> Thanks, >> David >> -- >> Thanks, >> Jc From zgu at redhat.com Mon Nov 12 12:40:22 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 12 Nov 2018 07:40:22 -0500 Subject: RFR(XXS): 8213718: [TEST] Wrong classname in vmTestbase/nsk/stress/except/except002 and except003 In-Reply-To: <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> Message-ID: Looks good to me. Thanks for fixing them. -Zhengyu On 11/11/18 8:53 PM, David Holmes wrote: > As a follow up to 8213707 ... > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213718 > webrev: http://cr.openjdk.java.net/~dholmes/8213718/webrev/ > > The extraneous package components are removed. > > Thanks, > David > From martin.doerr at sap.com Mon Nov 12 15:29:53 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 12 Nov 2018 15:29:53 +0000 Subject: RFR: 8212585: Clean up CompiledMethod::oops_reloc_begin() In-Reply-To: <5BE017EE.2000202@oracle.com> References: <5BC756EE.6040900@oracle.com> <1e1788aa-b484-3656-9515-00b9c28c1484@oracle.com> <5BE017EE.2000202@oracle.com> Message-ID: Hi Erik, I still see crashes on SPARC and PPC64 when running jck tests: V [libjvm.so+0xef36a0] void DependencyContext::remove_dependent_nmethod(nmethod*,bool)+0x90 V [libjvm.so+0x1243688] void InstanceKlass::remove_dependent_nmethod(nmethod*,bool)+0xa8 V [libjvm.so+0x1b19978] void nmethod::flush_dependencies(bool)+0x388 V [libjvm.so+0x1b1842c] void nmethod::make_unloaded()+0x8c V [libjvm.so+0x1c249cc] void CodeCacheUnloadingTask::work(unsigned)+0x7c V [libjvm.so+0x1c24cf0] void ParallelCleaningTask::work(unsigned)+0x10 V [libjvm.so+0x21330b4] void GangWorker::loop()+0xa4 V [libjvm.so+0x1f94908] void Thread::call_run()+0x128 V [libjvm.so+0x1bc297c] thread_native_entry+0x3ec I haven't tried to debug, but I found a problem by reading code: We shouldn't use NativeJump::instruction_size in CompiledMethod::oops_reloc_begin(). On SPARC and PPC64, this size is the size of a large instruction sequence, but NativeJump::patch_verified_entry only patches a 4 byte trapping instruction. AFAICS only 4 byte is guaranteed to contain to oop (on these 2 platforms, other ones look ok). We could fix this constant on PPC64, but SPARC uses is for other purposes. Do you have a good proposal to fix it? Thanks and best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of Erik ?sterlund Sent: Montag, 5. November 2018 11:14 To: Per Liden ; hotspot-dev developers Subject: Re: RFR: 8212585: Clean up CompiledMethod::oops_reloc_begin() Hi Per, Thanks for the review. /Erik On 2018-11-05 10:51, Per Liden wrote: > Hi Erik, > > On 10/18/18 11:07 PM, Erik ?sterlund wrote: >> Hi, >> >> I decided to make this less risky for platforms that do not yet need >> to be scanned concurrently, and hence don't really need to be "fixed". >> >> In this new webrev, I decided to check for the frame completed offset >> if available and further away than verified entry + native jump size. >> Then it is safe for me to go ahead and scan this stuff concurrently >> using nmethod entry barriers. Otherwise, the same good old behaviour >> we used to rely on applies so that we won't get any surprises where >> we do not want them. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8212585/webrev.01/ > > Looks good! > > /Per > >> >> Thanks, >> /Erik >> >> On 2018-10-17 17:36, Erik ?sterlund wrote: >>> Hi, >>> >>> In CompiledMethod::oops_reloc_begin() there is code to prevent >>> looking for oops at the verified entry of an nmethod that is not >>> entrant, as a native jump instruction has been overwritten there. >>> Except there would *never* be any immediate oops there for any >>> compiler, even before becoming not entrant, with or without OSR >>> compilation, so this special handling of not entrant vs entrant >>> nmethods is seemingly completely unnecessary. >>> >>> This gets increasingly awkward when oops_do is called concurrently >>> with concurrent class unloading, where an nmethod is racing to >>> become not entrant. >>> >>> To clean this up, I propose to change the boundary for immediate oop >>> scanning and start looking for oops only after the frame building is >>> completed, as there is absolutely no point in looking for oops >>> before that in the first place. This removes unnecessary workarounds >>> that exist today, and removes unnecessary races going forward. >>> >>> It seems like Graal as JIT also inlines oops into the code stream, >>> but never sets the CodeOffsets::Frame_Complete in the nmethod. So I >>> recognize such nmethods and unconditionally start scanning at the >>> verified entry + native jump instruction size. I spoke to the Graal >>> folks, and they said they do not put oops in there either. So I >>> think everyone involved should be happy with this solution. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8212585 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8212585/webrev.00/ >>> >>> Thanks, >>> /Erik >> From jcbeyler at google.com Mon Nov 12 21:11:27 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 12 Nov 2018 13:11:27 -0800 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> Message-ID: Hi all, I created this change instead: http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ It removes the sprintf calls for strlen and strncpy calls. I checked that the code works if I force an error on GetObjectClass for example: FATAL ERROR in native method: GetObjectClass : Return is NULL at nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) at nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) at nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalNative(Native Method) at nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalSection(JNILocalRefLocker.java:44) at nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) at nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) at java.lang.Thread.run(java.base at 12-internal/Thread.java:835) I'll pass it through the submit repo if this looks like a better fix. Let me know what you think, Jc On Sun, Nov 11, 2018 at 10:38 PM JC Beyler wrote: > Hi all, > > If I've caught up with the conversation, it sounds like: > - My code breaks VS2013 build but that soon that won't be supported > - We can't just do a renaming macro due to my use of sprintf with an > empty buffer > - So we need to either do what was suggested by Kim: > "That first snprintf call could be rewritten using several calls to > strlen to calculate the needed size in some different manner. The > later call could also be rewritten to use several calls to strcpy > rather than snprintf." > Or something to that effect > > I can work on a fix that will handle this if we want, I'm just not sure if > that's the path that is being recommended here though. It seems that in > this particular case, it is not really hard to fix the code for now; the > day VS2013 is no longer build-able by other pieces of code we can come back > to this solution. To be honest, the reason this is like this is due to not > being able to have C++ strings when I tried initially. The day we can have > those, then this code can become even simpler. > > Let me know what you think and would prefer, > Jc > > On Sun, Nov 11, 2018 at 9:01 PM David Holmes > wrote: > >> Hi Michal, >> >> On 12/11/2018 2:18 PM, Michal Vala wrote: >> > >> > >> > On 11/10/18 12:07 AM, David Holmes wrote: >> >> cc'ing JC Beyler as this was his code. >> >> >> >> On 10/11/2018 4:35 AM, Kim Barrett wrote: >> >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala wrote: >> >>>> >> >>>> Hi, >> >>>> >> >>>> please review following patch fixing build failure on Windows with >> >>>> VS2013 toolchain. >> >> >> >> Not sure we're still supporting VS2013 ... ?? >> > >> > Minimum accepted by configure is 2010. 2013 is 2nd in priorities after >> > 2017. It has `VS_SUPPORTED_2013=false` though, but why not keep it >> > buildable? >> >> That depends on how painful it is to achieve that. As Kim has explained >> the suggested fix may allow building but it isn't correct. >> >> And we may not be able to keep this ability to build with VS2013 going >> forward. >> >> Thanks, >> David >> >> >> >> >> >>>> >> >>>> http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ >> >>> >> >>> The failure is in a hotspot test. It should be using os::snprintf. >> >> >> >> Tests don't have access to os::snprintf. The test is just C++ code. >> >> >> >> Cheers, >> >> David >> >> >> >> >> > >> > > > -- > > Thanks, > Jc > -- Thanks, Jc From david.holmes at oracle.com Mon Nov 12 21:34:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Nov 2018 07:34:07 +1000 Subject: RFR(XXS): 8213718: [TEST] Wrong classname in vmTestbase/nsk/stress/except/except002 and except003 In-Reply-To: References: <08425e45-5313-a53d-a376-42c35f70fca5@redhat.com> <2fa56d53-d675-b8e8-3837-867433f0ea02@redhat.com> <3427c088-c29c-b9a9-4b3e-94699df6e888@redhat.com> <3a23aeab-3b6f-707b-60c8-98fa91f6b77a@oracle.com> Message-ID: <7cb1116b-c44b-392a-51b0-3692682d42b6@oracle.com> Thanks Zhengyu. David On 12/11/2018 10:40 PM, Zhengyu Gu wrote: > Looks good to me. > > Thanks for fixing them. > > -Zhengyu > > On 11/11/18 8:53 PM, David Holmes wrote: >> As a follow up to 8213707 ... >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213718 >> webrev: http://cr.openjdk.java.net/~dholmes/8213718/webrev/ >> >> The extraneous package components are removed. >> >> Thanks, >> David >> From erik.osterlund at oracle.com Mon Nov 12 22:02:06 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 12 Nov 2018 23:02:06 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod Message-ID: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> Hi, In my change 8209189, I moved a lot of code around relating to code cache unloading. Unfortunately, I failed to see that when making nmethods unloaded, and their dependents are flushed... they are not immediately removed. Instead, they are tagged to have stale entries, that are later cleaned during weak metadata cleaning of InstanceKlass. As for method handles, they leak instead, with some hacks to reduce the leaking by expunging entries while adding and removing. Obviously, with InstanceKlass cleanup and code cache unloading now possibly running in parallel after 8209189, things can go wrong. These two phases used to be separated by a "wait barrier" for the worker threads preventing that parallelism. The reason dependency contexts were not cleaned when they were found during code cache cleaning, is because it was not thread safe when code cache unloading was made parallel instead of serial. But now that we are implementing concurrent class unloading, we are going to need concurrent cleanup of dependency contexts anyway. Therefore I will propose a bug fix that fixes the problem in a way that works for both serial, parallel and concurrent class unloading. The solution is to during code cache unloading claim cleaning of encountered stale dependency contexts, and clean them straight away. Entries are unlinked in a lock-free fashion, and placed on a purge list that is walked and deleted during ClassLoaderDataGraph::purge(). This follows the thinking of first unlinking, then syncing all threads, and then purging. New accessors for the head and next links, hide is_unloading entries and helps unlinking them and putting them into the purge list. Webrev: http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213565 Thanks, /Erik From david.holmes at oracle.com Mon Nov 12 22:08:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Nov 2018 08:08:18 +1000 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> Message-ID: <5038b4fe-862c-5324-fd79-4a58d8fb6793@oracle.com> Hi Jc, This seems okay to me. Only minor query is why you do the +1 (presumably for terminating NUL) on the return_len instead of len ? Thanks, David On 13/11/2018 7:11 AM, JC Beyler wrote: > Hi all, > > I created this change instead: > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ > > > It removes the sprintf calls for strlen and strncpy calls. I checked > that the code works if I force an error on GetObjectClass for example: > > FATAL ERROR in native method: GetObjectClass : Return is NULL > at > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > at > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > at nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalNative(Native Method) > at > nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalSection(JNILocalRefLocker.java:44) > at > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > at > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > at java.lang.Thread.run(java.base at 12-internal/Thread.java:835) > > I'll pass it through the submit repo if this looks like a better fix. > > Let me know what you think, > Jc > > On Sun, Nov 11, 2018 at 10:38 PM JC Beyler > wrote: > > Hi all, > > If I've caught up with the conversation, it sounds like: > ? - My code breaks VS2013 build but that soon that won't be supported > ? - We can't just do a renaming macro due to my use of sprintf with > an empty buffer > ? ? ? - So we need to either do what was suggested by Kim: > "That first snprintf call could be rewritten using several calls to > strlen to calculate the needed size in some different manner. The > later call could also be rewritten to use several calls to strcpy > rather than snprintf." > ? ? ? ? ?Or something to that effect > > I can work on a fix that will handle this if we want, I'm just not > sure if that's the path that is being recommended here though. It > seems that in this particular case, it is not really hard to fix the > code for now; the day VS2013 is no longer build-able by other pieces > of code we can come back to this solution. To be honest, the reason > this is like this is due to not being able to have C++ strings when > I tried initially. The day we can have those, then this code can > become even simpler. > > Let me know what you think and would prefer, > Jc > > On Sun, Nov 11, 2018 at 9:01 PM David Holmes > > wrote: > > Hi Michal, > > On 12/11/2018 2:18 PM, Michal Vala wrote: > > > > > > On 11/10/18 12:07 AM, David Holmes wrote: > >> cc'ing JC Beyler as this was his code. > >> > >> On 10/11/2018 4:35 AM, Kim Barrett wrote: > >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala > wrote: > >>>> > >>>> Hi, > >>>> > >>>> please review following patch fixing build failure on > Windows with > >>>> VS2013 toolchain. > >> > >> Not sure we're still supporting VS2013 ... ?? > > > > Minimum accepted by configure is 2010. 2013 is 2nd in > priorities after > > 2017. It has `VS_SUPPORTED_2013=false` though, but why not > keep it > > buildable? > > That depends on how painful it is to achieve that. As Kim has > explained > the suggested fix may allow building but it isn't correct. > > And we may not be able to keep this ability to build with VS2013 > going > forward. > > Thanks, > David > > > >> > >>>> > >>>> > http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ > > >>> > >>> The failure is in a hotspot test.? It should be using > os::snprintf. > >> > >> Tests don't have access to os::snprintf. The test is just > C++ code. > >> > >> Cheers, > >> David > >> > >> > > > > > > -- > > Thanks, > Jc > > > > -- > > Thanks, > Jc From david.holmes at oracle.com Mon Nov 12 22:35:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Nov 2018 08:35:15 +1000 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <08ce91a2-e2cc-5386-0b23-80f7dc6b7562@bell-sw.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> <2e3be50f-b1c2-9a2f-e8c1-57df0cff3c62@oracle.com> <08ce91a2-e2cc-5386-0b23-80f7dc6b7562@bell-sw.com> Message-ID: <725eb131-529d-83a5-e047-f08390ed0dec@oracle.com> Hi Boris, On 12/11/2018 10:18 PM, Boris Ulasevich wrote: > Hi David, > > Ok. Here is the solution exploiting 64-bit & UseCompressedOops == true > check (works ok and ARM64 and makes tests disabled on ARM32): > http://cr.openjdk.java.net/~bulasevich/8213410/webrev.03/ Looks good. > I will try to go forward on code-tools-dev and I will correct these > tests later if we get short-circuit boolean evaluation in jtreg. Thanks! David > thank you, > Boris > > On 12.11.2018 0:45, David Holmes wrote: >> Hi Boris, >> >> On 9/11/2018 9:03 PM, Boris Ulasevich wrote: >>> Hi David, >>> >>> Thank you. >>> >>> Don't you think it is better to learn jtreg to perform short-circuit >>> boolean evaluation? >> >> Sure but that will take a lot longer to happen. You'll need to file a >> bug in the CODE-TOOLS project and then discuss on code-tools-dev. Once >> the fix is accepted we then have to wait for the next release of jtreg >> to be promoted and then OpenJDK updated to start using it. >> >> Meanwhile if you want the tests to actually run they will need to be >> modified to work with existing jtreg behaviour. >> >> Thanks, >> David >> >>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02 >>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02_jtreg >>> >>> By the way, with the given change in jtreg two failed tests from the >>> original list works Ok because they already have 64-bit check on >>> previous lines: >>> * @requires (sun.arch.data.model == "64") >>> * @requires vm.opt.final.UseCompressedOops >>> >>> regards, >>> Boris >>> >>> On 09.11.2018 10:39, David Holmes wrote: >>>> Hi Boris, >>>> >>>> On 7/11/2018 6:54 PM, David Holmes wrote: >>>>> Hi Boris, >>>>> >>>>> On 7/11/2018 6:06 PM, Boris Ulasevich wrote: >>>>>> Hi David, >>>>>> >>>>>> ?? Yes, at first glance it is weird. We have actually three states >>>>>> for vm.opt.final.UseCompressedOops: true, false and null. Null >>>>>> means "not applicable" - when current VM does not support the >>>>>> option with the given name. Here is another approach for the issue >>>>>> (I am not sure it is a good one): >>>>>> ?? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ >>>>> >>>>> I like that approach! I'm just not sure everyone else will >>>>> necessarily agree :) This proposal might need a wider audience than >>>>> just hotspot-dev. >>>>> >>>>> Let me make enquiries with jtreg-use at openjdk.java.net and see what >>>>> the general intent was here. >>>> >>>> That was uninformative as to intent as Jon initially thought both >>>> variants should fail the same way, but on further reflection it is >>>> likely due to the difference between trying to evaluate null as a >>>> boolean - which gives an error - and doing "true".equals(null) , >>>> which is just false. >>>> >>>> I also see now that this isn't a general facility for using >>>> >>>> @requires >>>> >>>> because we only actually support three directly: >>>> >>>> ??protected void vmOptFinalFlags(Map map) { >>>> ???????? vmOptFinalFlag(map, "ClassUnloading"); >>>> ???????? vmOptFinalFlag(map, "UseCompressedOops"); >>>> ???????? vmOptFinalFlag(map, "EnableJVMCI"); >>>> ?? } >>>> >>>> but that also shows this is not specific to UseCompressedOops >>>> because EnableJVMCI only exists in a VM that was built with JVMCI >>>> support. >>>> >>>> I still think the correct way for this current case to be handled in >>>> tests is by using: >>>> >>>> @requires vm.bits == 64 & UseCompressedOops >>>> >>>> though that requires jtreg to do short-circuit boolean evaluation, >>>> which it appears not to do. :( Though: >>>> >>>> @requires vm.bits == 64 & UseCompressedOops == true >>>> >>>> should work (albeit perhaps due to implementation accident). >>>> >>>> Cheers, >>>> David >>>> ----- >>>> >>>>> Aside: if the UseCompressOops checks is preceded by a 64-bit check >>>>> does it still fail? (Just wondering whether jtreg does >>>>> short-circuit evaluation for these @require statements.) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> thank you, >>>>>> Boris >>>>>> >>>>>> On 07.11.2018 1:36, David Holmes wrote: >>>>>>> Hi Boris, >>>>>>> >>>>>>> I'm somewhat perplexed as to what the difference is between: >>>>>>> >>>>>>> @requires vm.opt.final.UseCompressedOops >>>>>>> >>>>>>> and >>>>>>> >>>>>>> @requires vm.opt.final.UseCompressedOops == true >>>>>>> >>>>>>> if they are indeed different then that seems to be a bug in the >>>>>>> requirement handling in the jtreg support code. >>>>>>> >>>>>>> I'm also wondering whether any such requirement should always be >>>>>>> proceeded by a check for 64-bits? >>>>>>> >>>>>>> Otherwise I would expect >>>>>>> >>>>>>> @requires vm.opt.final.UseCompressedOops >>>>>>> >>>>>>> to be false in 32-bit, so again a problem with the jtreg support >>>>>>> code. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review this patch to fix jtreg @requires >>>>>>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced >>>>>>>> on ARM32: "invalid boolean value: null for expression >>>>>>>> vm.opt.final.UseCompressedOops". >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>>>>>> >>>>>>>> The fix was checked on ARM32 (tests disabled) and AARCH64 (works >>>>>>>> Ok by default, and becomes disabled with >>>>>>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Boris Ulasevich From erik.osterlund at oracle.com Mon Nov 12 22:46:28 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 12 Nov 2018 23:46:28 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> Message-ID: ..put in bug number in subject to make mail filters happy. /Erik On 2018-11-12 23:44, Erik ?sterlund wrote: > Hi, > > All current GCs perform code cache unloading in safepoints. Therefore, > nmethods that are alive but is_unloading() are never observed outside of > safepoints. With concurrent class unloading, nmethods that are alive but > is_unloading() will become observable outside of safepoints. This must > be handled appropriately. > > In this patch I changed the nmethod/compiled method iterators to accept > parameters describing whether they should filter out not is_alive() or > is_unloading() methods. Since there is no obvious default (all > combinations are used depending on call site), you have to explicitly > set what iteration mode you want. > > Other than that, I make sure that the sweeper stays away from > is_unloading() nmethods that are not yet is_unloaded(). To make the > interactions between the sweeper and concurrent GC threads safe, I had > to move down the store that sets the state to unloaded, and use a > release_store there, to make sure no accesses float below it at > compile-time. Once that store is observed, nmethods may be deleted. > > In the IC miss handler, I also need to lazily clean stale IC caches due > to calling is_unloading nmethods using nmethod entry barriers. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213755 > > Thanks, > /Erik From erik.osterlund at oracle.com Mon Nov 12 22:44:33 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 12 Nov 2018 23:44:33 +0100 Subject: RFR: Let nmethods be is_unloading() outside of safepoints Message-ID: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> Hi, All current GCs perform code cache unloading in safepoints. Therefore, nmethods that are alive but is_unloading() are never observed outside of safepoints. With concurrent class unloading, nmethods that are alive but is_unloading() will become observable outside of safepoints. This must be handled appropriately. In this patch I changed the nmethod/compiled method iterators to accept parameters describing whether they should filter out not is_alive() or is_unloading() methods. Since there is no obvious default (all combinations are used depending on call site), you have to explicitly set what iteration mode you want. Other than that, I make sure that the sweeper stays away from is_unloading() nmethods that are not yet is_unloaded(). To make the interactions between the sweeper and concurrent GC threads safe, I had to move down the store that sets the state to unloaded, and use a release_store there, to make sure no accesses float below it at compile-time. Once that store is observed, nmethods may be deleted. In the IC miss handler, I also need to lazily clean stale IC caches due to calling is_unloading nmethods using nmethod entry barriers. Webrev: http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213755 Thanks, /Erik From jcbeyler at google.com Tue Nov 13 00:08:21 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 12 Nov 2018 16:08:21 -0800 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: <5038b4fe-862c-5324-fd79-4a58d8fb6793@oracle.com> References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> <5038b4fe-862c-5324-fd79-4a58d8fb6793@oracle.com> Message-ID: Hi all, @Mark: good point, fixed in the new webrev @David: also good point, just because originally it was written differently and I moved the code to this format and didn't move the +1 to the "right" spot @Michal: do you mind if I take over the bug and push this fix, could you review it as well? Here is the new webrev: http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.01/ Here is the bug: https://bugs.openjdk.java.net/browse/JDK-8213622 Thanks, Jc On Mon, Nov 12, 2018 at 2:08 PM David Holmes wrote: > Hi Jc, > > This seems okay to me. Only minor query is why you do the +1 (presumably > for terminating NUL) on the return_len instead of len ? > > Thanks, > David > > On 13/11/2018 7:11 AM, JC Beyler wrote: > > Hi all, > > > > I created this change instead: > > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ > > > > > > It removes the sprintf calls for strlen and strncpy calls. I checked > > that the code works if I force an error on GetObjectClass for example: > > > > FATAL ERROR in native method: GetObjectClass : Return is NULL > > at > > > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > > at > > > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > > at nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalNative(Native > Method) > > at > > > nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalSection(JNILocalRefLocker.java:44) > > at > > > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > > at > > > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > > at java.lang.Thread.run(java.base at 12-internal/Thread.java:835) > > > > I'll pass it through the submit repo if this looks like a better fix. > > > > Let me know what you think, > > Jc > > > > On Sun, Nov 11, 2018 at 10:38 PM JC Beyler > > wrote: > > > > Hi all, > > > > If I've caught up with the conversation, it sounds like: > > - My code breaks VS2013 build but that soon that won't be > supported > > - We can't just do a renaming macro due to my use of sprintf with > > an empty buffer > > - So we need to either do what was suggested by Kim: > > "That first snprintf call could be rewritten using several calls to > > strlen to calculate the needed size in some different manner. The > > later call could also be rewritten to use several calls to strcpy > > rather than snprintf." > > Or something to that effect > > > > I can work on a fix that will handle this if we want, I'm just not > > sure if that's the path that is being recommended here though. It > > seems that in this particular case, it is not really hard to fix the > > code for now; the day VS2013 is no longer build-able by other pieces > > of code we can come back to this solution. To be honest, the reason > > this is like this is due to not being able to have C++ strings when > > I tried initially. The day we can have those, then this code can > > become even simpler. > > > > Let me know what you think and would prefer, > > Jc > > > > On Sun, Nov 11, 2018 at 9:01 PM David Holmes > > > wrote: > > > > Hi Michal, > > > > On 12/11/2018 2:18 PM, Michal Vala wrote: > > > > > > > > > On 11/10/18 12:07 AM, David Holmes wrote: > > >> cc'ing JC Beyler as this was his code. > > >> > > >> On 10/11/2018 4:35 AM, Kim Barrett wrote: > > >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala > > wrote: > > >>>> > > >>>> Hi, > > >>>> > > >>>> please review following patch fixing build failure on > > Windows with > > >>>> VS2013 toolchain. > > >> > > >> Not sure we're still supporting VS2013 ... ?? > > > > > > Minimum accepted by configure is 2010. 2013 is 2nd in > > priorities after > > > 2017. It has `VS_SUPPORTED_2013=false` though, but why not > > keep it > > > buildable? > > > > That depends on how painful it is to achieve that. As Kim has > > explained > > the suggested fix may allow building but it isn't correct. > > > > And we may not be able to keep this ability to build with VS2013 > > going > > forward. > > > > Thanks, > > David > > > > > > >> > > >>>> > > >>>> > > http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ > > < > http://cr.openjdk.java.net/%7Emvala/jdk/jdk/JDK-8213622/webrev.00/> > > >>> > > >>> The failure is in a hotspot test. It should be using > > os::snprintf. > > >> > > >> Tests don't have access to os::snprintf. The test is just > > C++ code. > > >> > > >> Cheers, > > >> David > > >> > > >> > > > > > > > > > > > -- > > > > Thanks, > > Jc > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc From david.holmes at oracle.com Tue Nov 13 00:20:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Nov 2018 10:20:37 +1000 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> <5038b4fe-862c-5324-fd79-4a58d8fb6793@oracle.com> Message-ID: Hi Jc, On 13/11/2018 10:08 AM, JC Beyler wrote: > Hi all, > > @Mark: good point, fixed in the new webrev I assume this was the strncpy -> memcpy change as I haven't see any email from Mark. What was the issue? Update is fine anyway. Thanks, David ----- > @David: also good point, just because originally it was written > differently and I moved the code to this format and didn't move the?+1 > to the "right" spot > @Michal: do you mind if I take over the bug and push this fix, could you > review it as well? > > Here is the new webrev: > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.01/ > > Here is the bug: https://bugs.openjdk.java.net/browse/JDK-8213622 > > Thanks, > Jc > > On Mon, Nov 12, 2018 at 2:08 PM David Holmes > wrote: > > Hi Jc, > > This seems okay to me. Only minor query is why you do the +1 > (presumably > for terminating NUL) on the return_len instead of len ? > > Thanks, > David > > On 13/11/2018 7:11 AM, JC Beyler wrote: > > Hi all, > > > > I created this change instead: > > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ > > > > > > > It removes the sprintf calls for strlen and strncpy calls. I checked > > that the code works if I force an error on GetObjectClass for > example: > > > > FATAL ERROR in native method: GetObjectClass : Return is NULL > > at > > > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > > at > > > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > > at > nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalNative(Native Method) > > at > > > nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalSection(JNILocalRefLocker.java:44) > > at > > > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > > at > > > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > > at java.lang.Thread.run(java.base at 12-internal/Thread.java:835) > > > > I'll pass it through the submit repo if this looks like a better fix. > > > > Let me know what you think, > > Jc > > > > On Sun, Nov 11, 2018 at 10:38 PM JC Beyler > > >> wrote: > > > >? ? ?Hi all, > > > >? ? ?If I've caught up with the conversation, it sounds like: > >? ? ? ? - My code breaks VS2013 build but that soon that won't be > supported > >? ? ? ? - We can't just do a renaming macro due to my use of > sprintf with > >? ? ?an empty buffer > >? ? ? ? ? ? - So we need to either do what was suggested by Kim: > >? ? ?"That first snprintf call could be rewritten using several > calls to > >? ? ?strlen to calculate the needed size in some different manner. The > >? ? ?later call could also be rewritten to use several calls to strcpy > >? ? ?rather than snprintf." > >? ? ? ? ? ? ? ?Or something to that effect > > > >? ? ?I can work on a fix that will handle this if we want, I'm > just not > >? ? ?sure if that's the path that is being recommended here though. It > >? ? ?seems that in this particular case, it is not really hard to > fix the > >? ? ?code for now; the day VS2013 is no longer build-able by other > pieces > >? ? ?of code we can come back to this solution. To be honest, the > reason > >? ? ?this is like this is due to not being able to have C++ > strings when > >? ? ?I tried initially. The day we can have those, then this code can > >? ? ?become even simpler. > > > >? ? ?Let me know what you think and would prefer, > >? ? ?Jc > > > >? ? ?On Sun, Nov 11, 2018 at 9:01 PM David Holmes > >? ? ? > >> > wrote: > > > >? ? ? ? ?Hi Michal, > > > >? ? ? ? ?On 12/11/2018 2:18 PM, Michal Vala wrote: > >? ? ? ? ? > > >? ? ? ? ? > > >? ? ? ? ? > On 11/10/18 12:07 AM, David Holmes wrote: > >? ? ? ? ? >> cc'ing JC Beyler as this was his code. > >? ? ? ? ? >> > >? ? ? ? ? >> On 10/11/2018 4:35 AM, Kim Barrett wrote: > >? ? ? ? ? >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala > > >? ? ? ? ?>> wrote: > >? ? ? ? ? >>>> > >? ? ? ? ? >>>> Hi, > >? ? ? ? ? >>>> > >? ? ? ? ? >>>> please review following patch fixing build failure on > >? ? ? ? ?Windows with > >? ? ? ? ? >>>> VS2013 toolchain. > >? ? ? ? ? >> > >? ? ? ? ? >> Not sure we're still supporting VS2013 ... ?? > >? ? ? ? ? > > >? ? ? ? ? > Minimum accepted by configure is 2010. 2013 is 2nd in > >? ? ? ? ?priorities after > >? ? ? ? ? > 2017. It has `VS_SUPPORTED_2013=false` though, but why not > >? ? ? ? ?keep it > >? ? ? ? ? > buildable? > > > >? ? ? ? ?That depends on how painful it is to achieve that. As Kim has > >? ? ? ? ?explained > >? ? ? ? ?the suggested fix may allow building but it isn't correct. > > > >? ? ? ? ?And we may not be able to keep this ability to build with > VS2013 > >? ? ? ? ?going > >? ? ? ? ?forward. > > > >? ? ? ? ?Thanks, > >? ? ? ? ?David > > > > > >? ? ? ? ? >> > >? ? ? ? ? >>>> > >? ? ? ? ? >>>> > > http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ > > > > ? > >? ? ? ? ? >>> > >? ? ? ? ? >>> The failure is in a hotspot test.? It should be using > >? ? ? ? ?os::snprintf. > >? ? ? ? ? >> > >? ? ? ? ? >> Tests don't have access to os::snprintf. The test is just > >? ? ? ? ?C++ code. > >? ? ? ? ? >> > >? ? ? ? ? >> Cheers, > >? ? ? ? ? >> David > >? ? ? ? ? >> > >? ? ? ? ? >> > >? ? ? ? ? > > > > > > > > >? ? ?-- > > > >? ? ?Thanks, > >? ? ?Jc > > > > > > > > -- > > > > Thanks, > > Jc > > > > -- > > Thanks, > Jc From jcbeyler at google.com Tue Nov 13 00:50:08 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 12 Nov 2018 16:50:08 -0800 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> <5038b4fe-862c-5324-fd79-4a58d8fb6793@oracle.com> Message-ID: Hi David, Yes sorry, I had not seen that Mark Ludwig had only sent to me his comment that included this part: "if you're going to bother with strlen(), you know exactly how many bytes to copy, so don't use any strXXX API at all - just memcpy()." Does that make sense? Thanks, Jc On Mon, Nov 12, 2018 at 4:20 PM David Holmes wrote: > Hi Jc, > > On 13/11/2018 10:08 AM, JC Beyler wrote: > > Hi all, > > > > @Mark: good point, fixed in the new webrev > > I assume this was the strncpy -> memcpy change as I haven't see any > email from Mark. What was the issue? > > Update is fine anyway. > > Thanks, > David > ----- > > > > > @David: also good point, just because originally it was written > > differently and I moved the code to this format and didn't move the +1 > > to the "right" spot > > @Michal: do you mind if I take over the bug and push this fix, could you > > review it as well? > > > > Here is the new webrev: > > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.01/ > > > > Here is the bug: https://bugs.openjdk.java.net/browse/JDK-8213622 > > > > Thanks, > > Jc > > > > On Mon, Nov 12, 2018 at 2:08 PM David Holmes > > wrote: > > > > Hi Jc, > > > > This seems okay to me. Only minor query is why you do the +1 > > (presumably > > for terminating NUL) on the return_len instead of len ? > > > > Thanks, > > David > > > > On 13/11/2018 7:11 AM, JC Beyler wrote: > > > Hi all, > > > > > > I created this change instead: > > > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ > > > > > > > > > > > It removes the sprintf calls for strlen and strncpy calls. I > checked > > > that the code works if I force an error on GetObjectClass for > > example: > > > > > > FATAL ERROR in native method: GetObjectClass : Return is NULL > > > at > > > > > > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > > > at > > > > > > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > > > at > > nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalNative(Native > Method) > > > at > > > > > > nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalSection(JNILocalRefLocker.java:44) > > > at > > > > > > nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > > > at > > > > > > nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > > > at java.lang.Thread.run(java.base at 12-internal/Thread.java:835) > > > > > > I'll pass it through the submit repo if this looks like a better > fix. > > > > > > Let me know what you think, > > > Jc > > > > > > On Sun, Nov 11, 2018 at 10:38 PM JC Beyler > > > > >> wrote: > > > > > > Hi all, > > > > > > If I've caught up with the conversation, it sounds like: > > > - My code breaks VS2013 build but that soon that won't be > > supported > > > - We can't just do a renaming macro due to my use of > > sprintf with > > > an empty buffer > > > - So we need to either do what was suggested by Kim: > > > "That first snprintf call could be rewritten using several > > calls to > > > strlen to calculate the needed size in some different manner. > The > > > later call could also be rewritten to use several calls to > strcpy > > > rather than snprintf." > > > Or something to that effect > > > > > > I can work on a fix that will handle this if we want, I'm > > just not > > > sure if that's the path that is being recommended here > though. It > > > seems that in this particular case, it is not really hard to > > fix the > > > code for now; the day VS2013 is no longer build-able by other > > pieces > > > of code we can come back to this solution. To be honest, the > > reason > > > this is like this is due to not being able to have C++ > > strings when > > > I tried initially. The day we can have those, then this code > can > > > become even simpler. > > > > > > Let me know what you think and would prefer, > > > Jc > > > > > > On Sun, Nov 11, 2018 at 9:01 PM David Holmes > > > > > >> > > wrote: > > > > > > Hi Michal, > > > > > > On 12/11/2018 2:18 PM, Michal Vala wrote: > > > > > > > > > > > > On 11/10/18 12:07 AM, David Holmes wrote: > > > >> cc'ing JC Beyler as this was his code. > > > >> > > > >> On 10/11/2018 4:35 AM, Kim Barrett wrote: > > > >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala > > > > > >> > wrote: > > > >>>> > > > >>>> Hi, > > > >>>> > > > >>>> please review following patch fixing build failure > on > > > Windows with > > > >>>> VS2013 toolchain. > > > >> > > > >> Not sure we're still supporting VS2013 ... ?? > > > > > > > > Minimum accepted by configure is 2010. 2013 is 2nd in > > > priorities after > > > > 2017. It has `VS_SUPPORTED_2013=false` though, but why > not > > > keep it > > > > buildable? > > > > > > That depends on how painful it is to achieve that. As Kim > has > > > explained > > > the suggested fix may allow building but it isn't correct. > > > > > > And we may not be able to keep this ability to build with > > VS2013 > > > going > > > forward. > > > > > > Thanks, > > > David > > > > > > > > > >> > > > >>>> > > > >>>> > > > http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ > > > > > > > < > http://cr.openjdk.java.net/%7Emvala/jdk/jdk/JDK-8213622/webrev.00/> > > > >>> > > > >>> The failure is in a hotspot test. It should be using > > > os::snprintf. > > > >> > > > >> Tests don't have access to os::snprintf. The test is > just > > > C++ code. > > > >> > > > >> Cheers, > > > >> David > > > >> > > > >> > > > > > > > > > > > > > > > > -- > > > > > > Thanks, > > > Jc > > > > > > > > > > > > -- > > > > > > Thanks, > > > Jc > > > > > > > > -- > > > > Thanks, > > Jc > -- Thanks, Jc From david.holmes at oracle.com Tue Nov 13 01:23:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 13 Nov 2018 11:23:19 +1000 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> <5038b4fe-862c-5324-fd79-4a58d8fb6793@oracle.com> Message-ID: <63815096-97a0-eaab-3bcb-7ff0cbd31c5f@oracle.com> Hi Jc, On 13/11/2018 10:50 AM, JC Beyler wrote: > Hi David, > > Yes sorry, I had not seen that Mark Ludwig had only sent to me his > comment that included this part: "if you're going to bother with > strlen(), you know exactly how many bytes to copy, so don't use any > strXXX API at all - just memcpy()." > > Does?that make sense? Sure - micro-optimization but that's fine. Cheers, David > Thanks, > Jc > > On Mon, Nov 12, 2018 at 4:20 PM David Holmes > wrote: > > Hi Jc, > > On 13/11/2018 10:08 AM, JC Beyler wrote: > > Hi all, > > > > @Mark: good point, fixed in the new webrev > > I assume this was the strncpy -> memcpy change as I haven't see any > email from Mark. What was the issue? > > Update is fine anyway. > > Thanks, > David > ----- > > > > > @David: also good point, just because originally it was written > > differently and I moved the code to this format and didn't move > the?+1 > > to the "right" spot > > @Michal: do you mind if I take over the bug and push this fix, > could you > > review it as well? > > > > Here is the new webrev: > > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.01/ > > > > > Here is the bug: https://bugs.openjdk.java.net/browse/JDK-8213622 > > > > Thanks, > > Jc > > > > On Mon, Nov 12, 2018 at 2:08 PM David Holmes > > > >> wrote: > > > >? ? ?Hi Jc, > > > >? ? ?This seems okay to me. Only minor query is why you do the +1 > >? ? ?(presumably > >? ? ?for terminating NUL) on the return_len instead of len ? > > > >? ? ?Thanks, > >? ? ?David > > > >? ? ?On 13/11/2018 7:11 AM, JC Beyler wrote: > >? ? ? > Hi all, > >? ? ? > > >? ? ? > I created this change instead: > >? ? ? > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ > > >? ? ? > >? ? ? > > >? ? ? > > >? ? ? > It removes the sprintf calls for strlen and strncpy calls. > I checked > >? ? ? > that the code works if I force an error on GetObjectClass for > >? ? ?example: > >? ? ? > > >? ? ? > FATAL ERROR in native method: GetObjectClass : Return is NULL > >? ? ? > at > >? ? ? > > > > ?nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > >? ? ? > at > >? ? ? > > > > ?nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > >? ? ? > at > > > ?nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalNative(Native > Method) > >? ? ? > at > >? ? ? > > > > ?nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalSection(JNILocalRefLocker.java:44) > >? ? ? > at > >? ? ? > > > > ?nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) > >? ? ? > at > >? ? ? > > > > ?nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) > >? ? ? > at java.lang.Thread.run(java.base at 12-internal/Thread.java:835) > >? ? ? > > >? ? ? > I'll pass it through the submit repo if this looks like a > better fix. > >? ? ? > > >? ? ? > Let me know what you think, > >? ? ? > Jc > >? ? ? > > >? ? ? > On Sun, Nov 11, 2018 at 10:38 PM JC Beyler > > >? ? ?> > >? ? ? > > >>> wrote: > >? ? ? > > >? ? ? >? ? ?Hi all, > >? ? ? > > >? ? ? >? ? ?If I've caught up with the conversation, it sounds like: > >? ? ? >? ? ? ? - My code breaks VS2013 build but that soon that > won't be > >? ? ?supported > >? ? ? >? ? ? ? - We can't just do a renaming macro due to my use of > >? ? ?sprintf with > >? ? ? >? ? ?an empty buffer > >? ? ? >? ? ? ? ? ? - So we need to either do what was suggested by > Kim: > >? ? ? >? ? ?"That first snprintf call could be rewritten using several > >? ? ?calls to > >? ? ? >? ? ?strlen to calculate the needed size in some different > manner. The > >? ? ? >? ? ?later call could also be rewritten to use several > calls to strcpy > >? ? ? >? ? ?rather than snprintf." > >? ? ? >? ? ? ? ? ? ? ?Or something to that effect > >? ? ? > > >? ? ? >? ? ?I can work on a fix that will handle this if we want, I'm > >? ? ?just not > >? ? ? >? ? ?sure if that's the path that is being recommended here > though. It > >? ? ? >? ? ?seems that in this particular case, it is not really > hard to > >? ? ?fix the > >? ? ? >? ? ?code for now; the day VS2013 is no longer build-able > by other > >? ? ?pieces > >? ? ? >? ? ?of code we can come back to this solution. To be > honest, the > >? ? ?reason > >? ? ? >? ? ?this is like this is due to not being able to have C++ > >? ? ?strings when > >? ? ? >? ? ?I tried initially. The day we can have those, then > this code can > >? ? ? >? ? ?become even simpler. > >? ? ? > > >? ? ? >? ? ?Let me know what you think and would prefer, > >? ? ? >? ? ?Jc > >? ? ? > > >? ? ? >? ? ?On Sun, Nov 11, 2018 at 9:01 PM David Holmes > >? ? ? >? ? ? > > >? ? ? >>> > >? ? ?wrote: > >? ? ? > > >? ? ? >? ? ? ? ?Hi Michal, > >? ? ? > > >? ? ? >? ? ? ? ?On 12/11/2018 2:18 PM, Michal Vala wrote: > >? ? ? >? ? ? ? ? > > >? ? ? >? ? ? ? ? > > >? ? ? >? ? ? ? ? > On 11/10/18 12:07 AM, David Holmes wrote: > >? ? ? >? ? ? ? ? >> cc'ing JC Beyler as this was his code. > >? ? ? >? ? ? ? ? >> > >? ? ? >? ? ? ? ? >> On 10/11/2018 4:35 AM, Kim Barrett wrote: > >? ? ? >? ? ? ? ? >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala > >? ? ? > > > >? ? ? >? ? ? ? ? > >>> wrote: > >? ? ? >? ? ? ? ? >>>> > >? ? ? >? ? ? ? ? >>>> Hi, > >? ? ? >? ? ? ? ? >>>> > >? ? ? >? ? ? ? ? >>>> please review following patch fixing build > failure on > >? ? ? >? ? ? ? ?Windows with > >? ? ? >? ? ? ? ? >>>> VS2013 toolchain. > >? ? ? >? ? ? ? ? >> > >? ? ? >? ? ? ? ? >> Not sure we're still supporting VS2013 ... ?? > >? ? ? >? ? ? ? ? > > >? ? ? >? ? ? ? ? > Minimum accepted by configure is 2010. 2013 is > 2nd in > >? ? ? >? ? ? ? ?priorities after > >? ? ? >? ? ? ? ? > 2017. It has `VS_SUPPORTED_2013=false` though, > but why not > >? ? ? >? ? ? ? ?keep it > >? ? ? >? ? ? ? ? > buildable? > >? ? ? > > >? ? ? >? ? ? ? ?That depends on how painful it is to achieve that. > As Kim has > >? ? ? >? ? ? ? ?explained > >? ? ? >? ? ? ? ?the suggested fix may allow building but it isn't > correct. > >? ? ? > > >? ? ? >? ? ? ? ?And we may not be able to keep this ability to > build with > >? ? ?VS2013 > >? ? ? >? ? ? ? ?going > >? ? ? >? ? ? ? ?forward. > >? ? ? > > >? ? ? >? ? ? ? ?Thanks, > >? ? ? >? ? ? ? ?David > >? ? ? > > >? ? ? > > >? ? ? >? ? ? ? ? >> > >? ? ? >? ? ? ? ? >>>> > >? ? ? >? ? ? ? ? >>>> > >? ? ? > > http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ > > > > ? > >? ? ? > > > > ? > >? ? ? >? ? ? ? ? >>> > >? ? ? >? ? ? ? ? >>> The failure is in a hotspot test.? It should > be using > >? ? ? >? ? ? ? ?os::snprintf. > >? ? ? >? ? ? ? ? >> > >? ? ? >? ? ? ? ? >> Tests don't have access to os::snprintf. The > test is just > >? ? ? >? ? ? ? ?C++ code. > >? ? ? >? ? ? ? ? >> > >? ? ? >? ? ? ? ? >> Cheers, > >? ? ? >? ? ? ? ? >> David > >? ? ? >? ? ? ? ? >> > >? ? ? >? ? ? ? ? >> > >? ? ? >? ? ? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? >? ? ?-- > >? ? ? > > >? ? ? >? ? ?Thanks, > >? ? ? >? ? ?Jc > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > -- > >? ? ? > > >? ? ? > Thanks, > >? ? ? > Jc > > > > > > > > -- > > > > Thanks, > > Jc > > > > -- > > Thanks, > Jc From mvala at redhat.com Tue Nov 13 09:32:06 2018 From: mvala at redhat.com (Michal Vala) Date: Tue, 13 Nov 2018 10:32:06 +0100 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> <5038b4fe-862c-5324-fd79-4a58d8fb6793@oracle.com> Message-ID: <7d2740e8-1699-10b8-04bf-fa01bd0160ae@redhat.com> Hi, On 11/13/18 1:08 AM, JC Beyler wrote: > Hi all, > > @Mark: good point, fixed in the new webrev > @David: also good point, just because originally it was written differently > and I moved the code to this format and didn't move the +1 to the "right" > spot > @Michal: do you mind if I take over the bug and push this fix, could you > review it as well? Please, take it. I'm not jdk reviewer nor have anough C++ skills to do a reviews. I don't see any issue there. I've also tried to build it with VS2013 and it works fine. Thanks! > > Here is the new webrev: > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.01/ > Here is the bug: https://bugs.openjdk.java.net/browse/JDK-8213622 > > Thanks, > Jc > > On Mon, Nov 12, 2018 at 2:08 PM David Holmes > wrote: > >> Hi Jc, >> >> This seems okay to me. Only minor query is why you do the +1 (presumably >> for terminating NUL) on the return_len instead of len ? >> >> Thanks, >> David >> >> On 13/11/2018 7:11 AM, JC Beyler wrote: >>> Hi all, >>> >>> I created this change instead: >>> http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ >>> >>> >>> It removes the sprintf calls for strlen and strncpy calls. I checked >>> that the code works if I force an error on GetObjectClass for example: >>> >>> FATAL ERROR in native method: GetObjectClass : Return is NULL >>> at >>> >> nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) >>> at >>> >> nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) >>> at nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalNative(Native >> Method) >>> at >>> >> nsk.share.gc.lock.jniref.JNILocalRefLocker.criticalSection(JNILocalRefLocker.java:44) >>> at >>> >> nsk.share.gc.lock.CriticalSectionTimedLocker.criticalSection(CriticalSectionTimedLocker.java:47) >>> at >>> >> nsk.share.gc.lock.CriticalSectionLocker$1.run(CriticalSectionLocker.java:56) >>> at java.lang.Thread.run(java.base at 12-internal/Thread.java:835) >>> >>> I'll pass it through the submit repo if this looks like a better fix. >>> >>> Let me know what you think, >>> Jc >>> >>> On Sun, Nov 11, 2018 at 10:38 PM JC Beyler >> > wrote: >>> >>> Hi all, >>> >>> If I've caught up with the conversation, it sounds like: >>> - My code breaks VS2013 build but that soon that won't be >> supported >>> - We can't just do a renaming macro due to my use of sprintf with >>> an empty buffer >>> - So we need to either do what was suggested by Kim: >>> "That first snprintf call could be rewritten using several calls to >>> strlen to calculate the needed size in some different manner. The >>> later call could also be rewritten to use several calls to strcpy >>> rather than snprintf." >>> Or something to that effect >>> >>> I can work on a fix that will handle this if we want, I'm just not >>> sure if that's the path that is being recommended here though. It >>> seems that in this particular case, it is not really hard to fix the >>> code for now; the day VS2013 is no longer build-able by other pieces >>> of code we can come back to this solution. To be honest, the reason >>> this is like this is due to not being able to have C++ strings when >>> I tried initially. The day we can have those, then this code can >>> become even simpler. >>> >>> Let me know what you think and would prefer, >>> Jc >>> >>> On Sun, Nov 11, 2018 at 9:01 PM David Holmes >>> > wrote: >>> >>> Hi Michal, >>> >>> On 12/11/2018 2:18 PM, Michal Vala wrote: >>> > >>> > >>> > On 11/10/18 12:07 AM, David Holmes wrote: >>> >> cc'ing JC Beyler as this was his code. >>> >> >>> >> On 10/11/2018 4:35 AM, Kim Barrett wrote: >>> >>>> On Nov 9, 2018, at 11:43 AM, Michal Vala >> > wrote: >>> >>>> >>> >>>> Hi, >>> >>>> >>> >>>> please review following patch fixing build failure on >>> Windows with >>> >>>> VS2013 toolchain. >>> >> >>> >> Not sure we're still supporting VS2013 ... ?? >>> > >>> > Minimum accepted by configure is 2010. 2013 is 2nd in >>> priorities after >>> > 2017. It has `VS_SUPPORTED_2013=false` though, but why not >>> keep it >>> > buildable? >>> >>> That depends on how painful it is to achieve that. As Kim has >>> explained >>> the suggested fix may allow building but it isn't correct. >>> >>> And we may not be able to keep this ability to build with VS2013 >>> going >>> forward. >>> >>> Thanks, >>> David >>> >>> >>> >> >>> >>>> >>> >>>> >>> http://cr.openjdk.java.net/~mvala/jdk/jdk/JDK-8213622/webrev.00/ >>> < >> http://cr.openjdk.java.net/%7Emvala/jdk/jdk/JDK-8213622/webrev.00/> >>> >>> >>> >>> The failure is in a hotspot test. It should be using >>> os::snprintf. >>> >> >>> >> Tests don't have access to os::snprintf. The test is just >>> C++ code. >>> >> >>> >> Cheers, >>> >> David >>> >> >>> >> >>> > >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >> > > -- Michal Vala OpenJDK QE Red Hat Czech From boris.ulasevich at bell-sw.com Tue Nov 13 10:15:25 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Tue, 13 Nov 2018 13:15:25 +0300 Subject: RFR(XS): 8213410: UseCompressedOops @requirement check fails fails on 32-bit system In-Reply-To: <725eb131-529d-83a5-e047-f08390ed0dec@oracle.com> References: <839f53dd-d020-4fcb-5005-ddb281d10491@bell-sw.com> <3b47c93d-8020-158d-e5ac-b68ff1391acd@oracle.com> <44c2249d-8329-d906-d625-129599b48e36@bell-sw.com> <2e3be50f-b1c2-9a2f-e8c1-57df0cff3c62@oracle.com> <08ce91a2-e2cc-5386-0b23-80f7dc6b7562@bell-sw.com> <725eb131-529d-83a5-e047-f08390ed0dec@oracle.com> Message-ID: <179c289e-b116-5c46-9eb9-6b2fcde65a1a@bell-sw.com> > Looks good. Thank you! On 13.11.2018 1:35, David Holmes wrote: > Hi Boris, > > On 12/11/2018 10:18 PM, Boris Ulasevich wrote: >> Hi David, >> >> Ok. Here is the solution exploiting 64-bit & UseCompressedOops == true >> check (works ok and ARM64 and makes tests disabled on ARM32): >> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.03/ > > Looks good. > >> I will try to go forward on code-tools-dev and I will correct these >> tests later if we get short-circuit boolean evaluation in jtreg. > > Thanks! > > David > >> thank you, >> Boris >> >> On 12.11.2018 0:45, David Holmes wrote: >>> Hi Boris, >>> >>> On 9/11/2018 9:03 PM, Boris Ulasevich wrote: >>>> Hi David, >>>> >>>> Thank you. >>>> >>>> Don't you think it is better to learn jtreg to perform short-circuit >>>> boolean evaluation? >>> >>> Sure but that will take a lot longer to happen. You'll need to file a >>> bug in the CODE-TOOLS project and then discuss on code-tools-dev. >>> Once the fix is accepted we then have to wait for the next release of >>> jtreg to be promoted and then OpenJDK updated to start using it. >>> >>> Meanwhile if you want the tests to actually run they will need to be >>> modified to work with existing jtreg behaviour. >>> >>> Thanks, >>> David >>> >>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02 >>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.02_jtreg >>>> >>>> By the way, with the given change in jtreg two failed tests from the >>>> original list works Ok because they already have 64-bit check on >>>> previous lines: >>>> * @requires (sun.arch.data.model == "64") >>>> * @requires vm.opt.final.UseCompressedOops >>>> >>>> regards, >>>> Boris >>>> >>>> On 09.11.2018 10:39, David Holmes wrote: >>>>> Hi Boris, >>>>> >>>>> On 7/11/2018 6:54 PM, David Holmes wrote: >>>>>> Hi Boris, >>>>>> >>>>>> On 7/11/2018 6:06 PM, Boris Ulasevich wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> ?? Yes, at first glance it is weird. We have actually three >>>>>>> states for vm.opt.final.UseCompressedOops: true, false and null. >>>>>>> Null means "not applicable" - when current VM does not support >>>>>>> the option with the given name. Here is another approach for the >>>>>>> issue (I am not sure it is a good one): >>>>>>> ?? http://cr.openjdk.java.net/~bulasevich/8213410/webrev.01/ >>>>>> >>>>>> I like that approach! I'm just not sure everyone else will >>>>>> necessarily agree :) This proposal might need a wider audience >>>>>> than just hotspot-dev. >>>>>> >>>>>> Let me make enquiries with jtreg-use at openjdk.java.net and see what >>>>>> the general intent was here. >>>>> >>>>> That was uninformative as to intent as Jon initially thought both >>>>> variants should fail the same way, but on further reflection it is >>>>> likely due to the difference between trying to evaluate null as a >>>>> boolean - which gives an error - and doing "true".equals(null) , >>>>> which is just false. >>>>> >>>>> I also see now that this isn't a general facility for using >>>>> >>>>> @requires >>>>> >>>>> because we only actually support three directly: >>>>> >>>>> ??protected void vmOptFinalFlags(Map map) { >>>>> ???????? vmOptFinalFlag(map, "ClassUnloading"); >>>>> ???????? vmOptFinalFlag(map, "UseCompressedOops"); >>>>> ???????? vmOptFinalFlag(map, "EnableJVMCI"); >>>>> ?? } >>>>> >>>>> but that also shows this is not specific to UseCompressedOops >>>>> because EnableJVMCI only exists in a VM that was built with JVMCI >>>>> support. >>>>> >>>>> I still think the correct way for this current case to be handled >>>>> in tests is by using: >>>>> >>>>> @requires vm.bits == 64 & UseCompressedOops >>>>> >>>>> though that requires jtreg to do short-circuit boolean evaluation, >>>>> which it appears not to do. :( Though: >>>>> >>>>> @requires vm.bits == 64 & UseCompressedOops == true >>>>> >>>>> should work (albeit perhaps due to implementation accident). >>>>> >>>>> Cheers, >>>>> David >>>>> ----- >>>>> >>>>>> Aside: if the UseCompressOops checks is preceded by a 64-bit check >>>>>> does it still fail? (Just wondering whether jtreg does >>>>>> short-circuit evaluation for these @require statements.) >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> thank you, >>>>>>> Boris >>>>>>> >>>>>>> On 07.11.2018 1:36, David Holmes wrote: >>>>>>>> Hi Boris, >>>>>>>> >>>>>>>> I'm somewhat perplexed as to what the difference is between: >>>>>>>> >>>>>>>> @requires vm.opt.final.UseCompressedOops >>>>>>>> >>>>>>>> and >>>>>>>> >>>>>>>> @requires vm.opt.final.UseCompressedOops == true >>>>>>>> >>>>>>>> if they are indeed different then that seems to be a bug in the >>>>>>>> requirement handling in the jtreg support code. >>>>>>>> >>>>>>>> I'm also wondering whether any such requirement should always be >>>>>>>> proceeded by a check for 64-bits? >>>>>>>> >>>>>>>> Otherwise I would expect >>>>>>>> >>>>>>>> @requires vm.opt.final.UseCompressedOops >>>>>>>> >>>>>>>> to be false in 32-bit, so again a problem with the jtreg support >>>>>>>> code. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> On 7/11/2018 12:43 AM, Boris Ulasevich wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please review this patch to fix jtreg @requires >>>>>>>>> vm.opt.final.UseCompressedOops flag evaluation fail reproduced >>>>>>>>> on ARM32: "invalid boolean value: null for expression >>>>>>>>> vm.opt.final.UseCompressedOops". >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~bulasevich/8213410/webrev.00/ >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213410 >>>>>>>>> >>>>>>>>> The fix was checked on ARM32 (tests disabled) and AARCH64 >>>>>>>>> (works Ok by default, and becomes disabled with >>>>>>>>> -javaoptions:"-XX:-UseCompressedOops" jtreg option). >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Boris Ulasevich From rkennke at redhat.com Tue Nov 13 12:17:13 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Nov 2018 13:17:13 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <5BE301F3.9080706@oracle.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <83C93627-ABF8-43E2-BAFA-70B9E9E7B40E@oracle.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> Message-ID: <18b993f3-8560-ccb3-2540-079f25c86e34@redhat.com> Apparently there is another quirk to this whole issue: It looks like offset=-1 is used in a few places (at least in C1), presumably as placeholder for offsets to be patched (to be honest, I am not totally sure). This would result in needs_explicit_null_check=false with Shenandoah because we cover [-8..page_size), but this results in weird problems later. It is expected that this placeholder results in needs_explicit_null_check=true for offset=-1. A possible fix is to explicitly check for this: diff --git a/src/hotspot/share/asm/assembler.cpp b/src/hotspot/share/asm/assembler.cpp --- a/src/hotspot/share/asm/assembler.cpp +++ b/src/hotspot/share/asm/assembler.cpp @@ -334,6 +334,7 @@ bool MacroAssembler::needs_explicit_null_check(intptr_t offset) { // Check if offset is outside of [-cell_header_size, os::vm_page_size) + if (offset == -1) return true; return offset < -Universe::heap()->cell_header_size() || offset >= os::vm_page_size(); } Thoughts? Any better ideas? Roman > Hi Roman, > > The si_addr is always void*, so the cast to void* is redundant. > Otherwise, looks good. > Don't need to see another webrev for that. > > Thanks, > /Erik > > On 2018-11-07 15:51, Roman Kennke wrote: >> Hi Erik, >> >>> The approach looks great! :p >> Ha! :-) >> >>> Looks like "you" forgot to call the new function in os_solaris_sparc.cpp >>> though, he he. With that fixed, we pass tier1-3 on our platforms. >>> Oh and the cast in the following line is unnecessary: intptr_t start = >>> (intptr_t)-cell_header_size; >> Both fixed here (solaris/sparc stuff blindly though): >> Incremental: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03.diff >> Full: >> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.03/ >> >> Good now? >> >> Thanks for reviewing and helping! >> >> Roman >> >>> Thanks, >>> /Erik >>> >>> On 2018-11-07 09:52, Roman Kennke wrote: >>>> Alright! thanks Erik! So here I'd like to propose your version plus my >>>> little fixes for review: >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/webrev.02/ >>>> >>>> It passes hotspot/jtreg:tier1 locally on x86 and aarch64 and works with >>>> Shenandoah too (and passes the Shenandoah testsuites) >>>> >>>> Good? >>>> >>>> Roman >>>> >>>> >>>>> Hi Roman, >>>>> >>>>> Glad you liked the idea! >>>>> >>>>> On 2018-11-06 22:37, Roman Kennke wrote: >>>>>> Hi Erik, >>>>>> >>>>>> I also like it. I needed to change a little bit to make it work with >>>>>> Shenandoah: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/JDK-8213199/JDK-8213199-eosterlund.patch >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I will give it a spin on aarch64 as soon as I get access to the >>>>>> machine. >>>>>> >>>>>> Do you want to take the bug and finish it, or do you want me to >>>>>> take it >>>>>> from here? >>>>> You can take it from here. I just wanted to show a possible direction. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> Roman >>>>>> >>>>>>> Hi Andrew, >>>>>>> >>>>>>> On 2018-11-06 17:46, Andrew Dinn wrote: >>>>>>>> On 06/11/18 16:04, Erik ?sterlund wrote: >>>>>>>>> New webrev: >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213199/webrev.01/ >>>>>>>> Thanks for the updated version. >>>>>>>> >>>>>>>>> On 2018-11-06 16:13, Andrew Dinn wrote: >>>>>>>>>> In that case, I hear you ask, why does the old code check for >>>>>>>>>> values in >>>>>>>>>> the range [heap_base - cell_header_size, heap_base + >>>>>>>>>> os::vm_page_size)? >>>>>>>>>> Good question. I don't know the answer. I think that test is >>>>>>>>>> bogus >>>>>>>>>> with >>>>>>>>>> no Shenandoah GC present. >>>>>>>>> Interesting. It turns out that both ranges are relevant >>>>>>>>> [-cell_header_size, os::vm_page_size) and [heap_base - >>>>>>>>> cell_header_size, >>>>>>>>> heap_base + os::vm_page_size). Looks like we both missed that >>>>>>>>> from the >>>>>>>>> original check. >>>>>>>>> I updated my webrev to explicitly reflect those two ranges, as two >>>>>>>>> different checks. >>>>>>>> Well, you say that but -- as I said before -- I am not sure why >>>>>>>> *both* >>>>>>>> are relevant to the non-Shenandoah case. And, given Roman's >>>>>>>> correction >>>>>>>> (doh, yes of course! the space for the Brooks pointer field had >>>>>>>> better >>>>>>>> lie in RW-mapped memory :-) I don't see why it is relevant to the >>>>>>>> Shenandoah case either. >>>>>>> The reason it is needed is because there may or may not be null >>>>>>> checks >>>>>>> in both the decode() step to get the base pointer of an access, and >>>>>>> the >>>>>>> access itself, given some base pointer. This results in two >>>>>>> ranges for >>>>>>> those two cases correspondingly. If the decoding uses >>>>>>> decode_not_null, >>>>>>> with an implicit null check in the decoding itself that blindly >>>>>>> adds the >>>>>>> heap base without checking for null, we will crash into the first >>>>>>> page >>>>>>> of the heap which is protected when performing accesses into >>>>>>> that. But >>>>>>> sometimes we have a decode()ed base pointer (with null check), that >>>>>>> has >>>>>>> already been fixed up to NULL, and then perform an access at an >>>>>>> offset >>>>>>> into that with an implicit null check. This results in a trap at the >>>>>>> first page. So in summary, both ranges are needed because the >>>>>>> implicitness of the null check may be both in the access and the >>>>>>> narrowOop decoding. >>>>>>> >>>>>>> But as I pointed out, this (seemingly incorrectly) assumes there >>>>>>> may not >>>>>>> be any offset values in the (heap_base, heap_base + >>>>>>> os::vm_page_size) >>>>>>> range. If you hit such an offset, we will generate incorrect >>>>>>> code, due >>>>>>> to the lack of distinction between addresses and offsets. >>>>>>> >>>>>>>> I'd be very glad to hear from someone who knows about the >>>>>>>> history of >>>>>>>> the >>>>>>>> original function as to why the extra test for addresses above >>>>>>>> heap_base >>>>>>>> is included [hint: John Rose? David Holmes? Coleen? anyone else who >>>>>>>> might have been around when it was added?] >>>>>>>> >>>>>>>> If that check is not actually needed then getting rid of it would >>>>>>>> make >>>>>>>> the whole thing a lot simpler. >>>>>>> It would indeed. But unfortunately we need it. >>>>>>> >>>>>>>>>>> Just expressing these trivial ranges separately would have been >>>>>>>>>>> very >>>>>>>>>>> easy to understand. >>>>>>>>>> Except it's not that easy. Also, with Shenandoah there is a new >>>>>>>>>> wrinkle. >>>>>>>>>> It is possible for an object to be allocated at the first >>>>>>>>>> non-guarded >>>>>>>>>> address in the heap area i.e. heap_base + os::vm_page_size. I >>>>>>>>>> that >>>>>>>>>> case >>>>>>>>>> a dereference of its Brooks pointer will lie in the protected >>>>>>>>>> page. >>>>>>>>> Looks like Roman just said that's not possible. >>>>>>>> Yes, my dumb! >>>>>>>> >>>>>>>>> Fixed. Might need some help testing this on AArch64 though if we >>>>>>>>> decide >>>>>>>>> to go with it. >>>>>>>> I am sure Roman and Aleksey can do that but if needed I will. >>>>>>>> >>>>>>>>> Do you agree though, that the current version that merges the >>>>>>>>> address >>>>>>>>> and offset case relies on offsets being smaller than the heap >>>>>>>>> base for >>>>>>>>> its magic check to work, and that we actually do not have such a >>>>>>>>> guarantee in the VM? Apart from readability preferences, this >>>>>>>>> seems >>>>>>>>> like >>>>>>>>> it is broken in the current form, unless there are seat belts I >>>>>>>>> am not >>>>>>>>> aware of. In particular, if you have an offset value that is in >>>>>>>>> the >>>>>>>>> virtual address range (heap_base, heap_base + os::vm_page_size()), >>>>>>>>> which >>>>>>>>> I argue can happen, then it looks like we will normalize that as >>>>>>>>> if it >>>>>>>>> was an address (even though it is not), and incorrectly say that >>>>>>>>> this >>>>>>>>> offset does not need explicit null checks, despite needing >>>>>>>>> explicit >>>>>>>>> null >>>>>>>>> checks. >>>>>>>> Well, without the answer as to whether the check against >>>>>>>> heap_base is >>>>>>>> needed I am not sure. Assuming it is then then ... >>>>>>>> >>>>>>>> Yes, it is possible, in theory, that an oop offset could turn >>>>>>>> out to >>>>>>>> have value (heap_base + D) where 0 <= D < os_page_size(). In that >>>>>>>> case >>>>>>>> the offset might wrongly be classified as *not* needing a null >>>>>>>> check >>>>>>>> even though it really does. I suspect it is unlikely in practice >>>>>>>> given >>>>>>>> how mmap hands out addresses but you never know. >>>>>>>> >>>>>>>> As for readibility, I think the two functions are probably >>>>>>>> clearer -- >>>>>>>> expecially if they stay adjacent in the source code. >>>>>>> Okay, sounds like we are on the same page. I agree it is unlikely >>>>>>> that >>>>>>> this will crash the VM (as evidenced by the lack of crashes >>>>>>> observed). >>>>>>> But I don't like relying on luck for correctness. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>>> regards, >>>>>>>> >>>>>>>> >>>>>>>> Andrew Dinn >>>>>>>> ----------- >>>>>>>> Senior Principal Software Engineer >>>>>>>> Red Hat UK Ltd >>>>>>>> Registered in England and Wales under Company Registration No. >>>>>>>> 03798903 >>>>>>>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric >>>>>>>> Shander >>>>>>>> > From adinn at redhat.com Tue Nov 13 12:23:13 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 13 Nov 2018 12:23:13 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <18b993f3-8560-ccb3-2540-079f25c86e34@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <074d2e70-2bd9-64d1-2f15-590b769ffd84@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> <18b993f3-8560-ccb3-2540-079f25c86e34@redhat.com> Message-ID: On 13/11/2018 12:17, Roman Kennke wrote: > Apparently there is another quirk to this whole issue: It looks like > offset=-1 is used in a few places (at least in C1), presumably as > placeholder for offsets to be patched (to be honest, I am not totally > sure). This would result in needs_explicit_null_check=false with > Shenandoah because we cover [-8..page_size), but this results in weird > problems later. It is expected that this placeholder results in > needs_explicit_null_check=true for offset=-1. Hmm, can you be more specific about where the -1 value comes from. That's really the crux of the matter. Giving -1 a free pass is not going to work if the offset that is patched in later turns out to be greater than os page size. > A possible fix is to explicitly check for this: > diff --git a/src/hotspot/share/asm/assembler.cpp > b/src/hotspot/share/asm/assembler.cpp > --- a/src/hotspot/share/asm/assembler.cpp > +++ b/src/hotspot/share/asm/assembler.cpp > @@ -334,6 +334,7 @@ > > bool MacroAssembler::needs_explicit_null_check(intptr_t offset) { > // Check if offset is outside of [-cell_header_size, os::vm_page_size) > + if (offset == -1) return true; > return offset < -Universe::heap()->cell_header_size() || > offset >= os::vm_page_size(); > } > > Thoughts? Any better ideas? Well, if -1 is always an innocuous offset value then I think this looks fine work but we may be in the realm of counter-factual conclusions here so ... see above. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From rkennke at redhat.com Tue Nov 13 12:26:33 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Nov 2018 13:26:33 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <5883BE00-D7DD-4EC4-A4AA-55250D708436@oracle.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> <18b993f3-8560-ccb3-2540-079f25c86e34@redhat.com> Message-ID: <3c0e0a13-940f-fd20-70a9-27df60ccb370@redhat.com> Hi Andrew, > On 13/11/2018 12:17, Roman Kennke wrote: >> Apparently there is another quirk to this whole issue: It looks like >> offset=-1 is used in a few places (at least in C1), presumably as >> placeholder for offsets to be patched (to be honest, I am not totally >> sure). This would result in needs_explicit_null_check=false with >> Shenandoah because we cover [-8..page_size), but this results in weird >> problems later. It is expected that this placeholder results in >> needs_explicit_null_check=true for offset=-1. > > Hmm, can you be more specific about where the -1 value comes from. > That's really the crux of the matter. Giving -1 a free pass is not going > to work if the offset that is patched in later turns out to be greater > than os page size. I think it's used for to-be-patched offsets in a few places in C1. The point is to *not* (accidentally) give it a free pass, but instead force explicit null-checking on it. Roman >> A possible fix is to explicitly check for this: >> diff --git a/src/hotspot/share/asm/assembler.cpp >> b/src/hotspot/share/asm/assembler.cpp >> --- a/src/hotspot/share/asm/assembler.cpp >> +++ b/src/hotspot/share/asm/assembler.cpp >> @@ -334,6 +334,7 @@ >> >> bool MacroAssembler::needs_explicit_null_check(intptr_t offset) { >> // Check if offset is outside of [-cell_header_size, os::vm_page_size) >> + if (offset == -1) return true; >> return offset < -Universe::heap()->cell_header_size() || >> offset >= os::vm_page_size(); >> } >> >> Thoughts? Any better ideas? > Well, if -1 is always an innocuous offset value then I think this looks > fine work but we may be in the realm of counter-factual conclusions here > so ... see above. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From thomas.schatzl at oracle.com Tue Nov 13 12:57:20 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 13 Nov 2018 13:57:20 +0100 Subject: VM deadlock between VM shutdown and G1 In-Reply-To: References: Message-ID: <9a6f01efbf0765c48aaa1422de727382bbf6c02f.camel@oracle.com> Hi Krystal, On Thu, 2018-09-13 at 01:57 -0700, Krystal Mok wrote: > Hi HotSpot devs, > > I'd like to report a VM deadlock situation we've experienced in > production a few weeks ago on JDK8. I checked the latest jdk/jdk code > and the same race condition is still there. > > I can file a JBS ticket if this looks like a valid problem. > > tl;dr: there's a race condition between the VM shutdown and G1, in > that the VM shutdown sequence may have just terminated all the > concurrent GC threads, and at the same time G1 has just started an > incremental collection cycle and is waiting for the concurrent marker > to finish. Since there's no concurrent GC thread alive to update the > status anymore, G1 waits indefinitely while holding a safepoint, > causing the VM to deadlock. I looked a bit through the sources and I think the situation you describe can't happen in jdk/jdk sources. The analysis assumes that the concurrent mark threads are actually stopped and shut down properly at the point of the CollectedHeap::stop() call. However this is not the case. CollectedHeap::stop() does not stop the marking threads at all (which is a different issue depending on your POV). I.e. the root region scan will be finish eventually.. Also the concurrent mark threads should show up in your stack traces - unfortunately the stack traces you posted did not include them. After some digging I think that on JDK8u I think what you are experiencing is JDK-8148992 and/or JDK-8151336. Both were fixed in 9 but simply never backported to 8u. I am likely to close the issue I opened (JDK-8210706) as duplicate of JDK-8148992 unless new evidence shows up as I also did not manage to reproduce the issue. We can try to get JDK-8148992 and JDK-8151336 backported instead if it is a significant issue to you. Thanks, Thomas From amith.pawar at gmail.com Tue Nov 13 13:34:39 2018 From: amith.pawar at gmail.com (amith pawar) Date: Tue, 13 Nov 2018 19:04:39 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: <3548029e-475f-586f-1df8-59813e4fc102@oracle.com> References: <5dd9d93c-9901-8756-d6be-c8c046726204@oracle.com> <3548029e-475f-586f-1df8-59813e4fc102@oracle.com> Message-ID: Hi David, I am from AMD and OCA processing is done. OK now ? Thanks Amit On Sat, Nov 10, 2018 at 6:07 AM David Holmes wrote: > Hi Amit, > > On 10/11/2018 10:34 AM, amith pawar wrote: > > Hi David, > > > > I have initiated for OCA processing and working as an individual > > contributor. > > Great! Welcome to OpenJDK. Please advise once your OCA is processed and > then we can properly look at your contribution. > > Thanks, > David > > > Thanks, > > Amit Pawar > > > > On Sat, Nov 10, 2018, 4:03 AM David Holmes > wrote: > > > > Hi Amith, > > > > I do not see you listed as a Contributor on the OCA list [1], nor > can I > > tell from your email address whether you work for a company that is a > > signatory. Please confirm your status. > > > > Thanks, > > David > > > > > > [1] https://www.oracle.com/technetwork/community/oca-486395.html > > > > On 9/11/2018 10:23 PM, amith pawar wrote: > > > Hi all, > > > > > > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old > > gen, S1 > > > and S2 region (if any other ) memory areas > > > > > > on requested Numa nodes and it should not configure itself to > > access other > > > Numa nodes. This issue is observed only when Java > > > > > > is allowed to run on fewer NUMA nodes than available on the > > system with > > > numactl membind and interleave options. Running > > > > > > on all the nodes does not have any effect. This will cause some > > > applications (objects residing in old gen and survivor region) to > > > > > > run slower on system with large Numa nodes. > > > > > > > > > > > > I have described below in more detail for both numactl MEMBIND and > > > INTERLEAVE options. Addresses from both GC log and > > > > > > process numa maps clearly shows that the JAVA process is > > configured to > > > access other memory nodes even though it is not allowed. > > > > > > > > > > > > Two scenarios are: > > > > > > 1. Numactl membind case : numactl -m 0-1, -N 0-1 > > > > arguments> > > > > > > > > > Numa map shows these regions are INTERLEAVED ON ALL NODES instead > of > > > specified Numa memory nodes 0 and 1. > > > > > > > > > > > > ----------------------------------- GC > > > > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > [602.180s][debug][gc,heap ] GC(20) Heap before GC > > invocations=21 > > > (full 4): PSYoungGen total 120604672K, used 11907587K > > > [0x00002afc4b200000, 0x00002b198b200000, 0x00002b198b200000) > > > > > > [602.180s][debug][gc,heap ] GC(20) eden space > > 118525952K, 8% used > > > [0x00002afc4b200000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > > > [602.180s][debug][gc,heap ] GC(20) lgrp 0 space > > 59262976K, 8% > > > used [0x00002afc4b200000,0x00002afd89bef450,0x00002b0a6c400000) > > > > > > [602.180s][debug][gc,heap ] GC(20) lgrp 1 space > > 59262976K, 8% > > > used [0x00002b0a6c400000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > > > [602.180s][debug][gc,heap ] GC(20) from space 2078720K, > > 65% used > > > [0x00002b190c400000,0x00002b195ef5a0d0,0x00002b198b200000) > > > > > > [602.180s][debug][gc,heap ] GC(20) to space 2078720K, > > 0% used > > > [0x00002b188d600000,0x00002b188d600000,0x00002b190c400000) > > > > > > [602.180s][debug][gc,heap ] GC(20) ParOldGen total > > 2097152K, > > > used 226685K [0x00002afbcb200000, 0x00002afc4b200000, > > 0x00002afc4b200000) > > > > > > [602.180s][debug][gc,heap ] GC(20) object space > > 2097152K, 10% used > > > [0x00002afbcb200000,0x00002afbd8f5f460,0x00002afc4b200000) > > > > > > [602.180s][debug][gc,heap ] GC(20) Metaspace used > > 28462K, > > > capacity 29008K, committed 29184K, reserved 30720K > > > > > > ----------------------------------- GC HEAP > > > > > > END-------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > --------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Command : cat /proc/4947/numa_maps > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > =======> Following addresses are interleaved on all nodes. > > > > > > 2afbb4f4c000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 > > N4=2 N5=2 > > > N6=2 N7=2 kernelpagesize_kB=4 > > > > > > 2afbb4f6c000 interleave:0-7 > > > > > > 2afbb7e88000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 > > N4=6 N5=6 > > > N6=6 N7=6 kernelpagesize_kB=4 > > > > > > 2afbbc000000 interleave:0-7 anon=8704 dirty=8704 N0=1600 N1=1088 > > N2=1088 > > > N3=576 N4=1088 N5=1088 N6=1088 N7=1088 kernelpagesize_kB=4 > > > > > > 2afbc3be6000 interleave:0-7 anon=6682 dirty=6682 N0=1027 N1=1027 > > N2=515 > > > N3=515 N4=515 N5=1027 N6=1028 N7=1028 kernelpagesize_kB=4 > > > > > > 2afbcb000000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 N2=6 N3=6 > > N4=6 N5=6 > > > N6=6 N7=6 kernelpagesize_kB=4 > > > > > > 2afbcb200000 interleave:0-7 anon=524288 dirty=524288 N0=65536 > > N1=65536 > > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > > kernelpagesize_kB=4 > > > > > > ==> OLD GEN Address > > > > > > 2afc4b200000 prefer:0 anon=1536 dirty=1536 N0=1536 > > kernelpagesize_kB=4 > > > > > > 2b0a6c400000 prefer:1 anon=512 dirty=512 N1=512 > kernelpagesize_kB=4 > > > > > > 2b188d600000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 > > N1=130048 > > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > > kernelpagesize_kB=4 > > > > > > ==> Survivor Region > > > > > > 2b198b600000 interleave:0-7 anon=60929 dirty=60929 N0=7233 > > N1=7744 N2=7232 > > > N3=7744 N4=7744 N5=7744 N6=7744 N7=7744 kernelpagesize_kB=4 > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > ---------------------------------- PROCESS status > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Command : cat /proc/4947/status > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > > > > > Cpus_allowed_list: 0-15,64-79 > > > > > > Mems_allowed: > > > > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > > > Mems_allowed_list: 0-7 > > > > > > --------------------------------- PROCESS status > > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > > 2. NUMACTL Interleave case: numactl -i 0-1, -N 0-1 > > > > its arguments> > > > > > > > > > NUMAmaps below shows interleaved on all nodes instead of > > specified Numa > > > memory nodes 0 and 1. > > > > > > ----------------------------------- GC > > > > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > [2216.439s][debug][gc,heap ] GC(159) Heap before GC > > invocations=160 > > > (full 9): PSYoungGen total 120604672K, used 30143454K > > > [0x00002b9d47c00000, 0x00002bba87c00000, 0x00002bba87c00000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) eden space > > 118525952K, 24% > > > used [0x00002b9d47c00000,0x00002ba458400000,0x00002bb98a000000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 0 space > > 14815232K, 98% > > > used [0x00002b9d47c00000,0x00002ba0beb87c90,0x00002ba0d0000000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 1 space > > 14815232K, > > > 100% used > [0x00002ba0d0000000,0x00002ba458400000,0x00002ba458400000) > > > > > > ==> Memory allocated on following nodes are unused. > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 2 space > > 14815232K, 0% > > > used [0x00002ba458400000,0x00002ba458400000,0x00002ba7e0800000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 3 space > > 14815232K, 0% > > > used [0x00002ba7e0800000,0x00002ba7e0800000,0x00002bab68c00000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 4 space > > 14815232K, 0% > > > used [0x00002bab68c00000,0x00002bab68c00000,0x00002baef1000000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 5 space > > 14815232K, 0% > > > used [0x00002baef1000000,0x00002baef1000000,0x00002bb279400000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 6 space > > 14815232K, 0% > > > used [0x00002bb279400000,0x00002bb279400000,0x00002bb601800000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 7 space > > 14819328K, 0% > > > used [0x00002bb601800000,0x00002bb601800000,0x00002bb98a000000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) from space > > 2078720K, 38% used > > > [0x00002bba08e00000,0x00002bba3976fb70,0x00002bba87c00000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) to space > > 2078720K, 0% used > > > [0x00002bb98a000000,0x00002bb98a000000,0x00002bba08e00000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) ParOldGen total > > > 2097152K, used 685229K [0x00002b9cc7c00000, 0x00002b9d47c00000, > > > 0x00002b9d47c00000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) object space > > 2097152K, 32% > > > used [0x00002b9cc7c00000,0x00002b9cf192b6e8,0x00002b9d47c00000) > > > > > > [2216.439s][debug][gc,heap ] GC(159) Metaspace used > > 28753K, > > > capacity 29257K, committed 29440K, reserved 30720K > > > > > > ----------------------------------- GC HEAP > > > > > > END------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Command : cat /proc/pid/numa_maps > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > ==> Following addresses are interleaved on all the nodes. > > > > > > 2b9cb1992000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 N2=2 N3=2 > > N4=2 N5=2 > > > N6=2 N7=2 kernelpagesize_kB=4 > > > > > > 2b9cb19b2000 interleave:0-7 > > > > > > 2b9cb3e65000 interleave:0-7 anon=50 dirty=50 N0=6 N1=6 N2=6 N3=6 > > N4=6 N5=7 > > > N6=7 N7=6 kernelpagesize_kB=4 > > > > > > 2b9cb8a69000 interleave:0-7 anon=8599 dirty=8599 N0=626 N1=1139 > > N2=1139 > > > N3=1139 N4=1139 N5=1139 N6=1139 N7=1139 kernelpagesize_kB=4 > > > > > > 2b9cc064f000 interleave:0-7 anon=6577 dirty=6577 N0=566 N1=566 > N2=566 > > > N3=1078 N4=1078 N5=1078 N6=1078 N7=567 kernelpagesize_kB=4 > > > > > > 2b9cc7a69000 interleave:0-7 anon=50 dirty=50 N0=6 N1=7 N2=7 N3=6 > > N4=6 N5=6 > > > N6=6 N7=6 kernelpagesize_kB=4 > > > > > > 2b9cc7c00000 interleave:0-7 anon=524288 dirty=524288 N0=65536 > > N1=65536 > > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > > kernelpagesize_kB=4 > > > > > > 2b9d47c00000 prefer:0 anon=2560 dirty=2560 N0=2560 > > kernelpagesize_kB=4 > > > ==> Logical group 1 > > > > > > 2ba0d0000000 > > > prefer:1 > > > ==> Logical group 2 > > > > > > 2ba458400000 prefer:2 > > > > > > ==> This one and below all are unnecessary and leaving memory > unused. > > > > > > 2ba7e0800000 prefer:3 > > > > > > 2bab68c00000 prefer:4 > > > > > > 2baef1000000 prefer:5 > > > > > > 2bb279400000 prefer:6 > > > > > > 2bb601800000 prefer:7 > > > > > > 2bb98a000000 interleave:0-7 anon=1040384 dirty=1040384 N0=130048 > > N1=130048 > > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > > kernelpagesize_kB=4 > > > > > > 2bba88000000 interleave:0-7 anon=60929 dirty=60929 N0=7745 > > N1=7744 N2=7744 > > > N3=7744 N4=7744 N5=7232 N6=7744 N7=7232 kernelpagesize_kB=4 > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > --------------------------------- PROCESS status > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Command : cat /proc/pid/status > > > > > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > > > > > Cpus_allowed_list: 0-15,64-79 > > > > > > Mems_allowed: > > > > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > > > Mems_allowed_list: 0-7 > > > > > > --------------------------------- PROCESS status > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > > > > > > Please note ?Mems_allowed? and ?Mems_allowed_list? list also shows > > > incorrect range. This could be LIBNUMA issue in case of membind. > > > > > > When application is externally bound to fewer nodes then calls to > > libNUMA > > > API?s SHOULD FAIL to interleave on all nodes but right now it > > > > > > does not with currently used/called API?s in JDK. > > > > > > > > > > > > How patch works ? > > > > > > The patch gets bitmask structure by calling following API's (Man > page > > > definition for these functions are also given below). > > > > > > 1. For Membind : Use numa_get_membind to get membind bitmask > > (already used > > > in the code) > > > > > > > > > numa_get_membind() returns the mask of nodes from which > > memory can > > > currently be allocated. If the returned mask is equal to > > numa_all_nodes, > > > then memory allocation is allowed from all nodes. > > > > > > 2. For Interleave: use numa_get_interleave_mask to get > > interleave mask > > > (currently not used/called in JDK) > > > numa_get_interleave_mask() returns the current interleave > > mask if the > > > task's memory allocation policy is page interleaved. Otherwise, > this > > > function returns an empty mask. > > > > > > > > > > > > Check node counts from both the Bitmasks to identify current > > running mode. > > > When Interleave mask is EMPTY then it is running in membind else > > it is in > > > interleave mode. > > > > > > Call ?numa_interleave_memory? (called indirectly through > > ?numa_make_global? > > > funcition) function with right bitmask identified above helps to > > fix this > > > issue. > > > > > > > > > > > > > > > > > > Improvement: > > > > > > This patch is tested on EPYC with SPECJBB benchmark and score > > improvements > > > are given below. > > > > > > 1. For NUMACTL membind > > > > > > Max-jOPS improved by 5-12 % and Critical-jOPS by 2-6 % > > > > > > 2. For NUMACTL interleave (This patch fixes memory usage when > > invoked with > > > numactl -i) > > > > > > Max-jOPS by 5-15% and Critical-jOPS by 11-100%. > > > > > > 3. With this fix, flag UseNUMAInterleaving turning on/off has no > > effect > > > when externally interleaved through NUMACTL. > > > > > > 4. Flag UseNUMA score improved by ~14 % when enabled for single > > NUMA node. > > > Currently it gets disabled when externally bound to single node. > > > > > > > > > > > > > > > > > > ---------------------- PATCH -------------------------- > > > diff --git a/src/hotspot/os/linux/os_linux.cpp > > > b/src/hotspot/os/linux/os_linux.cpp > > > --- a/src/hotspot/os/linux/os_linux.cpp > > > +++ b/src/hotspot/os/linux/os_linux.cpp > > > @@ -2720,6 +2720,8 @@ > > > } > > > > > > void os::numa_make_global(char *addr, size_t bytes) { > > > + if (!UseNUMAInterleaving) > > > + return ; > > > Linux::numa_interleave_memory(addr, bytes); > > > } > > > > > > @@ -2785,6 +2787,15 @@ > > > ids[i++] = node; > > > } > > > } > > > + > > > + if (Linux::_numa_interleave_ptr != NULL ) { > > > + i = 0; > > > + for (int node = 0; node <= highest_node_number; node++) { > > > + if > (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, > > > node)) { > > > + ids[i++] = node; > > > + } > > > + } > > > + } > > > return i; > > > } > > > > > > @@ -2884,11 +2895,18 @@ > > > libnuma_dlsym(handle, > > > "numa_distance"))); > > > > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > > > > libnuma_v2_dlsym(handle, > > > "numa_get_membind"))); > > > + > > > > > > set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, > > > + > libnuma_v2_dlsym(handle, > > > "numa_get_interleave_mask"))); > > > > > > if (numa_available() != -1) { > > > + struct bitmask *bmp; > > > set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > > > "numa_all_nodes")); > > > set_numa_all_nodes_ptr((struct bitmask > > **)libnuma_dlsym(handle, > > > "numa_all_nodes_ptr")); > > > set_numa_nodes_ptr((struct bitmask > **)libnuma_dlsym(handle, > > > "numa_nodes_ptr")); > > > + bmp = _numa_get_interleave_mask(); > > > + set_numa_interleave_ptr(&bmp); > > > + bmp = _numa_get_membind(); > > > + set_numa_membind_ptr(&bmp); > > > // Create an index -> node mapping, since nodes are not > > always > > > consecutive > > > _nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) > > > GrowableArray(0, true); > > > rebuild_nindex_to_node_map(); > > > @@ -3015,9 +3033,12 @@ > > > os::Linux::numa_bitmask_isbitset_func_t > > os::Linux::_numa_bitmask_isbitset; > > > os::Linux::numa_distance_func_t os::Linux::_numa_distance; > > > os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > > > +os::Linux::numa_get_interleave_mask_func_t > > > os::Linux::_numa_get_interleave_mask; > > > unsigned long* os::Linux::_numa_all_nodes; > > > struct bitmask* os::Linux::_numa_all_nodes_ptr; > > > struct bitmask* os::Linux::_numa_nodes_ptr; > > > +struct bitmask* os::Linux::_numa_interleave_ptr; > > > +struct bitmask* os::Linux::_numa_membind_ptr; > > > > > > bool os::pd_uncommit_memory(char* addr, size_t size) { > > > uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, > > > @@ -4997,13 +5018,38 @@ > > > if (!Linux::libnuma_init()) { > > > UseNUMA = false; > > > } else { > > > - if ((Linux::numa_max_node() < 1) || > > Linux::isbound_to_single_node()) > > > { > > > - // If there's only one node (they start from 0) or if > > the process > > > - // is bound explicitly to a single node using membind, > > disable > > > NUMA. > > > - UseNUMA = false; > > > + > > > + // Identify whether running in membind or interleave mode. > > > + struct bitmask *bmp; > > > + bool _is_membind = false; > > > + bool _is_interleaved = false; > > > + > > > + // Check for membind mode. > > > + bmp = Linux::_numa_membind_ptr; > > > + for (int node = 0; node <= Linux::numa_max_node() ; node++) { > > > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > > + _is_membind = true; > > > } > > > } > > > > > > + // Check for interleave mode. > > > + bmp = Linux::_numa_interleave_ptr; > > > + for (int node = 0; node <= Linux::numa_max_node() ; node++) { > > > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > > + _is_interleaved = true; > > > + // Set membind to false as interleave mode allows all > > nodes to be > > > used. > > > + _is_membind = false; > > > + } > > > + } > > > + > > > + if (_is_membind) > > > + Linux::set_numa_interleave_ptr (NULL); > > > + > > > + if (_is_interleaved) > > > + Linux::set_numa_membind_ptr (NULL); > > > + > > > + } > > > + > > > if (UseParallelGC && UseNUMA && UseLargePages && > > > !can_commit_large_page_memory()) { > > > // With SHM and HugeTLBFS large pages we cannot uncommit > > a page, so > > > there's no way > > > // we can make the adaptive lgrp chunk resizing work. If > > the user > > > specified both > > > diff --git a/src/hotspot/os/linux/os_linux.hpp > > > b/src/hotspot/os/linux/os_linux.hpp > > > --- a/src/hotspot/os/linux/os_linux.hpp > > > +++ b/src/hotspot/os/linux/os_linux.hpp > > > @@ -222,6 +222,7 @@ > > > typedef void (*numa_interleave_memory_func_t)(void *start, > > size_t size, > > > unsigned long *nodemask); > > > typedef void (*numa_interleave_memory_v2_func_t)(void *start, > > size_t > > > size, struct bitmask* mask); > > > typedef struct bitmask* (*numa_get_membind_func_t)(void); > > > + typedef struct bitmask* > (*numa_get_interleave_mask_func_t)(void); > > > > > > typedef void (*numa_set_bind_policy_func_t)(int policy); > > > typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask > *bmp, > > > unsigned int n); > > > @@ -239,9 +240,12 @@ > > > static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > > > static numa_distance_func_t _numa_distance; > > > static numa_get_membind_func_t _numa_get_membind; > > > + static numa_get_interleave_mask_func_t > _numa_get_interleave_mask; > > > static unsigned long* _numa_all_nodes; > > > static struct bitmask* _numa_all_nodes_ptr; > > > static struct bitmask* _numa_nodes_ptr; > > > + static struct bitmask* _numa_interleave_ptr; > > > + static struct bitmask* _numa_membind_ptr; > > > > > > static void set_sched_getcpu(sched_getcpu_func_t func) { > > _sched_getcpu = > > > func; } > > > static void set_numa_node_to_cpus(numa_node_to_cpus_func_t > > func) { > > > _numa_node_to_cpus = func; } > > > @@ -255,9 +259,12 @@ > > > static void > > set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) > > > { _numa_bitmask_isbitset = func; } > > > static void set_numa_distance(numa_distance_func_t func) { > > > _numa_distance = func; } > > > static void set_numa_get_membind(numa_get_membind_func_t > func) { > > > _numa_get_membind = func; } > > > + static void > > set_numa_get_interleave_mask(numa_get_interleave_mask_func_t > > > func) { _numa_get_interleave_mask = func; } > > > static void set_numa_all_nodes(unsigned long* ptr) { > > _numa_all_nodes = > > > ptr; } > > > static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > > > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > > > static void set_numa_nodes_ptr(struct bitmask **ptr) { > > _numa_nodes_ptr = > > > (ptr == NULL ? NULL : *ptr); } > > > + static void set_numa_interleave_ptr(struct bitmask **ptr) { > > > _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } > > > + static void set_numa_membind_ptr(struct bitmask **ptr) { > > > _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } > > > static int sched_getcpu_syscall(void); > > > public: > > > static int sched_getcpu() { return _sched_getcpu != NULL ? > > > _sched_getcpu() : -1; } > > > @@ -275,7 +282,10 @@ > > > static void numa_interleave_memory(void *start, size_t size) { > > > // Use v2 api if available > > > if (_numa_interleave_memory_v2 != NULL && > > _numa_all_nodes_ptr != NULL) > > > { > > > - _numa_interleave_memory_v2(start, size, > _numa_all_nodes_ptr); > > > + if (_numa_interleave_ptr != NULL) > > > + _numa_interleave_memory_v2(start, size, > > _numa_interleave_ptr); > > > + else > > > + _numa_interleave_memory_v2(start, size, > _numa_membind_ptr); > > > } else if (_numa_interleave_memory != NULL && > > _numa_all_nodes != NULL) > > > { > > > _numa_interleave_memory(start, size, _numa_all_nodes); > > > } > > > ----------------------- PATCH ----------------------------------- > > > > > > > > > Thanks > > > Amit Pawar > > > > > > -- With best regards, amit pawar From vicente.romero at oracle.com Tue Nov 13 14:00:25 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 13 Nov 2018 09:00:25 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> Message-ID: <12728fe8-b3ba-1794-4a98-03e963229b7f@oracle.com> any other comment after the last iteration? we are in a bit of a hurry to push this before the JDK 12 train departs :( Thanks, Vicente On 11/8/18 11:39 AM, Vicente Romero wrote: > Hi David, Igor > > On 11/7/18 10:03 PM, David Holmes wrote: >> Hi Vicente, >> >> All of the javadoc comment reformatting makes it nearly impossible to >> see the actual substantive changes :( >> >> ASM 7 also supports the Nestmate attributes and I was trying to see >> how/where that appeared but its somewhat obscure. Oh well. >> >> Is it that case that the code the uses the ASM library, like the JFR >> code and jlink code, and the tests, doesn't actually _have to_ change >> to specifying Opcodes.ASM7 unless they plan on using ASM7 features? > > I changed only the tests for which the new ASM was complaining about a > particular API available only for ASM7 > >> If so then you could split out the actual update of ASM from the >> updates to the users of ASM (some of which may be quite fine with ASM5). > > I have made two webrevs to make the review easier [1], contain only > the changes to the internal asm and [2] contains the changes to the > clients plus make files, legal, etc. I have also made the changes to > ClassWriterExt and affected test proposed by Igor in another mail, > >> >> Thanks, >> David > > Thanks, > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.only.00/ > [2] > http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.additional.changes.00/ >> >> On 8/11/2018 1:56 AM, Vicente Romero wrote: >>> Hi, >>> >>> Version 7.0 of ASM has been released. This version supports condy, >>> yay!, and we want to include it in JDK 12. Please review [1] which >>> includes: >>> - the new version perse substituting the preview ASM internal >>> version in the JDK >>> - changes to additional files in particular some tests, mostly >>> hotspot tests. >>> >>> Thanks, >>> Vicente >>> >>> [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.00/ > From Alan.Bateman at oracle.com Tue Nov 13 14:18:35 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 13 Nov 2018 14:18:35 +0000 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <12728fe8-b3ba-1794-4a98-03e963229b7f@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> <12728fe8-b3ba-1794-4a98-03e963229b7f@oracle.com> Message-ID: <03ad8985-b24a-f7e1-2d32-0f0a86ff1145@oracle.com> On 13/11/2018 14:00, Vicente Romero wrote: > any other comment after the last iteration? we are in a bit of a hurry > to push this before the JDK 12 train departs :( The original patch updated all the use sites (and tests) to specify ASM7 for the API version. I just checked the webrev again now and it seems to be just the ASM refresh now. Assuming all the tests are passing and you've sorted out the mvlm test issues with Igor then I suggest go ahead with this push and we can update the sites, as needed, at a later time. -Alan From brian.goetz at oracle.com Tue Nov 13 14:40:45 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 13 Nov 2018 15:40:45 +0100 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> Message-ID: But don?t they? We intend to start generating condy in classfiles from javac soon; at that point, anyone not using ASM7 will fail when reading classfiles generated by javac. > On Nov 8, 2018, at 4:03 AM, David Holmes wrote: > > Is it that case that the code the uses the ASM library, like the JFR code and jlink code, and the tests, doesn't actually _have to_ change to specifying Opcodes.ASM7 unless they plan on using ASM7 features? If so then you could split out the actual update of ASM from the updates to the users of ASM (some of which may be quite fine with ASM5). From adinn at redhat.com Tue Nov 13 15:06:58 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 13 Nov 2018 15:06:58 +0000 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <3c0e0a13-940f-fd20-70a9-27df60ccb370@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <03065a16-180a-1418-db56-a74c6b64bbf0@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> <18b993f3-8560-ccb3-2540-079f25c86e34@redhat.com> <3c0e0a13-940f-fd20-70a9-27df60ccb370@redhat.com> Message-ID: <3ff3e4f5-370e-0a17-c7c1-999dc8a391a0@redhat.com> On 13/11/2018 12:26, Roman Kennke wrote: > The point is to *not* (accidentally) give it a free pass, but instead > force explicit null-checking on it. Oops! Yes, which of course accords with the logic of the patch that I conveniently inverted. In which case the correction you posted good. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From rkennke at redhat.com Tue Nov 13 15:17:39 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 13 Nov 2018 16:17:39 +0100 Subject: RFR: JDK-8213199: GC abstraction for Assembler::needs_explicit_null_check() In-Reply-To: <3ff3e4f5-370e-0a17-c7c1-999dc8a391a0@redhat.com> References: <335c4dc1-7c78-2e6b-0eaf-672fa1f71559@redhat.com> <5BDB051B.6060703@oracle.com> <15d5425f-aa96-2bbc-8b92-642814bf210c@redhat.com> <95167fce-38b8-d945-35e3-ac54a851091b@oracle.com> <89a965c5-3d6c-8877-37ec-0eab2353a1ad@redhat.com> <5BE040E6.8080103@oracle.com> <09e6a3f6-ea07-61cd-2406-5b9a4c4debed@redhat.com> <5BE1A64D.9050005@oracle.com> <31d923a9-7721-8cd2-bf15-7064a7962f8c@redhat.com> <5BE1BBA7.40309@oracle.com> <452fa505-09cd-0763-dc35-fb379a12b115@redhat.com> <78c53d8c-be21-e863-60b1-e0f32f08554a@oracle.com> <5BE2EBB9.3040804@oracle.com> <232d434f-c8b8-3405-f61d-35b97c79f202@redhat.com> <5BE301F3.9080706@oracle.com> <18b993f3-8560-ccb3-2540-079f25c86e34@redhat.com> <3c0e0a13-940f-fd20-70a9-27df60ccb370@redhat.com> <3ff3e4f5-370e-0a17-c7c1-999dc8a391a0@redhat.com> Message-ID: <7e53b4c3-734c-a6df-ff9f-05ccb8f089d5@redhat.com> > On 13/11/2018 12:26, Roman Kennke wrote: >> The point is to *not* (accidentally) give it a free pass, but instead >> force explicit null-checking on it. > > Oops! Yes, which of course accords with the logic of the patch that I > conveniently inverted. > > In which case the correction you posted good. Moved to proper RFR: http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2018-November/031344.html Roman From aph at redhat.com Tue Nov 13 16:40:00 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 13 Nov 2018 16:40:00 +0000 Subject: RFR: AArch64: 8209415: Fix JVMTI test failure HS202 Message-ID: <5f2fd3f3-de58-a352-14f7-0d33f92371f5@redhat.com> In a moment of madness one of us (no names, no pack drill!) managed to translate from x86 __ cmpb(Address(rbcp, 0), Bytecodes::_invokestatic); __ jcc(Assembler::notEqual, L_done); to __ ldrb(rscratch1, Address(rbcp, 0)); __ cmpw(r1, Bytecodes::_invokestatic); __ br(Assembler::EQ, L_done); Two bugs in thee lines... fixed thusly. http://cr.openjdk.java.net/~aph/8209415/ OK for trunk and jdk8u ? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From jcbeyler at google.com Tue Nov 13 16:54:51 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 13 Nov 2018 08:54:51 -0800 Subject: RFR: AArch64: 8209415: Fix JVMTI test failure HS202 In-Reply-To: <5f2fd3f3-de58-a352-14f7-0d33f92371f5@redhat.com> References: <5f2fd3f3-de58-a352-14f7-0d33f92371f5@redhat.com> Message-ID: Hi Andrew, Not a reviewer but had fun finding the two bugs before looking at the webrev :). Looks good to me! Jc On Tue, Nov 13, 2018 at 8:41 AM Andrew Haley wrote: > In a moment of madness one of us (no names, no pack drill!) managed > to translate from x86 > > __ cmpb(Address(rbcp, 0), Bytecodes::_invokestatic); > __ jcc(Assembler::notEqual, L_done); > > to > > __ ldrb(rscratch1, Address(rbcp, 0)); > __ cmpw(r1, Bytecodes::_invokestatic); > __ br(Assembler::EQ, L_done); > > Two bugs in thee lines... fixed thusly. > > http://cr.openjdk.java.net/~aph/8209415/ > > OK for trunk and jdk8u ? > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -- Thanks, Jc From adinn at redhat.com Tue Nov 13 17:21:30 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 13 Nov 2018 17:21:30 +0000 Subject: RFR: AArch64: 8209415: Fix JVMTI test failure HS202 In-Reply-To: <5f2fd3f3-de58-a352-14f7-0d33f92371f5@redhat.com> References: <5f2fd3f3-de58-a352-14f7-0d33f92371f5@redhat.com> Message-ID: On 13/11/2018 16:40, Andrew Haley wrote: > In a moment of madness one of us (no names, no pack drill!) managed > to translate from x86 > > __ cmpb(Address(rbcp, 0), Bytecodes::_invokestatic); > __ jcc(Assembler::notEqual, L_done); > > to > > __ ldrb(rscratch1, Address(rbcp, 0)); > __ cmpw(r1, Bytecodes::_invokestatic); > __ br(Assembler::EQ, L_done); > > Two bugs in thee lines... fixed thusly. > > http://cr.openjdk.java.net/~aph/8209415/ > > OK for trunk and jdk8u ? Oh my goodness, that's a doozy. Amazing that it lay there for so long (the author seems to have checked it in in 2015 :-P ) Patch is good! regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From kim.barrett at oracle.com Tue Nov 13 19:00:18 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 13 Nov 2018 14:00:18 -0500 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> Message-ID: > On Nov 12, 2018, at 4:11 PM, JC Beyler wrote: > > Hi all, > > I created this change instead: > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp 80 full_message[len] = '\0?; That should be 80 full_message[len - 1] = '\0?; len includes the space for the terminating NUL. From jcbeyler at google.com Tue Nov 13 20:57:43 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 13 Nov 2018 12:57:43 -0800 Subject: RFR: 8213622 - Windows VS2013 build failure - "'snprintf': identifier not found" In-Reply-To: References: <310386DB-21F6-4982-A900-86C15C4D1E80@oracle.com> <5bb7617e-89f7-1f43-5744-a6659ef6012d@redhat.com> Message-ID: Thanks Kim! Great catch! (oops) Sent back to the submit repo for testing and then I'll push it, Jc On Tue, Nov 13, 2018 at 11:00 AM Kim Barrett wrote: > > On Nov 12, 2018, at 4:11 PM, JC Beyler wrote: > > > > Hi all, > > > > I created this change instead: > > http://cr.openjdk.java.net/~jcbeyler/8213622/webrev.00/ > > test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp > 80 full_message[len] = '\0?; > > That should be > 80 full_message[len - 1] = '\0?; > > len includes the space for the terminating NUL. > > -- Thanks, Jc From jiangli.zhou at oracle.com Wed Nov 14 00:05:03 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Tue, 13 Nov 2018 16:05:03 -0800 Subject: RFR (S) Allow ResourceHashtable size to be specified at runtime In-Reply-To: References: Message-ID: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> Hi Ioi, The change looks reasonable to me in general. It would be helpful to see the performance difference with the expendable table. Do you have any data when large number of classes are loaded (>20000)? How much saving does it provide? Thanks, Jiangli On 11/8/18 10:35 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8213587 > http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ > > > TL;DR: -- add a subclass to ResourceHashtable to allow the table size > to be > ? ? ? ? ? dynamically specified when the table is constructed. > > > ? ?? ?? *** C++ template guru alert *** > > > I don't know much about C++ templates, so my attempt on doing this may be > ill-advised. > > I *think* that with my patch, the performance of existing code, which > uses > a statically-defined SIZE,? should not be affected, as the C++ compiler > should be able to constant-propagate and reduce the new code: > > ? ALWAYSINLINE unsigned size() const { > ??? if (SIZE != CONFIGURABLE_SIZE) { > ????? return SIZE; > ??? } else { > ????? return _configured_table_size; > ??? } > ? } > > ? ALWAYSINLINE Node** get_table() const { > ??? if (SIZE != CONFIGURABLE_SIZE) { > ????? return (Node**)(&_static_table[0]); > ??? } else { > ????? return _configured_table; > ??? } > ? } > > ? Node** lookup_node(unsigned hash, K const& key) { > ??? unsigned index = hash % size();??? <----- > ??? Node** table = get_table(); > ??? Node** ptr = &table[index]; ? <----- > > back to the old code: > > ? Node** lookup_node(unsigned hash, K const& key) { > ??? unsigned index = hash % SIZE;????? <----- > ??? Node** ptr = &_table[index];?????? <----- > > > If anyone has a better way of doing this, I'd love to hear it! > > Thanks! > - Ioi > > > > From vicente.romero at oracle.com Wed Nov 14 01:11:10 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 13 Nov 2018 20:11:10 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <03ad8985-b24a-f7e1-2d32-0f0a86ff1145@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> <12728fe8-b3ba-1794-4a98-03e963229b7f@oracle.com> <03ad8985-b24a-f7e1-2d32-0f0a86ff1145@oracle.com> Message-ID: Hi Alan, On 11/13/18 9:18 AM, Alan Bateman wrote: > On 13/11/2018 14:00, Vicente Romero wrote: >> any other comment after the last iteration? we are in a bit of a >> hurry to push this before the JDK 12 train departs :( > The original patch updated all the use sites (and tests) to specify > ASM7 for the API version. I just checked the webrev again now and it > seems to be just the ASM refresh now. Assuming all the tests are > passing and you've sorted out the mvlm test issues with Igor then I > suggest go ahead with this push and we can update the sites, as > needed, at a later time. in the last update I sent links to two patches [1] is the ASM7 only changes and [2] is the changes to the use sites. My plan is to push both together, but I split them to ease the review process. But still I will get your go and push it as good ;) > > -Alan Vicente [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.only.00/ [2] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.additional.changes.00/ From igor.ignatyev at oracle.com Wed Nov 14 01:30:25 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 13 Nov 2018 17:30:25 -0800 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> <12728fe8-b3ba-1794-4a98-03e963229b7f@oracle.com> <03ad8985-b24a-f7e1-2d32-0f0a86ff1145@oracle.com> Message-ID: <8E932DD0-5E34-45F5-8D07-F28904D5EAFC@oracle.com> Hi Vicente, you need to replace "@ignore 8194951: some mlvm tests fail w/ ASMv7" w/ "@ignore 8194951" in all the occurrences, as we have monitoring tools which expect @ignore to be followed by a space-separated list of bug ids. the rest looks good to me. Thanks, -- Igor > On Nov 13, 2018, at 5:11 PM, Vicente Romero wrote: > > Hi Alan, > > On 11/13/18 9:18 AM, Alan Bateman wrote: >> On 13/11/2018 14:00, Vicente Romero wrote: >>> any other comment after the last iteration? we are in a bit of a hurry to push this before the JDK 12 train departs :( >> The original patch updated all the use sites (and tests) to specify ASM7 for the API version. I just checked the webrev again now and it seems to be just the ASM refresh now. Assuming all the tests are passing and you've sorted out the mvlm test issues with Igor then I suggest go ahead with this push and we can update the sites, as needed, at a later time. > > in the last update I sent links to two patches [1] is the ASM7 only changes and [2] is the changes to the use sites. My plan is to push both together, but I split them to ease the review process. But still I will get your go and push it as good ;) > >> >> -Alan > > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.only.00/ > [2] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.additional.changes.00/ From david.holmes at oracle.com Wed Nov 14 02:56:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Nov 2018 12:56:27 +1000 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <5dd9d93c-9901-8756-d6be-c8c046726204@oracle.com> <3548029e-475f-586f-1df8-59813e4fc102@oracle.com> Message-ID: <53ccf241-c54d-fc07-3f9e-b9c9bf6200ce@oracle.com> Hi Amit, On 13/11/2018 11:34 pm, amith pawar wrote: > Hi David, > > I am from AMD and OCA processing is done. OK now ? Yes all good thanks. I have filed: https://bugs.openjdk.java.net/browse/JDK-8213827 Now you just need some NUMA experts (which is not me!) to look at it. I tried to apply your patch so that I could create a webrev for you, but it failed to apply: > hg import --no-commit --force 8213827.patch applying 8213827.patch patching file src/hotspot/os/linux/os_linux.cpp Hunk #2 FAILED at 2786 abort: bad hunk #3 @@ -2884,11 +2895,18 @@ (15 11 18 18) Thanks, David > > Thanks > Amit > > On Sat, Nov 10, 2018 at 6:07 AM David Holmes > wrote: > > Hi Amit, > > On 10/11/2018 10:34 AM, amith pawar wrote: > > Hi David, > > > > I have initiated for OCA processing and working as an individual > > contributor. > > Great! Welcome to OpenJDK. Please advise once your OCA is processed and > then we can properly look at your contribution. > > Thanks, > David > > > Thanks, > > Amit Pawar > > > > On Sat, Nov 10, 2018, 4:03 AM David Holmes > > > > > wrote: > > > >? ? ?Hi Amith, > > > >? ? ?I do not see you listed as a Contributor on the OCA list [1], > nor can I > >? ? ?tell from your email address whether you work for a company > that is a > >? ? ?signatory. Please confirm your status. > > > >? ? ?Thanks, > >? ? ?David > > > > > >? ? ?[1] https://www.oracle.com/technetwork/community/oca-486395.html > > > >? ? ?On 9/11/2018 10:23 PM, amith pawar wrote: > >? ? ? > Hi all, > >? ? ? > > >? ? ? > > >? ? ? > The flag UseNUMA (or UseNUMAInterleaving), has to > interleave old > >? ? ?gen, S1 > >? ? ? > and S2 region (if any other ) memory areas > >? ? ? > > >? ? ? > on requested Numa nodes and it should not configure itself to > >? ? ?access other > >? ? ? > Numa nodes. This issue is observed only when Java > >? ? ? > > >? ? ? > is allowed to run on fewer NUMA nodes than available on the > >? ? ?system with > >? ? ? > numactl membind and interleave options. Running > >? ? ? > > >? ? ? > on all the nodes does not have any effect. This will cause > some > >? ? ? > applications (objects residing in old gen and survivor > region) to > >? ? ? > > >? ? ? > run slower on system with large Numa nodes. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > I have described below in more detail for both numactl > MEMBIND and > >? ? ? > INTERLEAVE options. Addresses from both GC log and > >? ? ? > > >? ? ? > process numa maps clearly shows that the JAVA process is > >? ? ?configured to > >? ? ? > access other memory nodes even though it is not allowed. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > Two scenarios are: > >? ? ? > > >? ? ? >? ? ? ? ? ? 1. Numactl membind case : numactl -m 0-1, -N 0-1 > >? ? ? >? ? ? > arguments> > >? ? ? > > >? ? ? > > >? ? ? > Numa map shows these regions are INTERLEAVED ON ALL NODES > instead of > >? ? ? > specified Numa memory nodes 0 and 1. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > ----------------------------------- GC > >? ? ? > > > > ?HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > GC Options: -Xlog:gc*=info,gc+heap=debug > >? ? ? > > >? ? ? > > > > ?-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20) Heap before GC > >? ? ?invocations=21 > >? ? ? > (full 4): PSYoungGen? ? ? total 120604672K, used 11907587K > >? ? ? > [0x00002afc4b200000, 0x00002b198b200000, 0x00002b198b200000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?eden space > >? ? ?118525952K, 8% used > >? ? ? > [0x00002afc4b200000,0x00002b0bb1b376e0,0x00002b188d600000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ? ?lgrp 0 space > >? ? ?59262976K, 8% > >? ? ? > used > [0x00002afc4b200000,0x00002afd89bef450,0x00002b0a6c400000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ? ?lgrp 1 space > >? ? ?59262976K, 8% > >? ? ? > used > [0x00002b0a6c400000,0x00002b0bb1b376e0,0x00002b188d600000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?from space > 2078720K, > >? ? ?65% used > >? ? ? > [0x00002b190c400000,0x00002b195ef5a0d0,0x00002b198b200000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?to? ?space > 2078720K, > >? ? ?0% used > >? ? ? > [0x00002b188d600000,0x00002b188d600000,0x00002b190c400000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ParOldGen > ?total > >? ? ?2097152K, > >? ? ? > used 226685K [0x00002afbcb200000, 0x00002afc4b200000, > >? ? ?0x00002afc4b200000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? ?object space > >? ? ?2097152K, 10% used > >? ? ? > [0x00002afbcb200000,0x00002afbd8f5f460,0x00002afc4b200000) > >? ? ? > > >? ? ? > [602.180s][debug][gc,heap? ? ? ? ] GC(20)? Metaspace > ?used > >? ? ?28462K, > >? ? ? > capacity 29008K, committed 29184K, reserved 30720K > >? ? ? > > >? ? ? > ----------------------------------- GC HEAP > >? ? ? > > > > ?END-------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > ---------------------------------- PROCESS NUMA Maps > >? ? ? > > > > ?--------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > Command : cat /proc/4947/numa_maps > >? ? ? > > >? ? ? > > > > ?-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? >? ? ? =======> Following addresses are interleaved on all > nodes. > >? ? ? > > >? ? ? > 2afbb4f4c000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 > N2=2 N3=2 > >? ? ?N4=2 N5=2 > >? ? ? > N6=2 N7=2 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2afbb4f6c000 interleave:0-7 > >? ? ? > > >? ? ? > 2afbb7e88000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 > N2=6 N3=6 > >? ? ?N4=6 N5=6 > >? ? ? > N6=6 N7=6 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2afbbc000000 interleave:0-7 anon=8704 dirty=8704 N0=1600 > N1=1088 > >? ? ?N2=1088 > >? ? ? > N3=576 N4=1088 N5=1088 N6=1088 N7=1088 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2afbc3be6000 interleave:0-7 anon=6682 dirty=6682 N0=1027 > N1=1027 > >? ? ?N2=515 > >? ? ? > N3=515 N4=515 N5=1027 N6=1028 N7=1028 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2afbcb000000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 > N2=6 N3=6 > >? ? ?N4=6 N5=6 > >? ? ? > N6=6 N7=6 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2afbcb200000 interleave:0-7 anon=524288 dirty=524288 N0=65536 > >? ? ?N1=65536 > >? ? ? > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > >? ? ?kernelpagesize_kB=4 > >? ? ? > > >? ? ? >? ? ? ? ? ? ? ==> OLD GEN Address > >? ? ? > > >? ? ? > 2afc4b200000 prefer:0 anon=1536 dirty=1536 N0=1536 > >? ? ?kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b0a6c400000 prefer:1 anon=512 dirty=512 N1=512 > kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b188d600000 interleave:0-7 anon=1040384 dirty=1040384 > N0=130048 > >? ? ?N1=130048 > >? ? ? > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > >? ? ? > kernelpagesize_kB=4 > >? ? ? > > >? ? ? >? ? ? ? ? ? ? ==> Survivor Region > >? ? ? > > >? ? ? > 2b198b600000 interleave:0-7 anon=60929 dirty=60929 N0=7233 > >? ? ?N1=7744 N2=7232 > >? ? ? > N3=7744 N4=7744 N5=7744 N6=7744 N7=7744 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > ---------------------------------- PROCESS NUMA Maps > >? ? ? > > > > ?----------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > ---------------------------------- PROCESS status > >? ? ? > > > > ?----------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > Command : cat /proc/4947/status > >? ? ? > > >? ? ? > > > > ?-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > Cpus_allowed:? ?00000000,0000ffff,00000000,0000ffff > >? ? ? > > >? ? ? > Cpus_allowed_list:? ? ? 0-15,64-79 > >? ? ? > > >? ? ? > Mems_allowed: > >? ? ? > > > > ?00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > >? ? ? > > >? ? ? > Mems_allowed_list:? ? ? 0-7 > >? ? ? > > >? ? ? > --------------------------------- PROCESS status > >? ? ? > > > > ?------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? >? ? ? ? ? 2. NUMACTL Interleave? case: numactl -i 0-1, -N 0-1 > >? ? ? >? ? ? > its arguments> > >? ? ? > > >? ? ? > > >? ? ? > NUMAmaps below shows interleaved on all nodes instead of > >? ? ?specified Numa > >? ? ? > memory nodes 0 and 1. > >? ? ? > > >? ? ? > ----------------------------------- GC > >? ? ? > > > > ?HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > GC Options: -Xlog:gc*=info,gc+heap=debug > >? ? ? > > >? ? ? > > > > ?-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159) Heap before GC > >? ? ?invocations=160 > >? ? ? > (full 9): PSYoungGen? ? ? total 120604672K, used 30143454K > >? ? ? > [0x00002b9d47c00000, 0x00002bba87c00000, 0x00002bba87c00000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?eden space > >? ? ?118525952K, 24% > >? ? ? > used > [0x00002b9d47c00000,0x00002ba458400000,0x00002bb98a000000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 0 space > >? ? ?14815232K, 98% > >? ? ? > used > [0x00002b9d47c00000,0x00002ba0beb87c90,0x00002ba0d0000000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 1 space > >? ? ?14815232K, > >? ? ? > 100% used > [0x00002ba0d0000000,0x00002ba458400000,0x00002ba458400000) > >? ? ? > > >? ? ? >? ? ? ? ?==> Memory allocated on following nodes are unused. > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 2 space > >? ? ?14815232K, 0% > >? ? ? > used > [0x00002ba458400000,0x00002ba458400000,0x00002ba7e0800000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 3 space > >? ? ?14815232K, 0% > >? ? ? > used > [0x00002ba7e0800000,0x00002ba7e0800000,0x00002bab68c00000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 4 space > >? ? ?14815232K, 0% > >? ? ? > used > [0x00002bab68c00000,0x00002bab68c00000,0x00002baef1000000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 5 space > >? ? ?14815232K, 0% > >? ? ? > used > [0x00002baef1000000,0x00002baef1000000,0x00002bb279400000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 6 space > >? ? ?14815232K, 0% > >? ? ? > used > [0x00002bb279400000,0x00002bb279400000,0x00002bb601800000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ? ?lgrp 7 space > >? ? ?14819328K, 0% > >? ? ? > used > [0x00002bb601800000,0x00002bb601800000,0x00002bb98a000000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?from space > >? ? ?2078720K, 38% used > >? ? ? > [0x00002bba08e00000,0x00002bba3976fb70,0x00002bba87c00000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?to? ?space > >? ? ?2078720K, 0% used > >? ? ? > [0x00002bb98a000000,0x00002bb98a000000,0x00002bba08e00000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ParOldGen > ? ?total > >? ? ? > 2097152K, used 685229K [0x00002b9cc7c00000, > 0x00002b9d47c00000, > >? ? ? > 0x00002b9d47c00000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? ?object space > >? ? ?2097152K, 32% > >? ? ? > used > [0x00002b9cc7c00000,0x00002b9cf192b6e8,0x00002b9d47c00000) > >? ? ? > > >? ? ? > [2216.439s][debug][gc,heap? ? ? ? ] GC(159)? Metaspace > ? ?used > >? ? ?28753K, > >? ? ? > capacity 29257K, committed 29440K, reserved 30720K > >? ? ? > > >? ? ? > ----------------------------------- GC HEAP > >? ? ? > > > > ?END------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > ---------------------------------- PROCESS NUMA Maps > >? ? ? > > > > ?-------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > Command : cat /proc/pid/numa_maps > >? ? ? > > >? ? ? > > > > ?-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? >? ? ?==> Following addresses are interleaved on all the nodes. > >? ? ? > > >? ? ? > 2b9cb1992000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 > N2=2 N3=2 > >? ? ?N4=2 N5=2 > >? ? ? > N6=2 N7=2 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b9cb19b2000 interleave:0-7 > >? ? ? > > >? ? ? > 2b9cb3e65000 interleave:0-7 anon=50 dirty=50 N0=6 N1=6 > N2=6 N3=6 > >? ? ?N4=6 N5=7 > >? ? ? > N6=7 N7=6 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b9cb8a69000 interleave:0-7 anon=8599 dirty=8599 N0=626 > N1=1139 > >? ? ?N2=1139 > >? ? ? > N3=1139 N4=1139 N5=1139 N6=1139 N7=1139 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b9cc064f000 interleave:0-7 anon=6577 dirty=6577 N0=566 > N1=566 N2=566 > >? ? ? > N3=1078 N4=1078 N5=1078 N6=1078 N7=567 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b9cc7a69000 interleave:0-7 anon=50 dirty=50 N0=6 N1=7 > N2=7 N3=6 > >? ? ?N4=6 N5=6 > >? ? ? > N6=6 N7=6 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b9cc7c00000 interleave:0-7 anon=524288 dirty=524288 N0=65536 > >? ? ?N1=65536 > >? ? ? > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > >? ? ?kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2b9d47c00000 prefer:0 anon=2560 dirty=2560 N0=2560 > >? ? ?kernelpagesize_kB=4 > >? ? ? >? ? ?==> Logical group 1 > >? ? ? > > >? ? ? > 2ba0d0000000 > >? ? ? > prefer:1 > >? ? ? > ==> Logical group 2 > >? ? ? > > >? ? ? > 2ba458400000 prefer:2 > >? ? ? > > >? ? ? > ==> This one and below all are unnecessary and leaving > memory unused. > >? ? ? > > >? ? ? > 2ba7e0800000 prefer:3 > >? ? ? > > >? ? ? > 2bab68c00000 prefer:4 > >? ? ? > > >? ? ? > 2baef1000000 prefer:5 > >? ? ? > > >? ? ? > 2bb279400000 prefer:6 > >? ? ? > > >? ? ? > 2bb601800000 prefer:7 > >? ? ? > > >? ? ? > 2bb98a000000 interleave:0-7 anon=1040384 dirty=1040384 > N0=130048 > >? ? ?N1=130048 > >? ? ? > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > >? ? ? > kernelpagesize_kB=4 > >? ? ? > > >? ? ? > 2bba88000000 interleave:0-7 anon=60929 dirty=60929 N0=7745 > >? ? ?N1=7744 N2=7744 > >? ? ? > N3=7744 N4=7744 N5=7232 N6=7744 N7=7232 kernelpagesize_kB=4 > >? ? ? > > >? ? ? > ---------------------------------- PROCESS NUMA Maps > >? ? ? > > > > ?-------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? >? ?--------------------------------- PROCESS status > >? ? ? > > > > ?----------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > Command : cat /proc/pid/status > >? ? ? > > >? ? ? > > > > ?------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > Cpus_allowed:? ?00000000,0000ffff,00000000,0000ffff > >? ? ? > > >? ? ? > Cpus_allowed_list:? ? ? 0-15,64-79 > >? ? ? > > >? ? ? > Mems_allowed: > >? ? ? > > > > ?00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > >? ? ? > > >? ? ? > Mems_allowed_list:? ? ? 0-7 > >? ? ? > > >? ? ? >? ?--------------------------------- PROCESS status > >? ? ? > > > > ?----------------------------------------------------------------------------------------------------------------------------------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > Please note ?Mems_allowed? and ?Mems_allowed_list? list > also shows > >? ? ? > incorrect range. This could be LIBNUMA issue in case of > membind. > >? ? ? > > >? ? ? > When application is externally bound to fewer nodes then > calls to > >? ? ?libNUMA > >? ? ? > API?s SHOULD FAIL to interleave on all nodes but right now it > >? ? ? > > >? ? ? > does not with currently used/called API?s in JDK. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > How patch works ? > >? ? ? > > >? ? ? > The patch gets bitmask structure by calling following > API's (Man page > >? ? ? > definition for these functions are also given below). > >? ? ? > > >? ? ? >? ?1. For Membind : Use numa_get_membind to get membind bitmask > >? ? ?(already used > >? ? ? > in the code) > >? ? ? > > >? ? ? > > >? ? ? >? ? ? numa_get_membind() returns the mask of nodes from which > >? ? ?memory can > >? ? ? > currently be allocated.? If the returned mask is equal to > >? ? ?numa_all_nodes, > >? ? ? > then memory allocation is allowed from all nodes. > >? ? ? > > >? ? ? >? ?2. For Interleave: use numa_get_interleave_mask to get > >? ? ?interleave mask > >? ? ? > (currently not used/called in JDK) > >? ? ? >? ? ? ?numa_get_interleave_mask() returns the current > interleave > >? ? ?mask if the > >? ? ? > task's memory allocation policy is page interleaved. > Otherwise, this > >? ? ? > function returns an empty mask. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > Check node counts from both the Bitmasks to identify current > >? ? ?running mode. > >? ? ? > When Interleave mask is EMPTY then it is running in > membind else > >? ? ?it is in > >? ? ? > interleave mode. > >? ? ? > > >? ? ? > Call ?numa_interleave_memory? (called indirectly through > >? ? ??numa_make_global? > >? ? ? > funcition) function with right bitmask identified above > helps to > >? ? ?fix this > >? ? ? > issue. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > Improvement: > >? ? ? > > >? ? ? > This patch is tested on EPYC with SPECJBB benchmark and score > >? ? ?improvements > >? ? ? > are given below. > >? ? ? > > >? ? ? > 1. For NUMACTL membind > >? ? ? > > >? ? ? >? ? ? Max-jOPS improved by 5-12 % and Critical-jOPS by 2-6 % > >? ? ? > > >? ? ? > 2. For NUMACTL interleave (This patch fixes memory usage when > >? ? ?invoked with > >? ? ? > numactl -i) > >? ? ? > > >? ? ? >? ? ? Max-jOPS by 5-15% and Critical-jOPS by 11-100%. > >? ? ? > > >? ? ? > 3. With this fix, flag UseNUMAInterleaving turning on/off > has no > >? ? ?effect > >? ? ? > when externally interleaved through NUMACTL. > >? ? ? > > >? ? ? > 4. Flag UseNUMA score improved by ~14 % when enabled for > single > >? ? ?NUMA node. > >? ? ? > Currently it gets disabled when externally bound to single > node. > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > > >? ? ? > ---------------------- PATCH -------------------------- > >? ? ? > diff --git a/src/hotspot/os/linux/os_linux.cpp > >? ? ? > b/src/hotspot/os/linux/os_linux.cpp > >? ? ? > --- a/src/hotspot/os/linux/os_linux.cpp > >? ? ? > +++ b/src/hotspot/os/linux/os_linux.cpp > >? ? ? > @@ -2720,6 +2720,8 @@ > >? ? ? >? ?} > >? ? ? > > >? ? ? >? ?void os::numa_make_global(char *addr, size_t bytes) { > >? ? ? > +? if (!UseNUMAInterleaving) > >? ? ? > +? ? return ; > >? ? ? >? ? ?Linux::numa_interleave_memory(addr, bytes); > >? ? ? >? ?} > >? ? ? > > >? ? ? > @@ -2785,6 +2787,15 @@ > >? ? ? >? ? ? ? ?ids[i++] = node; > >? ? ? >? ? ? ?} > >? ? ? >? ? ?} > >? ? ? > + > >? ? ? > +? if (Linux::_numa_interleave_ptr != NULL ) { > >? ? ? > +? ? i = 0; > >? ? ? > +? ? for (int node = 0; node <= highest_node_number; node++) { > >? ? ? > +? ? ? if > (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, > >? ? ? > node)) { > >? ? ? > +? ? ? ? ids[i++] = node; > >? ? ? > +? ? ? } > >? ? ? > +? ? } > >? ? ? > +? } > >? ? ? >? ? ?return i; > >? ? ? >? ?} > >? ? ? > > >? ? ? > @@ -2884,11 +2895,18 @@ > >? ? ? >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > >? ? ? > "numa_distance"))); > >? ? ? > > ?set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > >? ? ? > > ?libnuma_v2_dlsym(handle, > >? ? ? > "numa_get_membind"))); > >? ? ? > + > >? ? ? > > > > ?set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, > >? ? ? > + > libnuma_v2_dlsym(handle, > >? ? ? > "numa_get_interleave_mask"))); > >? ? ? > > >? ? ? >? ? ? ? ?if (numa_available() != -1) { > >? ? ? > +? ? ? ? struct bitmask *bmp; > >? ? ? >? ? ? ? ? ?set_numa_all_nodes((unsigned > long*)libnuma_dlsym(handle, > >? ? ? > "numa_all_nodes")); > >? ? ? >? ? ? ? ? ?set_numa_all_nodes_ptr((struct bitmask > >? ? ?**)libnuma_dlsym(handle, > >? ? ? > "numa_all_nodes_ptr")); > >? ? ? >? ? ? ? ? ?set_numa_nodes_ptr((struct bitmask > **)libnuma_dlsym(handle, > >? ? ? > "numa_nodes_ptr")); > >? ? ? > +? ? ? ? bmp = _numa_get_interleave_mask(); > >? ? ? > +? ? ? ? set_numa_interleave_ptr(&bmp); > >? ? ? > +? ? ? ? bmp = _numa_get_membind(); > >? ? ? > +? ? ? ? set_numa_membind_ptr(&bmp); > >? ? ? >? ? ? ? ? ?// Create an index -> node mapping, since nodes > are not > >? ? ?always > >? ? ? > consecutive > >? ? ? >? ? ? ? ? ?_nindex_to_node = new (ResourceObj::C_HEAP, > mtInternal) > >? ? ? > GrowableArray(0, true); > >? ? ? >? ? ? ? ? ?rebuild_nindex_to_node_map(); > >? ? ? > @@ -3015,9 +3033,12 @@ > >? ? ? >? ?os::Linux::numa_bitmask_isbitset_func_t > >? ? ?os::Linux::_numa_bitmask_isbitset; > >? ? ? >? ?os::Linux::numa_distance_func_t os::Linux::_numa_distance; > >? ? ? >? ?os::Linux::numa_get_membind_func_t > os::Linux::_numa_get_membind; > >? ? ? > +os::Linux::numa_get_interleave_mask_func_t > >? ? ? > os::Linux::_numa_get_interleave_mask; > >? ? ? >? ?unsigned long* os::Linux::_numa_all_nodes; > >? ? ? >? ?struct bitmask* os::Linux::_numa_all_nodes_ptr; > >? ? ? >? ?struct bitmask* os::Linux::_numa_nodes_ptr; > >? ? ? > +struct bitmask* os::Linux::_numa_interleave_ptr; > >? ? ? > +struct bitmask* os::Linux::_numa_membind_ptr; > >? ? ? > > >? ? ? >? ?bool os::pd_uncommit_memory(char* addr, size_t size) { > >? ? ? >? ? ?uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, > >? ? ? > @@ -4997,13 +5018,38 @@ > >? ? ? >? ? ? ?if (!Linux::libnuma_init()) { > >? ? ? >? ? ? ? ?UseNUMA = false; > >? ? ? >? ? ? ?} else { > >? ? ? > -? ? ? if ((Linux::numa_max_node() < 1) || > >? ? ?Linux::isbound_to_single_node()) > >? ? ? > { > >? ? ? > -? ? ? ? // If there's only one node (they start from 0) or if > >? ? ?the process > >? ? ? > -? ? ? ? // is bound explicitly to a single node using > membind, > >? ? ?disable > >? ? ? > NUMA. > >? ? ? > -? ? ? ? UseNUMA = false; > >? ? ? > + > >? ? ? > +? ? // Identify whether running in membind or interleave > mode. > >? ? ? > +? ? struct bitmask *bmp; > >? ? ? > +? ? bool _is_membind = false; > >? ? ? > +? ? bool _is_interleaved = false; > >? ? ? > + > >? ? ? > +? ? // Check for membind mode. > >? ? ? > +? ? bmp = Linux::_numa_membind_ptr; > >? ? ? > +? ? for (int node = 0; node <= Linux::numa_max_node() ; > node++) { > >? ? ? > +? ? ? if (Linux::_numa_bitmask_isbitset(bmp, node)) { > >? ? ? > +? ? ? ? _is_membind = true; > >? ? ? >? ? ? ? ?} > >? ? ? >? ? ? ?} > >? ? ? > > >? ? ? > +? ? // Check for interleave mode. > >? ? ? > +? ? bmp = Linux::_numa_interleave_ptr; > >? ? ? > +? ? for (int node = 0; node <= Linux::numa_max_node() ; > node++) { > >? ? ? > +? ? ? if (Linux::_numa_bitmask_isbitset(bmp, node)) { > >? ? ? > +? ? ? ? _is_interleaved = true; > >? ? ? > +? ? ? ? // Set membind to false as interleave mode allows all > >? ? ?nodes to be > >? ? ? > used. > >? ? ? > +? ? ? ? _is_membind = false; > >? ? ? > +? ? ? } > >? ? ? > +? ? } > >? ? ? > + > >? ? ? > +? ? if (_is_membind) > >? ? ? > +? ? ? Linux::set_numa_interleave_ptr (NULL); > >? ? ? > + > >? ? ? > +? ? if (_is_interleaved) > >? ? ? > +? ? ? Linux::set_numa_membind_ptr (NULL); > >? ? ? > + > >? ? ? > +? ? } > >? ? ? > + > >? ? ? >? ? ? ?if (UseParallelGC && UseNUMA && UseLargePages && > >? ? ? > !can_commit_large_page_memory()) { > >? ? ? >? ? ? ? ?// With SHM and HugeTLBFS large pages we cannot > uncommit > >? ? ?a page, so > >? ? ? > there's no way > >? ? ? >? ? ? ? ?// we can make the adaptive lgrp chunk resizing > work. If > >? ? ?the user > >? ? ? > specified both > >? ? ? > diff --git a/src/hotspot/os/linux/os_linux.hpp > >? ? ? > b/src/hotspot/os/linux/os_linux.hpp > >? ? ? > --- a/src/hotspot/os/linux/os_linux.hpp > >? ? ? > +++ b/src/hotspot/os/linux/os_linux.hpp > >? ? ? > @@ -222,6 +222,7 @@ > >? ? ? >? ? ?typedef void (*numa_interleave_memory_func_t)(void *start, > >? ? ?size_t size, > >? ? ? > unsigned long *nodemask); > >? ? ? >? ? ?typedef void (*numa_interleave_memory_v2_func_t)(void > *start, > >? ? ?size_t > >? ? ? > size, struct bitmask* mask); > >? ? ? >? ? ?typedef struct bitmask* (*numa_get_membind_func_t)(void); > >? ? ? > +? typedef struct bitmask* > (*numa_get_interleave_mask_func_t)(void); > >? ? ? > > >? ? ? >? ? ?typedef void (*numa_set_bind_policy_func_t)(int policy); > >? ? ? >? ? ?typedef int (*numa_bitmask_isbitset_func_t)(struct > bitmask *bmp, > >? ? ? > unsigned int n); > >? ? ? > @@ -239,9 +240,12 @@ > >? ? ? >? ? ?static numa_bitmask_isbitset_func_t > _numa_bitmask_isbitset; > >? ? ? >? ? ?static numa_distance_func_t _numa_distance; > >? ? ? >? ? ?static numa_get_membind_func_t _numa_get_membind; > >? ? ? > +? static numa_get_interleave_mask_func_t > _numa_get_interleave_mask; > >? ? ? >? ? ?static unsigned long* _numa_all_nodes; > >? ? ? >? ? ?static struct bitmask* _numa_all_nodes_ptr; > >? ? ? >? ? ?static struct bitmask* _numa_nodes_ptr; > >? ? ? > +? static struct bitmask* _numa_interleave_ptr; > >? ? ? > +? static struct bitmask* _numa_membind_ptr; > >? ? ? > > >? ? ? >? ? ?static void set_sched_getcpu(sched_getcpu_func_t func) { > >? ? ?_sched_getcpu = > >? ? ? > func; } > >? ? ? >? ? ?static void set_numa_node_to_cpus(numa_node_to_cpus_func_t > >? ? ?func) { > >? ? ? > _numa_node_to_cpus = func; } > >? ? ? > @@ -255,9 +259,12 @@ > >? ? ? >? ? ?static void > >? ? ?set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) > >? ? ? > { _numa_bitmask_isbitset = func; } > >? ? ? >? ? ?static void set_numa_distance(numa_distance_func_t func) { > >? ? ? > _numa_distance = func; } > >? ? ? >? ? ?static void > set_numa_get_membind(numa_get_membind_func_t func) { > >? ? ? > _numa_get_membind = func; } > >? ? ? > +? static void > >? ? ?set_numa_get_interleave_mask(numa_get_interleave_mask_func_t > >? ? ? > func) { _numa_get_interleave_mask = func; } > >? ? ? >? ? ?static void set_numa_all_nodes(unsigned long* ptr) { > >? ? ?_numa_all_nodes = > >? ? ? > ptr; } > >? ? ? >? ? ?static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > >? ? ? > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > >? ? ? >? ? ?static void set_numa_nodes_ptr(struct bitmask **ptr) { > >? ? ?_numa_nodes_ptr = > >? ? ? > (ptr == NULL ? NULL : *ptr); } > >? ? ? > +? static void set_numa_interleave_ptr(struct bitmask **ptr) { > >? ? ? > _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } > >? ? ? > +? static void set_numa_membind_ptr(struct bitmask **ptr) { > >? ? ? > _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } > >? ? ? >? ? ?static int sched_getcpu_syscall(void); > >? ? ? >? ? public: > >? ? ? >? ? ?static int sched_getcpu()? { return _sched_getcpu != > NULL ? > >? ? ? > _sched_getcpu() : -1; } > >? ? ? > @@ -275,7 +282,10 @@ > >? ? ? >? ? ?static void numa_interleave_memory(void *start, size_t > size) { > >? ? ? >? ? ? ?// Use v2 api if available > >? ? ? >? ? ? ?if (_numa_interleave_memory_v2 != NULL && > >? ? ?_numa_all_nodes_ptr != NULL) > >? ? ? > { > >? ? ? > -? ? ? _numa_interleave_memory_v2(start, size, > _numa_all_nodes_ptr); > >? ? ? > +? ? ? if (_numa_interleave_ptr != NULL) > >? ? ? > +? ? ? ? _numa_interleave_memory_v2(start, size, > >? ? ?_numa_interleave_ptr); > >? ? ? > +? ? ? else > >? ? ? > +? ? ? ? _numa_interleave_memory_v2(start, size, > _numa_membind_ptr); > >? ? ? >? ? ? ?} else if (_numa_interleave_memory != NULL && > >? ? ?_numa_all_nodes != NULL) > >? ? ? > { > >? ? ? >? ? ? ? ?_numa_interleave_memory(start, size, _numa_all_nodes); > >? ? ? >? ? ? ?} > >? ? ? > ----------------------- PATCH > ----------------------------------- > >? ? ? > > >? ? ? > > >? ? ? > Thanks > >? ? ? > Amit Pawar > >? ? ? > > > > > > > -- > With best regards, > amit pawar From vicente.romero at oracle.com Wed Nov 14 03:31:25 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 13 Nov 2018 22:31:25 -0500 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: <8E932DD0-5E34-45F5-8D07-F28904D5EAFC@oracle.com> References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> <53250b15-b7f2-e12e-bc6e-2c117e57c088@oracle.com> <12728fe8-b3ba-1794-4a98-03e963229b7f@oracle.com> <03ad8985-b24a-f7e1-2d32-0f0a86ff1145@oracle.com> <8E932DD0-5E34-45F5-8D07-F28904D5EAFC@oracle.com> Message-ID: <94506b83-b579-f494-1a34-c7cb9b1b4808@oracle.com> Hi Igor, Thanks for your comment. I have already applied in my local copy. Vicente On 11/13/18 8:30 PM, Igor Ignatyev wrote: > Hi Vicente, > > you need to replace "@ignore 8194951: some mlvm tests fail w/ ASMv7" w/ "@ignore 8194951" in all the occurrences, as we have monitoring tools which expect @ignore to be followed by a space-separated list of bug ids. the rest looks good to me. > > Thanks, > -- Igor > >> On Nov 13, 2018, at 5:11 PM, Vicente Romero wrote: >> >> Hi Alan, >> >> On 11/13/18 9:18 AM, Alan Bateman wrote: >>> On 13/11/2018 14:00, Vicente Romero wrote: >>>> any other comment after the last iteration? we are in a bit of a hurry to push this before the JDK 12 train departs :( >>> The original patch updated all the use sites (and tests) to specify ASM7 for the API version. I just checked the webrev again now and it seems to be just the ASM refresh now. Assuming all the tests are passing and you've sorted out the mvlm test issues with Igor then I suggest go ahead with this push and we can update the sites, as needed, at a later time. >> in the last update I sent links to two patches [1] is the ASM7 only changes and [2] is the changes to the use sites. My plan is to push both together, but I split them to ease the review process. But still I will get your go and push it as good ;) >> >>> -Alan >> Vicente >> >> [1] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.only.00/ >> [2] http://cr.openjdk.java.net/~vromero/8213480/webrev.asm.7.additional.changes.00/ From david.holmes at oracle.com Wed Nov 14 04:13:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 14 Nov 2018 14:13:00 +1000 Subject: RFR: JDK-8213480: update internal ASM version to 7.0 In-Reply-To: References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> Message-ID: On 14/11/2018 12:40 am, Brian Goetz wrote: > But don?t they? ?We intend to start generating condy in classfiles from > javac soon; at that point, anyone not using ASM7 will fail when reading > classfiles generated by javac. See later email. They need this for nestmates now as well. Cheers, David >> On Nov 8, 2018, at 4:03 AM, David Holmes > > wrote: >> >> Is it that case that the code the uses the ASM library, like the JFR >> code and jlink code, and the tests, doesn't actually _have to_ change >> to specifying Opcodes.ASM7 unless they plan on using ASM7 features? If >> so then you could split out the actual update of ASM from the updates >> to the users of ASM (some of which may be quite fine with ASM5). > From matthias.baesken at sap.com Wed Nov 14 08:50:54 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 14 Nov 2018 08:50:54 +0000 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: <02bc7237-d7c7-dd8f-2d3d-eebe64a63eee@oracle.com> References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> <02bc7237-d7c7-dd8f-2d3d-eebe64a63eee@oracle.com> Message-ID: Hello, looks like finally Windows Server 2019 is availability after the release stop . David found this info from November 13, 2018 : https://cloudblogs.microsoft.com/windowsserver/2018/11/13/update-on-windows-server-2019-availability/ " ... In addition to extensive internal validation, we have taken time to closely monitor feedback and diagnostic data from our Windows Insiders and from millions of devices on the Windows 10 October 2018 Update. There is no further evidence of data loss. Based on this data, today we are beginning the re-release of Windows Server 2019, ... " Alan / Bob - can I you as reviewers ? Best regards , Matthias > -----Original Message----- > From: Alan Bateman > Sent: Samstag, 20. Oktober 2018 10:06 > To: Baesken, Matthias > Cc: hotspot-dev Source Developers ; core- > libs-dev at openjdk.java.net; Moldenhauer, Niclas > > Subject: Re: RFR : 8211106: [windows] Update OS detection code to > recognize Windows Server 2019 > > On 19/10/2018 13:11, Baesken, Matthias wrote: > > Hi Alan, I agree it is a pity that the minor version (dwMinorVersion) was > not increased . > > Looking at the build number was not my preferred way to handle this . > > > > My understanding (checked with our internal SAP-MS porting guys) is > that the buildNumber of Windows server 2016 will not get larger than > > the minimum introduced buildNumber of Windows server 2019 . > > > The changes in your webrev look okay to me although I assume we need to > be cautious about back porting until it becomes clear if Windows Server > 2019 will be re-released or not.? The Windows Server blog [1] suggests > that the release is paused while an issue is investigated and maybe a > re-release will mean a new build number at least. > > -Alan > > [1] > https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows- > server-2019-now-generally-available From amith.pawar at gmail.com Wed Nov 14 09:05:27 2018 From: amith.pawar at gmail.com (amith pawar) Date: Wed, 14 Nov 2018 14:35:27 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: <53ccf241-c54d-fc07-3f9e-b9c9bf6200ce@oracle.com> References: <5dd9d93c-9901-8756-d6be-c8c046726204@oracle.com> <3548029e-475f-586f-1df8-59813e4fc102@oracle.com> <53ccf241-c54d-fc07-3f9e-b9c9bf6200ce@oracle.com> Message-ID: Hi David, Thanks for your help and filing a new defect. Import works fine when copied from this email but fails form the defect link. Can anyone please help to review this patch ? Thanks, Amit On Wed, Nov 14, 2018 at 8:26 AM David Holmes wrote: > Hi Amit, > > On 13/11/2018 11:34 pm, amith pawar wrote: > > Hi David, > > > > I am from AMD and OCA processing is done. OK now ? > > Yes all good thanks. > > I have filed: > > https://bugs.openjdk.java.net/browse/JDK-8213827 > > Now you just need some NUMA experts (which is not me!) to look at it. > > I tried to apply your patch so that I could create a webrev for you, but > it failed to apply: > > > hg import --no-commit --force 8213827.patch > applying 8213827.patch > patching file src/hotspot/os/linux/os_linux.cpp > Hunk #2 FAILED at 2786 > abort: bad hunk #3 @@ -2884,11 +2895,18 @@ > (15 11 18 18) > > Thanks, > David > > > > > Thanks > > Amit > > > > On Sat, Nov 10, 2018 at 6:07 AM David Holmes > > wrote: > > > > Hi Amit, > > > > On 10/11/2018 10:34 AM, amith pawar wrote: > > > Hi David, > > > > > > I have initiated for OCA processing and working as an individual > > > contributor. > > > > Great! Welcome to OpenJDK. Please advise once your OCA is processed > and > > then we can properly look at your contribution. > > > > Thanks, > > David > > > > > Thanks, > > > Amit Pawar > > > > > > On Sat, Nov 10, 2018, 4:03 AM David Holmes > > > > > > > > wrote: > > > > > > Hi Amith, > > > > > > I do not see you listed as a Contributor on the OCA list [1], > > nor can I > > > tell from your email address whether you work for a company > > that is a > > > signatory. Please confirm your status. > > > > > > Thanks, > > > David > > > > > > > > > [1] > https://www.oracle.com/technetwork/community/oca-486395.html > > > > > > On 9/11/2018 10:23 PM, amith pawar wrote: > > > > Hi all, > > > > > > > > > > > > The flag UseNUMA (or UseNUMAInterleaving), has to > > interleave old > > > gen, S1 > > > > and S2 region (if any other ) memory areas > > > > > > > > on requested Numa nodes and it should not configure itself > to > > > access other > > > > Numa nodes. This issue is observed only when Java > > > > > > > > is allowed to run on fewer NUMA nodes than available on the > > > system with > > > > numactl membind and interleave options. Running > > > > > > > > on all the nodes does not have any effect. This will cause > > some > > > > applications (objects residing in old gen and survivor > > region) to > > > > > > > > run slower on system with large Numa nodes. > > > > > > > > > > > > > > > > I have described below in more detail for both numactl > > MEMBIND and > > > > INTERLEAVE options. Addresses from both GC log and > > > > > > > > process numa maps clearly shows that the JAVA process is > > > configured to > > > > access other memory nodes even though it is not allowed. > > > > > > > > > > > > > > > > Two scenarios are: > > > > > > > > 1. Numactl membind case : numactl -m 0-1, -N 0-1 > > > > > > arguments> > > > > > > > > > > > > Numa map shows these regions are INTERLEAVED ON ALL NODES > > instead of > > > > specified Numa memory nodes 0 and 1. > > > > > > > > > > > > > > > > ----------------------------------- GC > > > > > > > > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > [602.180s][debug][gc,heap ] GC(20) Heap before GC > > > invocations=21 > > > > (full 4): PSYoungGen total 120604672K, used 11907587K > > > > [0x00002afc4b200000, 0x00002b198b200000, > 0x00002b198b200000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) eden space > > > 118525952K, 8% used > > > > [0x00002afc4b200000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) lgrp 0 space > > > 59262976K, 8% > > > > used > > [0x00002afc4b200000,0x00002afd89bef450,0x00002b0a6c400000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) lgrp 1 space > > > 59262976K, 8% > > > > used > > [0x00002b0a6c400000,0x00002b0bb1b376e0,0x00002b188d600000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) from space > > 2078720K, > > > 65% used > > > > [0x00002b190c400000,0x00002b195ef5a0d0,0x00002b198b200000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) to space > > 2078720K, > > > 0% used > > > > [0x00002b188d600000,0x00002b188d600000,0x00002b190c400000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) ParOldGen > > total > > > 2097152K, > > > > used 226685K [0x00002afbcb200000, 0x00002afc4b200000, > > > 0x00002afc4b200000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) object space > > > 2097152K, 10% used > > > > [0x00002afbcb200000,0x00002afbd8f5f460,0x00002afc4b200000) > > > > > > > > [602.180s][debug][gc,heap ] GC(20) Metaspace > > used > > > 28462K, > > > > capacity 29008K, committed 29184K, reserved 30720K > > > > > > > > ----------------------------------- GC HEAP > > > > > > > > > > END-------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > > > > > --------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > Command : cat /proc/4947/numa_maps > > > > > > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > =======> Following addresses are interleaved on all > > nodes. > > > > > > > > 2afbb4f4c000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 > > N2=2 N3=2 > > > N4=2 N5=2 > > > > N6=2 N7=2 kernelpagesize_kB=4 > > > > > > > > 2afbb4f6c000 interleave:0-7 > > > > > > > > 2afbb7e88000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 > > N2=6 N3=6 > > > N4=6 N5=6 > > > > N6=6 N7=6 kernelpagesize_kB=4 > > > > > > > > 2afbbc000000 interleave:0-7 anon=8704 dirty=8704 N0=1600 > > N1=1088 > > > N2=1088 > > > > N3=576 N4=1088 N5=1088 N6=1088 N7=1088 kernelpagesize_kB=4 > > > > > > > > 2afbc3be6000 interleave:0-7 anon=6682 dirty=6682 N0=1027 > > N1=1027 > > > N2=515 > > > > N3=515 N4=515 N5=1027 N6=1028 N7=1028 kernelpagesize_kB=4 > > > > > > > > 2afbcb000000 interleave:0-7 anon=50 dirty=50 N0=7 N1=7 > > N2=6 N3=6 > > > N4=6 N5=6 > > > > N6=6 N7=6 kernelpagesize_kB=4 > > > > > > > > 2afbcb200000 interleave:0-7 anon=524288 dirty=524288 > N0=65536 > > > N1=65536 > > > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > > > kernelpagesize_kB=4 > > > > > > > > ==> OLD GEN Address > > > > > > > > 2afc4b200000 prefer:0 anon=1536 dirty=1536 N0=1536 > > > kernelpagesize_kB=4 > > > > > > > > 2b0a6c400000 prefer:1 anon=512 dirty=512 N1=512 > > kernelpagesize_kB=4 > > > > > > > > 2b188d600000 interleave:0-7 anon=1040384 dirty=1040384 > > N0=130048 > > > N1=130048 > > > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > > > kernelpagesize_kB=4 > > > > > > > > ==> Survivor Region > > > > > > > > 2b198b600000 interleave:0-7 anon=60929 dirty=60929 N0=7233 > > > N1=7744 N2=7232 > > > > N3=7744 N4=7744 N5=7744 N6=7744 N7=7744 kernelpagesize_kB=4 > > > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > ---------------------------------- PROCESS status > > > > > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > Command : cat /proc/4947/status > > > > > > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > > > > > > > Cpus_allowed_list: 0-15,64-79 > > > > > > > > Mems_allowed: > > > > > > > > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > > > > > Mems_allowed_list: 0-7 > > > > > > > > --------------------------------- PROCESS status > > > > > > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > > > > > 2. NUMACTL Interleave case: numactl -i 0-1, -N > 0-1 > > > > > > its arguments> > > > > > > > > > > > > NUMAmaps below shows interleaved on all nodes instead of > > > specified Numa > > > > memory nodes 0 and 1. > > > > > > > > ----------------------------------- GC > > > > > > > > > > HEAP-------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > GC Options: -Xlog:gc*=info,gc+heap=debug > > > > > > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) Heap before GC > > > invocations=160 > > > > (full 9): PSYoungGen total 120604672K, used 30143454K > > > > [0x00002b9d47c00000, 0x00002bba87c00000, > 0x00002bba87c00000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) eden space > > > 118525952K, 24% > > > > used > > [0x00002b9d47c00000,0x00002ba458400000,0x00002bb98a000000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 0 > space > > > 14815232K, 98% > > > > used > > [0x00002b9d47c00000,0x00002ba0beb87c90,0x00002ba0d0000000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 1 > space > > > 14815232K, > > > > 100% used > > [0x00002ba0d0000000,0x00002ba458400000,0x00002ba458400000) > > > > > > > > ==> Memory allocated on following nodes are unused. > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 2 > space > > > 14815232K, 0% > > > > used > > [0x00002ba458400000,0x00002ba458400000,0x00002ba7e0800000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 3 > space > > > 14815232K, 0% > > > > used > > [0x00002ba7e0800000,0x00002ba7e0800000,0x00002bab68c00000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 4 > space > > > 14815232K, 0% > > > > used > > [0x00002bab68c00000,0x00002bab68c00000,0x00002baef1000000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 5 > space > > > 14815232K, 0% > > > > used > > [0x00002baef1000000,0x00002baef1000000,0x00002bb279400000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 6 > space > > > 14815232K, 0% > > > > used > > [0x00002bb279400000,0x00002bb279400000,0x00002bb601800000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) lgrp 7 > space > > > 14819328K, 0% > > > > used > > [0x00002bb601800000,0x00002bb601800000,0x00002bb98a000000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) from space > > > 2078720K, 38% used > > > > [0x00002bba08e00000,0x00002bba3976fb70,0x00002bba87c00000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) to space > > > 2078720K, 0% used > > > > [0x00002bb98a000000,0x00002bb98a000000,0x00002bba08e00000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) ParOldGen > > total > > > > 2097152K, used 685229K [0x00002b9cc7c00000, > > 0x00002b9d47c00000, > > > > 0x00002b9d47c00000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) object space > > > 2097152K, 32% > > > > used > > [0x00002b9cc7c00000,0x00002b9cf192b6e8,0x00002b9d47c00000) > > > > > > > > [2216.439s][debug][gc,heap ] GC(159) Metaspace > > used > > > 28753K, > > > > capacity 29257K, committed 29440K, reserved 30720K > > > > > > > > ----------------------------------- GC HEAP > > > > > > > > > > END------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > Command : cat /proc/pid/numa_maps > > > > > > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > ==> Following addresses are interleaved on all the > nodes. > > > > > > > > 2b9cb1992000 interleave:0-7 anon=16 dirty=16 N0=2 N1=2 > > N2=2 N3=2 > > > N4=2 N5=2 > > > > N6=2 N7=2 kernelpagesize_kB=4 > > > > > > > > 2b9cb19b2000 interleave:0-7 > > > > > > > > 2b9cb3e65000 interleave:0-7 anon=50 dirty=50 N0=6 N1=6 > > N2=6 N3=6 > > > N4=6 N5=7 > > > > N6=7 N7=6 kernelpagesize_kB=4 > > > > > > > > 2b9cb8a69000 interleave:0-7 anon=8599 dirty=8599 N0=626 > > N1=1139 > > > N2=1139 > > > > N3=1139 N4=1139 N5=1139 N6=1139 N7=1139 kernelpagesize_kB=4 > > > > > > > > 2b9cc064f000 interleave:0-7 anon=6577 dirty=6577 N0=566 > > N1=566 N2=566 > > > > N3=1078 N4=1078 N5=1078 N6=1078 N7=567 kernelpagesize_kB=4 > > > > > > > > 2b9cc7a69000 interleave:0-7 anon=50 dirty=50 N0=6 N1=7 > > N2=7 N3=6 > > > N4=6 N5=6 > > > > N6=6 N7=6 kernelpagesize_kB=4 > > > > > > > > 2b9cc7c00000 interleave:0-7 anon=524288 dirty=524288 > N0=65536 > > > N1=65536 > > > > N2=65536 N3=65536 N4=65536 N5=65536 N6=65536 N7=65536 > > > kernelpagesize_kB=4 > > > > > > > > 2b9d47c00000 prefer:0 anon=2560 dirty=2560 N0=2560 > > > kernelpagesize_kB=4 > > > > ==> Logical group 1 > > > > > > > > 2ba0d0000000 > > > > prefer:1 > > > > ==> Logical group 2 > > > > > > > > 2ba458400000 prefer:2 > > > > > > > > ==> This one and below all are unnecessary and leaving > > memory unused. > > > > > > > > 2ba7e0800000 prefer:3 > > > > > > > > 2bab68c00000 prefer:4 > > > > > > > > 2baef1000000 prefer:5 > > > > > > > > 2bb279400000 prefer:6 > > > > > > > > 2bb601800000 prefer:7 > > > > > > > > 2bb98a000000 interleave:0-7 anon=1040384 dirty=1040384 > > N0=130048 > > > N1=130048 > > > > N2=130048 N3=130048 N4=130048 N5=130048 N6=130048 N7=130048 > > > > kernelpagesize_kB=4 > > > > > > > > 2bba88000000 interleave:0-7 anon=60929 dirty=60929 N0=7745 > > > N1=7744 N2=7744 > > > > N3=7744 N4=7744 N5=7232 N6=7744 N7=7232 kernelpagesize_kB=4 > > > > > > > > ---------------------------------- PROCESS NUMA Maps > > > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > --------------------------------- PROCESS status > > > > > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > Command : cat /proc/pid/status > > > > > > > > > > > > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > Cpus_allowed: 00000000,0000ffff,00000000,0000ffff > > > > > > > > Cpus_allowed_list: 0-15,64-79 > > > > > > > > Mems_allowed: > > > > > > > > > > 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,000000ff > > > > > > > > Mems_allowed_list: 0-7 > > > > > > > > --------------------------------- PROCESS status > > > > > > > > > > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please note ?Mems_allowed? and ?Mems_allowed_list? list > > also shows > > > > incorrect range. This could be LIBNUMA issue in case of > > membind. > > > > > > > > When application is externally bound to fewer nodes then > > calls to > > > libNUMA > > > > API?s SHOULD FAIL to interleave on all nodes but right now > it > > > > > > > > does not with currently used/called API?s in JDK. > > > > > > > > > > > > > > > > How patch works ? > > > > > > > > The patch gets bitmask structure by calling following > > API's (Man page > > > > definition for these functions are also given below). > > > > > > > > 1. For Membind : Use numa_get_membind to get membind > bitmask > > > (already used > > > > in the code) > > > > > > > > > > > > numa_get_membind() returns the mask of nodes from > which > > > memory can > > > > currently be allocated. If the returned mask is equal to > > > numa_all_nodes, > > > > then memory allocation is allowed from all nodes. > > > > > > > > 2. For Interleave: use numa_get_interleave_mask to get > > > interleave mask > > > > (currently not used/called in JDK) > > > > numa_get_interleave_mask() returns the current > > interleave > > > mask if the > > > > task's memory allocation policy is page interleaved. > > Otherwise, this > > > > function returns an empty mask. > > > > > > > > > > > > > > > > Check node counts from both the Bitmasks to identify > current > > > running mode. > > > > When Interleave mask is EMPTY then it is running in > > membind else > > > it is in > > > > interleave mode. > > > > > > > > Call ?numa_interleave_memory? (called indirectly through > > > ?numa_make_global? > > > > funcition) function with right bitmask identified above > > helps to > > > fix this > > > > issue. > > > > > > > > > > > > > > > > > > > > > > > > Improvement: > > > > > > > > This patch is tested on EPYC with SPECJBB benchmark and > score > > > improvements > > > > are given below. > > > > > > > > 1. For NUMACTL membind > > > > > > > > Max-jOPS improved by 5-12 % and Critical-jOPS by 2-6 % > > > > > > > > 2. For NUMACTL interleave (This patch fixes memory usage > when > > > invoked with > > > > numactl -i) > > > > > > > > Max-jOPS by 5-15% and Critical-jOPS by 11-100%. > > > > > > > > 3. With this fix, flag UseNUMAInterleaving turning on/off > > has no > > > effect > > > > when externally interleaved through NUMACTL. > > > > > > > > 4. Flag UseNUMA score improved by ~14 % when enabled for > > single > > > NUMA node. > > > > Currently it gets disabled when externally bound to single > > node. > > > > > > > > > > > > > > > > > > > > > > > > ---------------------- PATCH -------------------------- > > > > diff --git a/src/hotspot/os/linux/os_linux.cpp > > > > b/src/hotspot/os/linux/os_linux.cpp > > > > --- a/src/hotspot/os/linux/os_linux.cpp > > > > +++ b/src/hotspot/os/linux/os_linux.cpp > > > > @@ -2720,6 +2720,8 @@ > > > > } > > > > > > > > void os::numa_make_global(char *addr, size_t bytes) { > > > > + if (!UseNUMAInterleaving) > > > > + return ; > > > > Linux::numa_interleave_memory(addr, bytes); > > > > } > > > > > > > > @@ -2785,6 +2787,15 @@ > > > > ids[i++] = node; > > > > } > > > > } > > > > + > > > > + if (Linux::_numa_interleave_ptr != NULL ) { > > > > + i = 0; > > > > + for (int node = 0; node <= highest_node_number; > node++) { > > > > + if > > (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, > > > > node)) { > > > > + ids[i++] = node; > > > > + } > > > > + } > > > > + } > > > > return i; > > > > } > > > > > > > > @@ -2884,11 +2895,18 @@ > > > > > libnuma_dlsym(handle, > > > > "numa_distance"))); > > > > > > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > > > > > > libnuma_v2_dlsym(handle, > > > > "numa_get_membind"))); > > > > + > > > > > > > > > > set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, > > > > + > > libnuma_v2_dlsym(handle, > > > > "numa_get_interleave_mask"))); > > > > > > > > if (numa_available() != -1) { > > > > + struct bitmask *bmp; > > > > set_numa_all_nodes((unsigned > > long*)libnuma_dlsym(handle, > > > > "numa_all_nodes")); > > > > set_numa_all_nodes_ptr((struct bitmask > > > **)libnuma_dlsym(handle, > > > > "numa_all_nodes_ptr")); > > > > set_numa_nodes_ptr((struct bitmask > > **)libnuma_dlsym(handle, > > > > "numa_nodes_ptr")); > > > > + bmp = _numa_get_interleave_mask(); > > > > + set_numa_interleave_ptr(&bmp); > > > > + bmp = _numa_get_membind(); > > > > + set_numa_membind_ptr(&bmp); > > > > // Create an index -> node mapping, since nodes > > are not > > > always > > > > consecutive > > > > _nindex_to_node = new (ResourceObj::C_HEAP, > > mtInternal) > > > > GrowableArray(0, true); > > > > rebuild_nindex_to_node_map(); > > > > @@ -3015,9 +3033,12 @@ > > > > os::Linux::numa_bitmask_isbitset_func_t > > > os::Linux::_numa_bitmask_isbitset; > > > > os::Linux::numa_distance_func_t > os::Linux::_numa_distance; > > > > os::Linux::numa_get_membind_func_t > > os::Linux::_numa_get_membind; > > > > +os::Linux::numa_get_interleave_mask_func_t > > > > os::Linux::_numa_get_interleave_mask; > > > > unsigned long* os::Linux::_numa_all_nodes; > > > > struct bitmask* os::Linux::_numa_all_nodes_ptr; > > > > struct bitmask* os::Linux::_numa_nodes_ptr; > > > > +struct bitmask* os::Linux::_numa_interleave_ptr; > > > > +struct bitmask* os::Linux::_numa_membind_ptr; > > > > > > > > bool os::pd_uncommit_memory(char* addr, size_t size) { > > > > uintptr_t res = (uintptr_t) ::mmap(addr, size, > PROT_NONE, > > > > @@ -4997,13 +5018,38 @@ > > > > if (!Linux::libnuma_init()) { > > > > UseNUMA = false; > > > > } else { > > > > - if ((Linux::numa_max_node() < 1) || > > > Linux::isbound_to_single_node()) > > > > { > > > > - // If there's only one node (they start from 0) > or if > > > the process > > > > - // is bound explicitly to a single node using > > membind, > > > disable > > > > NUMA. > > > > - UseNUMA = false; > > > > + > > > > + // Identify whether running in membind or interleave > > mode. > > > > + struct bitmask *bmp; > > > > + bool _is_membind = false; > > > > + bool _is_interleaved = false; > > > > + > > > > + // Check for membind mode. > > > > + bmp = Linux::_numa_membind_ptr; > > > > + for (int node = 0; node <= Linux::numa_max_node() ; > > node++) { > > > > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > > > + _is_membind = true; > > > > } > > > > } > > > > > > > > + // Check for interleave mode. > > > > + bmp = Linux::_numa_interleave_ptr; > > > > + for (int node = 0; node <= Linux::numa_max_node() ; > > node++) { > > > > + if (Linux::_numa_bitmask_isbitset(bmp, node)) { > > > > + _is_interleaved = true; > > > > + // Set membind to false as interleave mode allows > all > > > nodes to be > > > > used. > > > > + _is_membind = false; > > > > + } > > > > + } > > > > + > > > > + if (_is_membind) > > > > + Linux::set_numa_interleave_ptr (NULL); > > > > + > > > > + if (_is_interleaved) > > > > + Linux::set_numa_membind_ptr (NULL); > > > > + > > > > + } > > > > + > > > > if (UseParallelGC && UseNUMA && UseLargePages && > > > > !can_commit_large_page_memory()) { > > > > // With SHM and HugeTLBFS large pages we cannot > > uncommit > > > a page, so > > > > there's no way > > > > // we can make the adaptive lgrp chunk resizing > > work. If > > > the user > > > > specified both > > > > diff --git a/src/hotspot/os/linux/os_linux.hpp > > > > b/src/hotspot/os/linux/os_linux.hpp > > > > --- a/src/hotspot/os/linux/os_linux.hpp > > > > +++ b/src/hotspot/os/linux/os_linux.hpp > > > > @@ -222,6 +222,7 @@ > > > > typedef void (*numa_interleave_memory_func_t)(void > *start, > > > size_t size, > > > > unsigned long *nodemask); > > > > typedef void (*numa_interleave_memory_v2_func_t)(void > > *start, > > > size_t > > > > size, struct bitmask* mask); > > > > typedef struct bitmask* > (*numa_get_membind_func_t)(void); > > > > + typedef struct bitmask* > > (*numa_get_interleave_mask_func_t)(void); > > > > > > > > typedef void (*numa_set_bind_policy_func_t)(int > policy); > > > > typedef int (*numa_bitmask_isbitset_func_t)(struct > > bitmask *bmp, > > > > unsigned int n); > > > > @@ -239,9 +240,12 @@ > > > > static numa_bitmask_isbitset_func_t > > _numa_bitmask_isbitset; > > > > static numa_distance_func_t _numa_distance; > > > > static numa_get_membind_func_t _numa_get_membind; > > > > + static numa_get_interleave_mask_func_t > > _numa_get_interleave_mask; > > > > static unsigned long* _numa_all_nodes; > > > > static struct bitmask* _numa_all_nodes_ptr; > > > > static struct bitmask* _numa_nodes_ptr; > > > > + static struct bitmask* _numa_interleave_ptr; > > > > + static struct bitmask* _numa_membind_ptr; > > > > > > > > static void set_sched_getcpu(sched_getcpu_func_t func) > { > > > _sched_getcpu = > > > > func; } > > > > static void > set_numa_node_to_cpus(numa_node_to_cpus_func_t > > > func) { > > > > _numa_node_to_cpus = func; } > > > > @@ -255,9 +259,12 @@ > > > > static void > > > set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) > > > > { _numa_bitmask_isbitset = func; } > > > > static void set_numa_distance(numa_distance_func_t > func) { > > > > _numa_distance = func; } > > > > static void > > set_numa_get_membind(numa_get_membind_func_t func) { > > > > _numa_get_membind = func; } > > > > + static void > > > set_numa_get_interleave_mask(numa_get_interleave_mask_func_t > > > > func) { _numa_get_interleave_mask = func; } > > > > static void set_numa_all_nodes(unsigned long* ptr) { > > > _numa_all_nodes = > > > > ptr; } > > > > static void set_numa_all_nodes_ptr(struct bitmask > **ptr) { > > > > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > > > > static void set_numa_nodes_ptr(struct bitmask **ptr) { > > > _numa_nodes_ptr = > > > > (ptr == NULL ? NULL : *ptr); } > > > > + static void set_numa_interleave_ptr(struct bitmask > **ptr) { > > > > _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } > > > > + static void set_numa_membind_ptr(struct bitmask **ptr) { > > > > _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } > > > > static int sched_getcpu_syscall(void); > > > > public: > > > > static int sched_getcpu() { return _sched_getcpu != > > NULL ? > > > > _sched_getcpu() : -1; } > > > > @@ -275,7 +282,10 @@ > > > > static void numa_interleave_memory(void *start, size_t > > size) { > > > > // Use v2 api if available > > > > if (_numa_interleave_memory_v2 != NULL && > > > _numa_all_nodes_ptr != NULL) > > > > { > > > > - _numa_interleave_memory_v2(start, size, > > _numa_all_nodes_ptr); > > > > + if (_numa_interleave_ptr != NULL) > > > > + _numa_interleave_memory_v2(start, size, > > > _numa_interleave_ptr); > > > > + else > > > > + _numa_interleave_memory_v2(start, size, > > _numa_membind_ptr); > > > > } else if (_numa_interleave_memory != NULL && > > > _numa_all_nodes != NULL) > > > > { > > > > _numa_interleave_memory(start, size, > _numa_all_nodes); > > > > } > > > > ----------------------- PATCH > > ----------------------------------- > > > > > > > > > > > > Thanks > > > > Amit Pawar > > > > > > > > > > > > > > > -- > > With best regards, > > amit pawar > -- With best regards, amit pawar From coleen.phillimore at oracle.com Wed Nov 14 12:44:07 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 14 Nov 2018 07:44:07 -0500 Subject: RFR 8213751: ClassLoaderDataGraph::cld_do() should sometimes require CLDG_lock Message-ID: <4c0d62ed-da1c-f07e-0b37-beb14c80323c@oracle.com> Summary: Give GC own version of cld_oops_do and use cld_do for runtime calls. This also adds a NoSafepointVerifier in the CLDG iterator. Tested with hs-tier1-3, vmTestbase/nsk, gc, runtime and JFR tests locally. open webrev at http://cr.openjdk.java.net/~coleenp/8213751.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8213751 Thanks, Coleen From vicente.romero at oracle.com Wed Nov 14 14:13:13 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Wed, 14 Nov 2018 09:13:13 -0500 Subject: JDK12 talks ASM7 now! In-Reply-To: References: <888cb7a5-5e32-6480-c4da-a5b98822ed9f@oracle.com> <0f4294c0-f4e5-e8ba-e310-f9b1472da45f@oracle.com> Message-ID: <70c308c2-53f3-9b85-eaf2-1f3eba575a59@oracle.com> Hi guys, Thanks everyone for the feedback. I pushed the patch yesterday, Vicente From bob.vandette at oracle.com Wed Nov 14 16:59:09 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 14 Nov 2018 11:59:09 -0500 Subject: RFR : 8211106: [windows] Update OS detection code to recognize Windows Server 2019 In-Reply-To: References: <3F7E2792-DA3E-4AFD-9869-95841D4ED896@oracle.com> <02bc7237-d7c7-dd8f-2d3d-eebe64a63eee@oracle.com> Message-ID: <8E16D3C3-ADA9-4E5D-9036-EBAE81D0959D@oracle.com> > On Nov 14, 2018, at 3:50 AM, Baesken, Matthias wrote: > > Hello, looks like finally Windows Server 2019 is availability after the release stop . > > David found this info from November 13, 2018 : > > https://cloudblogs.microsoft.com/windowsserver/2018/11/13/update-on-windows-server-2019-availability/ > > > " ... In addition to extensive internal validation, we have taken time to closely monitor feedback and diagnostic data from our Windows Insiders and from millions of devices on the Windows 10 October 2018 Update. > There is no further evidence of data loss. Based on this data, today we are beginning the re-release of Windows Server 2019, ... " > > > Alan / Bob - can I you as reviewers ? I?m ok with this. Bob. > > > Best regards , Matthias > > > >> -----Original Message----- >> From: Alan Bateman >> Sent: Samstag, 20. Oktober 2018 10:06 >> To: Baesken, Matthias >> Cc: hotspot-dev Source Developers ; core- >> libs-dev at openjdk.java.net; Moldenhauer, Niclas >> >> Subject: Re: RFR : 8211106: [windows] Update OS detection code to >> recognize Windows Server 2019 >> >> On 19/10/2018 13:11, Baesken, Matthias wrote: >>> Hi Alan, I agree it is a pity that the minor version (dwMinorVersion) was >> not increased . >>> Looking at the build number was not my preferred way to handle this . >>> >>> My understanding (checked with our internal SAP-MS porting guys) is >> that the buildNumber of Windows server 2016 will not get larger than >>> the minimum introduced buildNumber of Windows server 2019 . >>> >> The changes in your webrev look okay to me although I assume we need to >> be cautious about back porting until it becomes clear if Windows Server >> 2019 will be re-released or not. The Windows Server blog [1] suggests >> that the release is paused while an issue is investigated and maybe a >> re-release will mean a new build number at least. >> >> -Alan >> >> [1] >> https://cloudblogs.microsoft.com/windowsserver/2018/10/02/windows- >> server-2019-now-generally-available From ioi.lam at oracle.com Wed Nov 14 17:09:58 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 14 Nov 2018 09:09:58 -0800 Subject: RFR (S) Allow ResourceHashtable size to be specified at runtime In-Reply-To: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> Message-ID: On 11/13/18 4:05 PM, Jiangli Zhou wrote: > Hi Ioi, > > The change looks reasonable to me in general. It would be helpful to > see the performance difference with the expendable table. Do you have > any data when large number of classes are loaded (>20000)? How much > saving does it provide? > Hi Jiangli, thanks for the review. For dumping 30292 classes: BEFORE: 93.971 sec AFTER:? 34.761 sec Thanks - Ioi > Thanks, > > Jiangli > > > On 11/8/18 10:35 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8213587 >> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >> >> >> TL;DR: -- add a subclass to ResourceHashtable to allow the table size >> to be >> ? ? ? ? ? dynamically specified when the table is constructed. >> >> >> ? ?? ?? *** C++ template guru alert *** >> >> >> I don't know much about C++ templates, so my attempt on doing this >> may be >> ill-advised. >> >> I *think* that with my patch, the performance of existing code, which >> uses >> a statically-defined SIZE,? should not be affected, as the C++ compiler >> should be able to constant-propagate and reduce the new code: >> >> ? ALWAYSINLINE unsigned size() const { >> ??? if (SIZE != CONFIGURABLE_SIZE) { >> ????? return SIZE; >> ??? } else { >> ????? return _configured_table_size; >> ??? } >> ? } >> >> ? ALWAYSINLINE Node** get_table() const { >> ??? if (SIZE != CONFIGURABLE_SIZE) { >> ????? return (Node**)(&_static_table[0]); >> ??? } else { >> ????? return _configured_table; >> ??? } >> ? } >> >> ? Node** lookup_node(unsigned hash, K const& key) { >> ??? unsigned index = hash % size();??? <----- >> ??? Node** table = get_table(); >> ??? Node** ptr = &table[index]; ? <----- >> >> back to the old code: >> >> ? Node** lookup_node(unsigned hash, K const& key) { >> ??? unsigned index = hash % SIZE;????? <----- >> ??? Node** ptr = &_table[index];?????? <----- >> >> >> If anyone has a better way of doing this, I'd love to hear it! >> >> Thanks! >> - Ioi >> >> >> >> > From matthias.baesken at sap.com Wed Nov 14 17:16:10 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 14 Nov 2018 17:16:10 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Thomas , so are you fine with the latest revision http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ of the patch , can I add you as reviewer ? David - is the info added by Thomas sufficient for you? Any other reviewers ? > -----Original Message----- > From: Thomas St?fe > Sent: Samstag, 10. November 2018 14:04 > To: David Holmes > Cc: Baesken, Matthias ; Langer, Christoph > ; Volker Simonis ; > HotSpot Open Source Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > Hi David, > > On Sat, Nov 10, 2018 at 12:59 PM David Holmes > wrote: > > > > Hi Matthias, > > > > On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk > , I created another webrev : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > + STEP("printing user info") > > + if (ExtensiveErrorReports) { > > + if (_verbose) { > > + os::Posix::print_user_info(st, buf, sizeof(buf)); > > + } > > + } > > > > I don't understand why we explicitly need _verbose if we've asked for > > ExtensiveErrorreports? > > That flag has a different purpose: _verbose distinguishes the two > calls to VMError::report(): the first one to print a small report to > stderr, the second print a large report to the hs-err file. We only > want to print to the hs-err file, so _verbose is still needed. > > > > > Also, ideally the STEP would be inside the guard as otherwise we will > > just print the step description followed by nothing. If the macro > > expansion makes that impossible then we should have an else clause that > > prints something like: > > > > - disabled (use -XX:+ExtensiveErrorReports to see these details) > > > > We may even go a step further (but in a separate patch) and make > "ExtensiveErrorReports" a property of the STEP, to be given as > parameter of the STEP macro. > > Thanks, Thomas > From thomas.stuefe at gmail.com Wed Nov 14 17:42:45 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 14 Nov 2018 18:42:45 +0100 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Matthias, none of my notes from my earlier mail are addressed? I guess my mail got lost in all that discussion. Here you go: + static void print_user_info(outputStream* st, char* buf, size_t buflen); You can get rid of buf+buflen arguments, not needed anymore --- http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/posix/os_posix.cpp.udiff.html In os::Posix::print_user_info(), for brevity, I would print it all one line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: rwxr--r--" --- The "removing" text does not make sense and can be removed. Best Regards, Thomas On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias wrote: > > Hi Thomas , so are you fine with the latest revision > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > of the patch , can I add you as reviewer ? > > David - is the info added by Thomas sufficient for you? > > Any other reviewers ? > > > > > -----Original Message----- > > From: Thomas St?fe > > Sent: Samstag, 10. November 2018 14:04 > > To: David Holmes > > Cc: Baesken, Matthias ; Langer, Christoph > > ; Volker Simonis ; > > HotSpot Open Source Developers > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > Hi David, > > > > On Sat, Nov 10, 2018 at 12:59 PM David Holmes > > wrote: > > > > > > Hi Matthias, > > > > > > On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk > > , I created another webrev : > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > > > + STEP("printing user info") > > > + if (ExtensiveErrorReports) { > > > + if (_verbose) { > > > + os::Posix::print_user_info(st, buf, sizeof(buf)); > > > + } > > > + } > > > > > > I don't understand why we explicitly need _verbose if we've asked for > > > ExtensiveErrorreports? > > > > That flag has a different purpose: _verbose distinguishes the two > > calls to VMError::report(): the first one to print a small report to > > stderr, the second print a large report to the hs-err file. We only > > want to print to the hs-err file, so _verbose is still needed. > > > > > > > > Also, ideally the STEP would be inside the guard as otherwise we will > > > just print the step description followed by nothing. If the macro > > > expansion makes that impossible then we should have an else clause that > > > prints something like: > > > > > > - disabled (use -XX:+ExtensiveErrorReports to see these details) > > > > > > > We may even go a step further (but in a separate patch) and make > > "ExtensiveErrorReports" a property of the STEP, to be given as > > parameter of the STEP macro. > > > > Thanks, Thomas > > > From david.holmes at oracle.com Wed Nov 14 22:32:29 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Nov 2018 08:32:29 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: <6bcff027-1e03-aeb9-3397-e17351449a0a@oracle.com> Hi Matthias, On 15/11/2018 3:16 am, Baesken, Matthias wrote: > Hi Thomas , so are you fine with the latest revision > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > of the patch , can I add you as reviewer ? > > David - is the info added by Thomas sufficient for you? Thomas explained about _verbose. I still have concerns that we will print the header but then no info - I think that needs to be addressed. I did not review the actual print_user_info as I'm not familiar with the actual content. Thanks, David > Any other reviewers ? > > > >> -----Original Message----- >> From: Thomas St?fe >> Sent: Samstag, 10. November 2018 14:04 >> To: David Holmes >> Cc: Baesken, Matthias ; Langer, Christoph >> ; Volker Simonis ; >> HotSpot Open Source Developers >> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >> >> Hi David, >> >> On Sat, Nov 10, 2018 at 12:59 PM David Holmes >> wrote: >>> >>> Hi Matthias, >>> >>> On 9/11/2018 3:13 AM, Baesken, Matthias wrote: >>>> Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into jdk/jdk >> , I created another webrev : >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ >>>> >>>> >>>> The user info output is now guarded by ExtensiveErrorReports . >>> >>> + STEP("printing user info") >>> + if (ExtensiveErrorReports) { >>> + if (_verbose) { >>> + os::Posix::print_user_info(st, buf, sizeof(buf)); >>> + } >>> + } >>> >>> I don't understand why we explicitly need _verbose if we've asked for >>> ExtensiveErrorreports? >> >> That flag has a different purpose: _verbose distinguishes the two >> calls to VMError::report(): the first one to print a small report to >> stderr, the second print a large report to the hs-err file. We only >> want to print to the hs-err file, so _verbose is still needed. >> >>> >>> Also, ideally the STEP would be inside the guard as otherwise we will >>> just print the step description followed by nothing. If the macro >>> expansion makes that impossible then we should have an else clause that >>> prints something like: >>> >>> - disabled (use -XX:+ExtensiveErrorReports to see these details) >>> >> >> We may even go a step further (but in a separate patch) and make >> "ExtensiveErrorReports" a property of the STEP, to be given as >> parameter of the STEP macro. >> >> Thanks, Thomas >> > From jiangli.zhou at oracle.com Thu Nov 15 01:14:03 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 14 Nov 2018 17:14:03 -0800 Subject: RFR (S) Allow ResourceHashtable size to be specified at runtime In-Reply-To: References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> Message-ID: <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> Hi Ioi, On 11/14/18 9:09 AM, Ioi Lam wrote: > > > On 11/13/18 4:05 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> The change looks reasonable to me in general. It would be helpful to >> see the performance difference with the expendable table. Do you have >> any data when large number of classes are loaded (>20000)? How much >> saving does it provide? >> > > Hi Jiangli, thanks for the review. For dumping 30292 classes: > > BEFORE: 93.971 sec > AFTER:? 34.761 sec Thanks for the data! That's about 2.6x improvement with large set of classes. Thanks, Jiangli > > Thanks > - Ioi > >> Thanks, >> >> Jiangli >> >> >> On 11/8/18 10:35 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>> >>> >>> TL;DR: -- add a subclass to ResourceHashtable to allow the table >>> size to be >>> ? ? ? ? ? dynamically specified when the table is constructed. >>> >>> >>> ? ?? ?? *** C++ template guru alert *** >>> >>> >>> I don't know much about C++ templates, so my attempt on doing this >>> may be >>> ill-advised. >>> >>> I *think* that with my patch, the performance of existing code, >>> which uses >>> a statically-defined SIZE,? should not be affected, as the C++ compiler >>> should be able to constant-propagate and reduce the new code: >>> >>> ? ALWAYSINLINE unsigned size() const { >>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>> ????? return SIZE; >>> ??? } else { >>> ????? return _configured_table_size; >>> ??? } >>> ? } >>> >>> ? ALWAYSINLINE Node** get_table() const { >>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>> ????? return (Node**)(&_static_table[0]); >>> ??? } else { >>> ????? return _configured_table; >>> ??? } >>> ? } >>> >>> ? Node** lookup_node(unsigned hash, K const& key) { >>> ??? unsigned index = hash % size();??? <----- >>> ??? Node** table = get_table(); >>> ??? Node** ptr = &table[index]; ? <----- >>> >>> back to the old code: >>> >>> ? Node** lookup_node(unsigned hash, K const& key) { >>> ??? unsigned index = hash % SIZE;????? <----- >>> ??? Node** ptr = &_table[index];?????? <----- >>> >>> >>> If anyone has a better way of doing this, I'd love to hear it! >>> >>> Thanks! >>> - Ioi >>> >>> >>> >>> >> > From anthony.scarpino at oracle.com Thu Nov 15 01:33:27 2018 From: anthony.scarpino at oracle.com (Anthony Scarpino) Date: Wed, 14 Nov 2018 17:33:27 -0800 Subject: Problems with AES-GCM native acceleration In-Reply-To: <1b1fe9e4-ad71-7d48-13eb-48dbe267962e@oracle.com> References: <92a16009b9e047eacf102b006bc0612bac4a2cfb.camel@redhat.com> <1b1fe9e4-ad71-7d48-13eb-48dbe267962e@oracle.com> Message-ID: <864a763d-0d67-d894-234c-fbbef78e0320@oracle.com> I agree with Adam that this is more of a tuning issue and not a problem with the crypto. Sending multiple updates is a hack. I've been aware of this bug for a while and I do not understand why this is a significant problem. The stackoverflow comments say it takes 50 seconds to trigger the intrinsic. If this is a long running server application slowness for the first 50 seconds is trivial. For smaller operations, those are commonly small transactions, not decrypting a 3GB file. If it cannot be resolved by commandline options and this is occurring in a real world situation, please explain it fully. If this is only for benchmarking, then that's not a real world situation. Tony On 11/14/18 8:41 AM, Adam Petcher wrote: > I'm adding back in hotspot-dev, because this is a somewhat tricky topic > related to intrinsics and JIT. Hopefully, a Hotspot expert can correct > anything that I say below that is wrong, and suggest any solutions that > I missed. > > The AES acceleration is implemented in a HotSpot intrinsic. In order for > it to kick in, the code must be JIT compiled by the VM. As I understand > it, this only happens to some particular method after it has been called > a certain number of times. The rules that determine this number are > somewhat complicated, but I think you can guarantee JIT in the default > configuration by calling a method 10,000 times. > > The doFinal method calls the update method, so either one should trigger > the acceleration as long as you call it enough. Breaking the message up > into smaller chunks and calling update on each one works only because it > ends up calling the update method more. You should be able to trigger > the acceleration by calling doFinal more, too. > > The reason why the workaround doesn't work with decryption is that the > decryption routine buffers the ciphertext and then decrypts it all at > the end. So calling update multiple times and then calling doFinal at > the end is essentially the same as calling doFinal once with the entire > ciphertext. > > So here are some solutions that you may want to try: > > 1) In your benchmark, run at least 10,000 "warmup" iterations of > whatever you are trying to do at the beginning, without timing it. This > is a good idea for benchmarks, anyway. If it helps, you can try using > smaller buffers in your "warmup" phase in order to get it to complete > faster. > > 2) Try -XX:CompileThreshold=(some number smaller than 10000) as an > argument to java. This will make JIT kick in sooner across the board. > Obviously, this should be done carefully in production, since it will > impact the performance of the entire program. > > 3) I haven't tried this, but running with an AOTed java.base module may > also help. See the section titled "Steps to generate and use an AOT > library for the java.base module" in the AOT JEP[1]. > > "Fixing" this issue in the JDK is non-trivial, because it gets into the > behavior of the VM and JIT. I don't really like the idea of modifying > doFinal (to break up the operation into multiple update calls) or > modifying the decryption operation (to decrypt immediately and buffer > plaintext) in order to work around this issue. Perhaps there is a better > way for the VM to handle cases like this, in which a method is not > called often, but the interpreted execution takes a long time to > complete when it is called. Perhaps a VM expert will have some > additional thoughts here. > > [1] https://openjdk.java.net/jeps/295 > > On 11/14/2018 9:49 AM, Severin Gehwolf wrote: >> Dropping hotspot-dev and adding security-dev. >> >> On Wed, 2018-11-14 at 14:39 +0200, Gidon Gershinsky wrote: >>> Hi, >>> >>> We are working on an encryption mechanism at the Apache Parquet - >>> that will enable efficient analytics on encrypted data by frameworks >>> such as Apache Spark. >>> https://github.com/apache/parquet-format/blob/encryption/Encryption.md >>> https://www.slideshare.net/databricks/efficient-spark-analytics-on-encrypted-data-with-gidon-gershinsky >>> >>> >>> We came across an AES-related issue in the Java HostSpot engine that >>> looks like a substantial problem for us in both Spark and Parquet >>> workloads. The bug report had been accepted a while ago: >>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8201633 >>> >>> The fix should hopefully be rather straightforward though. >>> Could you help us with that? I have a couple of small samples >>> reproducing the problem. >>> >>> (If I'm writing to a wrong mailing list - I apologize, please point >>> me in the right direction). >>> >>> Cheers, Gidon. From ioi.lam at oracle.com Thu Nov 15 05:31:36 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 14 Nov 2018 21:31:36 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> Message-ID: Coleen pointed out to me off-line that the good old (and ugly) BasicHashtable already supports resizing. I think that might be a better starting point for this RFE: http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ I wrote a new template class called "KVHashtable" (copying the style from ResourceHashtable). That way, you can instantiate different (Key -> Value) mappings without writing tons of boilerplate code. The performance is similar to my previous version, and the code is much cleaner. I also renamed the RFE title, as well as the subject line of this RFR e-mail. Thanks - Ioi On 11/14/18 5:14 PM, Jiangli Zhou wrote: > Hi Ioi, > > On 11/14/18 9:09 AM, Ioi Lam wrote: >> >> >> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> The change looks reasonable to me in general. It would be helpful to >>> see the performance difference with the expendable table. Do you >>> have any data when large number of classes are loaded (>20000)? How >>> much saving does it provide? >>> >> >> Hi Jiangli, thanks for the review. For dumping 30292 classes: >> >> BEFORE: 93.971 sec >> AFTER:? 34.761 sec > > Thanks for the data! That's about 2.6x improvement with large set of > classes. > > Thanks, > Jiangli >> >> Thanks >> - Ioi >> >>> Thanks, >>> >>> Jiangli >>> >>> >>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>> >>>> >>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table >>>> size to be >>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>> >>>> >>>> ? ?? ?? *** C++ template guru alert *** >>>> >>>> >>>> I don't know much about C++ templates, so my attempt on doing this >>>> may be >>>> ill-advised. >>>> >>>> I *think* that with my patch, the performance of existing code, >>>> which uses >>>> a statically-defined SIZE,? should not be affected, as the C++ >>>> compiler >>>> should be able to constant-propagate and reduce the new code: >>>> >>>> ? ALWAYSINLINE unsigned size() const { >>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>> ????? return SIZE; >>>> ??? } else { >>>> ????? return _configured_table_size; >>>> ??? } >>>> ? } >>>> >>>> ? ALWAYSINLINE Node** get_table() const { >>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>> ????? return (Node**)(&_static_table[0]); >>>> ??? } else { >>>> ????? return _configured_table; >>>> ??? } >>>> ? } >>>> >>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>> ??? unsigned index = hash % size();??? <----- >>>> ??? Node** table = get_table(); >>>> ??? Node** ptr = &table[index]; ? <----- >>>> >>>> back to the old code: >>>> >>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>> ??? unsigned index = hash % SIZE;????? <----- >>>> ??? Node** ptr = &_table[index];?????? <----- >>>> >>>> >>>> If anyone has a better way of doing this, I'd love to hear it! >>>> >>>> Thanks! >>>> - Ioi >>>> >>>> >>>> >>>> >>> >> > From matthias.baesken at sap.com Thu Nov 15 11:51:09 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 15 Nov 2018 11:51:09 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: Hi Thomas, http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.3/ - got rid of buf+buflen arguments of print_user_info - removed "Removing" - adjusted the output a bit in os::Posix::print_user_info() - vmError.cpp added a short output line with info in case ( ExtensiveErrorReports && _verbose ) is false Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Mittwoch, 14. November 2018 18:43 > To: Baesken, Matthias > Cc: David Holmes ; Langer, Christoph > ; Volker Simonis ; > HotSpot Open Source Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > Hi Matthias, > > none of my notes from my earlier mail are addressed? I guess my mail > got lost in all that discussion. Here you go: > > + static void print_user_info(outputStream* st, char* buf, size_t buflen); > > You can get rid of buf+buflen arguments, not needed anymore > --- > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > posix/os_posix.cpp.udiff.html > > In os::Posix::print_user_info(), for brevity, I would print it all one > line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: > rwxr--r--" > > --- > > The "removing" text does not make sense and can be removed. > > Best Regards, Thomas > > > On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias > wrote: > > > > Hi Thomas , so are you fine with the latest revision > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > of the patch , can I add you as reviewer ? > > > > David - is the info added by Thomas sufficient for you? > > > > Any other reviewers ? > > > > > > > > > -----Original Message----- > > > From: Thomas St?fe > > > Sent: Samstag, 10. November 2018 14:04 > > > To: David Holmes > > > Cc: Baesken, Matthias ; Langer, Christoph > > > ; Volker Simonis > ; > > > HotSpot Open Source Developers > > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > > > Hi David, > > > > > > On Sat, Nov 10, 2018 at 12:59 PM David Holmes > > > > wrote: > > > > > > > > Hi Matthias, > > > > > > > > On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > > > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into > jdk/jdk > > > , I created another webrev : > > > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > > > > > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > > > > > + STEP("printing user info") > > > > + if (ExtensiveErrorReports) { > > > > + if (_verbose) { > > > > + os::Posix::print_user_info(st, buf, sizeof(buf)); > > > > + } > > > > + } > > > > > > > > I don't understand why we explicitly need _verbose if we've asked for > > > > ExtensiveErrorreports? > > > > > > That flag has a different purpose: _verbose distinguishes the two > > > calls to VMError::report(): the first one to print a small report to > > > stderr, the second print a large report to the hs-err file. We only > > > want to print to the hs-err file, so _verbose is still needed. > > > > > > > > > > > Also, ideally the STEP would be inside the guard as otherwise we will > > > > just print the step description followed by nothing. If the macro > > > > expansion makes that impossible then we should have an else clause > that > > > > prints something like: > > > > > > > > - disabled (use -XX:+ExtensiveErrorReports to see these details) > > > > > > > > > > We may even go a step further (but in a separate patch) and make > > > "ExtensiveErrorReports" a property of the STEP, to be given as > > > parameter of the STEP macro. > > > > > > Thanks, Thomas > > > > > From thomas.stuefe at gmail.com Thu Nov 15 12:33:24 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 15 Nov 2018 13:33:24 +0100 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <2d1cbb12-e9d6-fcb4-08ad-0a7b0c44a260@oracle.com> <1806b53b-dcdb-1c26-a43e-b82401ccd952@oracle.com> <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: I'm fine with this, Matthias. BTW, I looked and the STEP header line is only printed when secondary exceptions happen inside the step, not always as David assumed (?). So your first form STEP("blabla") if (ExtensiveErrorReports && _verbose) { blub } would have been fine too I think. But I do not want to send you through another review iteration, so I am fine with this version. Thanks for incorporating my remarks. Cheers,Thomas On Thu, Nov 15, 2018 at 12:51 PM Baesken, Matthias wrote: > > Hi Thomas, > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.3/ > > > - got rid of buf+buflen arguments of print_user_info > - removed "Removing" > - adjusted the output a bit in os::Posix::print_user_info() > - vmError.cpp added a short output line with info in case ( ExtensiveErrorReports && _verbose ) is false > > > Best regards, Matthias > > > > -----Original Message----- > > From: Thomas St?fe > > Sent: Mittwoch, 14. November 2018 18:43 > > To: Baesken, Matthias > > Cc: David Holmes ; Langer, Christoph > > ; Volker Simonis ; > > HotSpot Open Source Developers > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > Hi Matthias, > > > > none of my notes from my earlier mail are addressed? I guess my mail > > got lost in all that discussion. Here you go: > > > > + static void print_user_info(outputStream* st, char* buf, size_t buflen); > > > > You can get rid of buf+buflen arguments, not needed anymore > > --- > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > > posix/os_posix.cpp.udiff.html > > > > In os::Posix::print_user_info(), for brevity, I would print it all one > > line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: > > rwxr--r--" > > > > --- > > > > The "removing" text does not make sense and can be removed. > > > > Best Regards, Thomas > > > > > > On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias > > wrote: > > > > > > Hi Thomas , so are you fine with the latest revision > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > of the patch , can I add you as reviewer ? > > > > > > David - is the info added by Thomas sufficient for you? > > > > > > Any other reviewers ? > > > > > > > > > > > > > -----Original Message----- > > > > From: Thomas St?fe > > > > Sent: Samstag, 10. November 2018 14:04 > > > > To: David Holmes > > > > Cc: Baesken, Matthias ; Langer, Christoph > > > > ; Volker Simonis > > ; > > > > HotSpot Open Source Developers > > > > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > > > > > > > Hi David, > > > > > > > > On Sat, Nov 10, 2018 at 12:59 PM David Holmes > > > > > > wrote: > > > > > > > > > > Hi Matthias, > > > > > > > > > > On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > > > > > > Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into > > jdk/jdk > > > > , I created another webrev : > > > > > > > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > > > > > > > > > > > > > > > > > The user info output is now guarded by ExtensiveErrorReports . > > > > > > > > > > + STEP("printing user info") > > > > > + if (ExtensiveErrorReports) { > > > > > + if (_verbose) { > > > > > + os::Posix::print_user_info(st, buf, sizeof(buf)); > > > > > + } > > > > > + } > > > > > > > > > > I don't understand why we explicitly need _verbose if we've asked for > > > > > ExtensiveErrorreports? > > > > > > > > That flag has a different purpose: _verbose distinguishes the two > > > > calls to VMError::report(): the first one to print a small report to > > > > stderr, the second print a large report to the hs-err file. We only > > > > want to print to the hs-err file, so _verbose is still needed. > > > > > > > > > > > > > > Also, ideally the STEP would be inside the guard as otherwise we will > > > > > just print the step description followed by nothing. If the macro > > > > > expansion makes that impossible then we should have an else clause > > that > > > > > prints something like: > > > > > > > > > > - disabled (use -XX:+ExtensiveErrorReports to see these details) > > > > > > > > > > > > > We may even go a step further (but in a separate patch) and make > > > > "ExtensiveErrorReports" a property of the STEP, to be given as > > > > parameter of the STEP macro. > > > > > > > > Thanks, Thomas > > > > > > > From david.holmes at oracle.com Thu Nov 15 12:41:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Nov 2018 22:41:37 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: On 15/11/2018 9:51 pm, Baesken, Matthias wrote: > Hi Thomas, > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.3/ > > > - got rid of buf+buflen arguments of print_user_info > - removed "Removing" > - adjusted the output a bit in os::Posix::print_user_info() > - vmError.cpp added a short output line with info in case ( ExtensiveErrorReports && _verbose ) is false Sorry Matthias I misunderstood how the string in STEP was used. The else part here is not necessary: + if (ExtensiveErrorReports && _verbose) { + os::Posix::print_user_info(st); + } else { + st->print_cr("no extensive user info"); + st->cr(); + } David ----- > > > Best regards, Matthias > > >> -----Original Message----- >> From: Thomas St?fe >> Sent: Mittwoch, 14. November 2018 18:43 >> To: Baesken, Matthias >> Cc: David Holmes ; Langer, Christoph >> ; Volker Simonis ; >> HotSpot Open Source Developers >> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >> >> Hi Matthias, >> >> none of my notes from my earlier mail are addressed? I guess my mail >> got lost in all that discussion. Here you go: >> >> + static void print_user_info(outputStream* st, char* buf, size_t buflen); >> >> You can get rid of buf+buflen arguments, not needed anymore >> --- >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ >> posix/os_posix.cpp.udiff.html >> >> In os::Posix::print_user_info(), for brevity, I would print it all one >> line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: >> rwxr--r--" >> >> --- >> >> The "removing" text does not make sense and can be removed. >> >> Best Regards, Thomas >> >> >> On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias >> wrote: >>> >>> Hi Thomas , so are you fine with the latest revision >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ >>> >>> of the patch , can I add you as reviewer ? >>> >>> David - is the info added by Thomas sufficient for you? >>> >>> Any other reviewers ? >>> >>> >>> >>>> -----Original Message----- >>>> From: Thomas St?fe >>>> Sent: Samstag, 10. November 2018 14:04 >>>> To: David Holmes >>>> Cc: Baesken, Matthias ; Langer, Christoph >>>> ; Volker Simonis >> ; >>>> HotSpot Open Source Developers >>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>>> >>>> Hi David, >>>> >>>> On Sat, Nov 10, 2018 at 12:59 PM David Holmes >> >>>> wrote: >>>>> >>>>> Hi Matthias, >>>>> >>>>> On 9/11/2018 3:13 AM, Baesken, Matthias wrote: >>>>>> Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into >> jdk/jdk >>>> , I created another webrev : >>>>>> >>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ >>>>>> >>>>>> >>>>>> The user info output is now guarded by ExtensiveErrorReports . >>>>> >>>>> + STEP("printing user info") >>>>> + if (ExtensiveErrorReports) { >>>>> + if (_verbose) { >>>>> + os::Posix::print_user_info(st, buf, sizeof(buf)); >>>>> + } >>>>> + } >>>>> >>>>> I don't understand why we explicitly need _verbose if we've asked for >>>>> ExtensiveErrorreports? >>>> >>>> That flag has a different purpose: _verbose distinguishes the two >>>> calls to VMError::report(): the first one to print a small report to >>>> stderr, the second print a large report to the hs-err file. We only >>>> want to print to the hs-err file, so _verbose is still needed. >>>> >>>>> >>>>> Also, ideally the STEP would be inside the guard as otherwise we will >>>>> just print the step description followed by nothing. If the macro >>>>> expansion makes that impossible then we should have an else clause >> that >>>>> prints something like: >>>>> >>>>> - disabled (use -XX:+ExtensiveErrorReports to see these details) >>>>> >>>> >>>> We may even go a step further (but in a separate patch) and make >>>> "ExtensiveErrorReports" a property of the STEP, to be given as >>>> parameter of the STEP macro. >>>> >>>> Thanks, Thomas >>>> >>> From david.holmes at oracle.com Thu Nov 15 12:42:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Nov 2018 22:42:56 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: On 15/11/2018 10:33 pm, Thomas St?fe wrote: > I'm fine with this, Matthias. > > BTW, I looked and the STEP header line is only printed when secondary > exceptions happen inside the step, not always as David assumed (?). So Yes my bad - sorry. > your first form > > STEP("blabla") > if (ExtensiveErrorReports && _verbose) { > blub > } > > would have been fine too I think. But I do not want to send you > through another review iteration, so I am fine with this version. Deleting the else doesn't need another round of review. Thanks, David > Thanks for incorporating my remarks. > > Cheers,Thomas > > > On Thu, Nov 15, 2018 at 12:51 PM Baesken, Matthias > wrote: >> >> Hi Thomas, >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.3/ >> >> >> - got rid of buf+buflen arguments of print_user_info >> - removed "Removing" >> - adjusted the output a bit in os::Posix::print_user_info() >> - vmError.cpp added a short output line with info in case ( ExtensiveErrorReports && _verbose ) is false >> >> >> Best regards, Matthias >> >> >>> -----Original Message----- >>> From: Thomas St?fe >>> Sent: Mittwoch, 14. November 2018 18:43 >>> To: Baesken, Matthias >>> Cc: David Holmes ; Langer, Christoph >>> ; Volker Simonis ; >>> HotSpot Open Source Developers >>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>> >>> Hi Matthias, >>> >>> none of my notes from my earlier mail are addressed? I guess my mail >>> got lost in all that discussion. Here you go: >>> >>> + static void print_user_info(outputStream* st, char* buf, size_t buflen); >>> >>> You can get rid of buf+buflen arguments, not needed anymore >>> --- >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ >>> posix/os_posix.cpp.udiff.html >>> >>> In os::Posix::print_user_info(), for brevity, I would print it all one >>> line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: >>> rwxr--r--" >>> >>> --- >>> >>> The "removing" text does not make sense and can be removed. >>> >>> Best Regards, Thomas >>> >>> >>> On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias >>> wrote: >>>> >>>> Hi Thomas , so are you fine with the latest revision >>>> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ >>>> >>>> of the patch , can I add you as reviewer ? >>>> >>>> David - is the info added by Thomas sufficient for you? >>>> >>>> Any other reviewers ? >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: Thomas St?fe >>>>> Sent: Samstag, 10. November 2018 14:04 >>>>> To: David Holmes >>>>> Cc: Baesken, Matthias ; Langer, Christoph >>>>> ; Volker Simonis >>> ; >>>>> HotSpot Open Source Developers >>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>>>> >>>>> Hi David, >>>>> >>>>> On Sat, Nov 10, 2018 at 12:59 PM David Holmes >>> >>>>> wrote: >>>>>> >>>>>> Hi Matthias, >>>>>> >>>>>> On 9/11/2018 3:13 AM, Baesken, Matthias wrote: >>>>>>> Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into >>> jdk/jdk >>>>> , I created another webrev : >>>>>>> >>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ >>>>>>> >>>>>>> >>>>>>> The user info output is now guarded by ExtensiveErrorReports . >>>>>> >>>>>> + STEP("printing user info") >>>>>> + if (ExtensiveErrorReports) { >>>>>> + if (_verbose) { >>>>>> + os::Posix::print_user_info(st, buf, sizeof(buf)); >>>>>> + } >>>>>> + } >>>>>> >>>>>> I don't understand why we explicitly need _verbose if we've asked for >>>>>> ExtensiveErrorreports? >>>>> >>>>> That flag has a different purpose: _verbose distinguishes the two >>>>> calls to VMError::report(): the first one to print a small report to >>>>> stderr, the second print a large report to the hs-err file. We only >>>>> want to print to the hs-err file, so _verbose is still needed. >>>>> >>>>>> >>>>>> Also, ideally the STEP would be inside the guard as otherwise we will >>>>>> just print the step description followed by nothing. If the macro >>>>>> expansion makes that impossible then we should have an else clause >>> that >>>>>> prints something like: >>>>>> >>>>>> - disabled (use -XX:+ExtensiveErrorReports to see these details) >>>>>> >>>>> >>>>> We may even go a step further (but in a separate patch) and make >>>>> "ExtensiveErrorReports" a property of the STEP, to be given as >>>>> parameter of the STEP macro. >>>>> >>>>> Thanks, Thomas >>>>> >>>> From thomas.stuefe at gmail.com Thu Nov 15 13:15:03 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 15 Nov 2018 14:15:03 +0100 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: On Thu, Nov 15, 2018 at 1:43 PM David Holmes wrote: > > On 15/11/2018 10:33 pm, Thomas St?fe wrote: > > I'm fine with this, Matthias. > > > > BTW, I looked and the STEP header line is only printed when secondary > > exceptions happen inside the step, not always as David assumed (?). So > > Yes my bad - sorry. no problem. I did not see it right away either. > > > your first form > > > > STEP("blabla") > > if (ExtensiveErrorReports && _verbose) { > > blub > > } > > > > would have been fine too I think. But I do not want to send you > > through another review iteration, so I am fine with this version. > > Deleting the else doesn't need another round of review. Same from me. Remove the else path and ship it. ..Thomas > > Thanks, > David > > > > Thanks for incorporating my remarks. > > > > Cheers,Thomas > > > > > > On Thu, Nov 15, 2018 at 12:51 PM Baesken, Matthias > > wrote: > >> > >> Hi Thomas, > >> > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.3/ > >> > >> > >> - got rid of buf+buflen arguments of print_user_info > >> - removed "Removing" > >> - adjusted the output a bit in os::Posix::print_user_info() > >> - vmError.cpp added a short output line with info in case ( ExtensiveErrorReports && _verbose ) is false > >> > >> > >> Best regards, Matthias > >> > >> > >>> -----Original Message----- > >>> From: Thomas St?fe > >>> Sent: Mittwoch, 14. November 2018 18:43 > >>> To: Baesken, Matthias > >>> Cc: David Holmes ; Langer, Christoph > >>> ; Volker Simonis ; > >>> HotSpot Open Source Developers > >>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > >>> > >>> Hi Matthias, > >>> > >>> none of my notes from my earlier mail are addressed? I guess my mail > >>> got lost in all that discussion. Here you go: > >>> > >>> + static void print_user_info(outputStream* st, char* buf, size_t buflen); > >>> > >>> You can get rid of buf+buflen arguments, not needed anymore > >>> --- > >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > >>> posix/os_posix.cpp.udiff.html > >>> > >>> In os::Posix::print_user_info(), for brevity, I would print it all one > >>> line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: > >>> rwxr--r--" > >>> > >>> --- > >>> > >>> The "removing" text does not make sense and can be removed. > >>> > >>> Best Regards, Thomas > >>> > >>> > >>> On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias > >>> wrote: > >>>> > >>>> Hi Thomas , so are you fine with the latest revision > >>>> > >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > >>>> > >>>> of the patch , can I add you as reviewer ? > >>>> > >>>> David - is the info added by Thomas sufficient for you? > >>>> > >>>> Any other reviewers ? > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Thomas St?fe > >>>>> Sent: Samstag, 10. November 2018 14:04 > >>>>> To: David Holmes > >>>>> Cc: Baesken, Matthias ; Langer, Christoph > >>>>> ; Volker Simonis > >>> ; > >>>>> HotSpot Open Source Developers > >>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > >>>>> > >>>>> Hi David, > >>>>> > >>>>> On Sat, Nov 10, 2018 at 12:59 PM David Holmes > >>> > >>>>> wrote: > >>>>>> > >>>>>> Hi Matthias, > >>>>>> > >>>>>> On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > >>>>>>> Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into > >>> jdk/jdk > >>>>> , I created another webrev : > >>>>>>> > >>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > >>>>>>> > >>>>>>> > >>>>>>> The user info output is now guarded by ExtensiveErrorReports . > >>>>>> > >>>>>> + STEP("printing user info") > >>>>>> + if (ExtensiveErrorReports) { > >>>>>> + if (_verbose) { > >>>>>> + os::Posix::print_user_info(st, buf, sizeof(buf)); > >>>>>> + } > >>>>>> + } > >>>>>> > >>>>>> I don't understand why we explicitly need _verbose if we've asked for > >>>>>> ExtensiveErrorreports? > >>>>> > >>>>> That flag has a different purpose: _verbose distinguishes the two > >>>>> calls to VMError::report(): the first one to print a small report to > >>>>> stderr, the second print a large report to the hs-err file. We only > >>>>> want to print to the hs-err file, so _verbose is still needed. > >>>>> > >>>>>> > >>>>>> Also, ideally the STEP would be inside the guard as otherwise we will > >>>>>> just print the step description followed by nothing. If the macro > >>>>>> expansion makes that impossible then we should have an else clause > >>> that > >>>>>> prints something like: > >>>>>> > >>>>>> - disabled (use -XX:+ExtensiveErrorReports to see these details) > >>>>>> > >>>>> > >>>>> We may even go a step further (but in a separate patch) and make > >>>>> "ExtensiveErrorReports" a property of the STEP, to be given as > >>>>> parameter of the STEP macro. > >>>>> > >>>>> Thanks, Thomas > >>>>> > >>>> From matthias.baesken at sap.com Thu Nov 15 13:20:58 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 15 Nov 2018 13:20:58 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: I'll remove the else - but I think I need a second reviewer, isn?t it ? Best regards, Matthias > -----Original Message----- > From: Thomas St?fe > Sent: Donnerstag, 15. November 2018 14:15 > To: David Holmes > Cc: Baesken, Matthias ; Langer, Christoph > ; Volker Simonis ; > HotSpot Open Source Developers > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > On Thu, Nov 15, 2018 at 1:43 PM David Holmes > wrote: > > > > On 15/11/2018 10:33 pm, Thomas St?fe wrote: > > > I'm fine with this, Matthias. > > > > > > BTW, I looked and the STEP header line is only printed when secondary > > > exceptions happen inside the step, not always as David assumed (?). So > > > > Yes my bad - sorry. > > no problem. I did not see it right away either. > > > > > > your first form > > > > > > STEP("blabla") > > > if (ExtensiveErrorReports && _verbose) { > > > blub > > > } > > > > > > would have been fine too I think. But I do not want to send you > > > through another review iteration, so I am fine with this version. > > > > Deleting the else doesn't need another round of review. > > Same from me. Remove the else path and ship it. > > ..Thomas > > > > > Thanks, > > David > > > > > > > Thanks for incorporating my remarks. > > > > > > Cheers,Thomas > > > > > > > > > On Thu, Nov 15, 2018 at 12:51 PM Baesken, Matthias > > > wrote: > > >> > > >> Hi Thomas, > > >> > > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.3/ > > >> > > >> > > >> - got rid of buf+buflen arguments of print_user_info > > >> - removed "Removing" > > >> - adjusted the output a bit in os::Posix::print_user_info() > > >> - vmError.cpp added a short output line with info in case ( > ExtensiveErrorReports && _verbose ) is false > > >> > > >> > > >> Best regards, Matthias > > >> > > >> > > >>> -----Original Message----- > > >>> From: Thomas St?fe > > >>> Sent: Mittwoch, 14. November 2018 18:43 > > >>> To: Baesken, Matthias > > >>> Cc: David Holmes ; Langer, Christoph > > >>> ; Volker Simonis > ; > > >>> HotSpot Open Source Developers > > >>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err > file > > >>> > > >>> Hi Matthias, > > >>> > > >>> none of my notes from my earlier mail are addressed? I guess my mail > > >>> got lost in all that discussion. Here you go: > > >>> > > >>> + static void print_user_info(outputStream* st, char* buf, size_t > buflen); > > >>> > > >>> You can get rid of buf+buflen arguments, not needed anymore > > >>> --- > > >>> > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ > > >>> posix/os_posix.cpp.udiff.html > > >>> > > >>> In os::Posix::print_user_info(), for brevity, I would print it all one > > >>> line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: > > >>> rwxr--r--" > > >>> > > >>> --- > > >>> > > >>> The "removing" text does not make sense and can be removed. > > >>> > > >>> Best Regards, Thomas > > >>> > > >>> > > >>> On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias > > >>> wrote: > > >>>> > > >>>> Hi Thomas , so are you fine with the latest revision > > >>>> > > >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > >>>> > > >>>> of the patch , can I add you as reviewer ? > > >>>> > > >>>> David - is the info added by Thomas sufficient for you? > > >>>> > > >>>> Any other reviewers ? > > >>>> > > >>>> > > >>>> > > >>>>> -----Original Message----- > > >>>>> From: Thomas St?fe > > >>>>> Sent: Samstag, 10. November 2018 14:04 > > >>>>> To: David Holmes > > >>>>> Cc: Baesken, Matthias ; Langer, > Christoph > > >>>>> ; Volker Simonis > > >>> ; > > >>>>> HotSpot Open Source Developers dev at openjdk.java.net> > > >>>>> Subject: Re: RFR : 8211326 : add OS user related information to > hs_err file > > >>>>> > > >>>>> Hi David, > > >>>>> > > >>>>> On Sat, Nov 10, 2018 at 12:59 PM David Holmes > > >>> > > >>>>> wrote: > > >>>>>> > > >>>>>> Hi Matthias, > > >>>>>> > > >>>>>> On 9/11/2018 3:13 AM, Baesken, Matthias wrote: > > >>>>>>> Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into > > >>> jdk/jdk > > >>>>> , I created another webrev : > > >>>>>>> > > >>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ > > >>>>>>> > > >>>>>>> > > >>>>>>> The user info output is now guarded by ExtensiveErrorReports . > > >>>>>> > > >>>>>> + STEP("printing user info") > > >>>>>> + if (ExtensiveErrorReports) { > > >>>>>> + if (_verbose) { > > >>>>>> + os::Posix::print_user_info(st, buf, sizeof(buf)); > > >>>>>> + } > > >>>>>> + } > > >>>>>> > > >>>>>> I don't understand why we explicitly need _verbose if we've asked > for > > >>>>>> ExtensiveErrorreports? > > >>>>> > > >>>>> That flag has a different purpose: _verbose distinguishes the two > > >>>>> calls to VMError::report(): the first one to print a small report to > > >>>>> stderr, the second print a large report to the hs-err file. We only > > >>>>> want to print to the hs-err file, so _verbose is still needed. > > >>>>> > > >>>>>> > > >>>>>> Also, ideally the STEP would be inside the guard as otherwise we > will > > >>>>>> just print the step description followed by nothing. If the macro > > >>>>>> expansion makes that impossible then we should have an else > clause > > >>> that > > >>>>>> prints something like: > > >>>>>> > > >>>>>> - disabled (use -XX:+ExtensiveErrorReports to see these details) > > >>>>>> > > >>>>> > > >>>>> We may even go a step further (but in a separate patch) and make > > >>>>> "ExtensiveErrorReports" a property of the STEP, to be given as > > >>>>> parameter of the STEP macro. > > >>>>> > > >>>>> Thanks, Thomas > > >>>>> > > >>>> From david.holmes at oracle.com Thu Nov 15 13:38:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 15 Nov 2018 23:38:12 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> Message-ID: <5ddb8e00-37e6-ee29-aa51-b0241e36d5ab@oracle.com> On 15/11/2018 11:20 pm, Baesken, Matthias wrote: > I'll remove the else - but I think I need a second reviewer, isn?t it ? I hadn't reviewed the actual printing of the user info but now I have had a good look at it. My only quibble is about the treatment of uid_t and gid_t as being the same. It would be marginally cleaner to me if the cast to unsigned was done upfront on the library call: unsigned id = (unsigned) ::getuid(); st->print("uid : %u ", id); id = (unsigned) ::geteuid(); ... id = (unsigned) ::getgid(); ... Thanks, David ----- > Best regards, Matthias > >> -----Original Message----- >> From: Thomas St?fe >> Sent: Donnerstag, 15. November 2018 14:15 >> To: David Holmes >> Cc: Baesken, Matthias ; Langer, Christoph >> ; Volker Simonis ; >> HotSpot Open Source Developers >> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >> >> On Thu, Nov 15, 2018 at 1:43 PM David Holmes >> wrote: >>> >>> On 15/11/2018 10:33 pm, Thomas St?fe wrote: >>>> I'm fine with this, Matthias. >>>> >>>> BTW, I looked and the STEP header line is only printed when secondary >>>> exceptions happen inside the step, not always as David assumed (?). So >>> >>> Yes my bad - sorry. >> >> no problem. I did not see it right away either. >> >>> >>>> your first form >>>> >>>> STEP("blabla") >>>> if (ExtensiveErrorReports && _verbose) { >>>> blub >>>> } >>>> >>>> would have been fine too I think. But I do not want to send you >>>> through another review iteration, so I am fine with this version. >>> >>> Deleting the else doesn't need another round of review. >> >> Same from me. Remove the else path and ship it. >> >> ..Thomas >> >>> >>> Thanks, >>> David >>> >>> >>>> Thanks for incorporating my remarks. >>>> >>>> Cheers,Thomas >>>> >>>> >>>> On Thu, Nov 15, 2018 at 12:51 PM Baesken, Matthias >>>> wrote: >>>>> >>>>> Hi Thomas, >>>>> >>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.3/ >>>>> >>>>> >>>>> - got rid of buf+buflen arguments of print_user_info >>>>> - removed "Removing" >>>>> - adjusted the output a bit in os::Posix::print_user_info() >>>>> - vmError.cpp added a short output line with info in case ( >> ExtensiveErrorReports && _verbose ) is false >>>>> >>>>> >>>>> Best regards, Matthias >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Thomas St?fe >>>>>> Sent: Mittwoch, 14. November 2018 18:43 >>>>>> To: Baesken, Matthias >>>>>> Cc: David Holmes ; Langer, Christoph >>>>>> ; Volker Simonis >> ; >>>>>> HotSpot Open Source Developers >>>>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err >> file >>>>>> >>>>>> Hi Matthias, >>>>>> >>>>>> none of my notes from my earlier mail are addressed? I guess my mail >>>>>> got lost in all that discussion. Here you go: >>>>>> >>>>>> + static void print_user_info(outputStream* st, char* buf, size_t >> buflen); >>>>>> >>>>>> You can get rid of buf+buflen arguments, not needed anymore >>>>>> --- >>>>>> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.2/src/hotspot/os/ >>>>>> posix/os_posix.cpp.udiff.html >>>>>> >>>>>> In os::Posix::print_user_info(), for brevity, I would print it all one >>>>>> line like this: "uid: 4711, euid: 0, euid: 4711, egid: 4711, umask: >>>>>> rwxr--r--" >>>>>> >>>>>> --- >>>>>> >>>>>> The "removing" text does not make sense and can be removed. >>>>>> >>>>>> Best Regards, Thomas >>>>>> >>>>>> >>>>>> On Wed, Nov 14, 2018 at 6:16 PM Baesken, Matthias >>>>>> wrote: >>>>>>> >>>>>>> Hi Thomas , so are you fine with the latest revision >>>>>>> >>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ >>>>>>> >>>>>>> of the patch , can I add you as reviewer ? >>>>>>> >>>>>>> David - is the info added by Thomas sufficient for you? >>>>>>> >>>>>>> Any other reviewers ? >>>>>>> >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Thomas St?fe >>>>>>>> Sent: Samstag, 10. November 2018 14:04 >>>>>>>> To: David Holmes >>>>>>>> Cc: Baesken, Matthias ; Langer, >> Christoph >>>>>>>> ; Volker Simonis >>>>>> ; >>>>>>>> HotSpot Open Source Developers > dev at openjdk.java.net> >>>>>>>> Subject: Re: RFR : 8211326 : add OS user related information to >> hs_err file >>>>>>>> >>>>>>>> Hi David, >>>>>>>> >>>>>>>> On Sat, Nov 10, 2018 at 12:59 PM David Holmes >>>>>> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi Matthias, >>>>>>>>> >>>>>>>>> On 9/11/2018 3:13 AM, Baesken, Matthias wrote: >>>>>>>>>> Hello , after the flag "-XX:+-ExtensiveErrorReports" make it into >>>>>> jdk/jdk >>>>>>>> , I created another webrev : >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.1/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The user info output is now guarded by ExtensiveErrorReports . >>>>>>>>> >>>>>>>>> + STEP("printing user info") >>>>>>>>> + if (ExtensiveErrorReports) { >>>>>>>>> + if (_verbose) { >>>>>>>>> + os::Posix::print_user_info(st, buf, sizeof(buf)); >>>>>>>>> + } >>>>>>>>> + } >>>>>>>>> >>>>>>>>> I don't understand why we explicitly need _verbose if we've asked >> for >>>>>>>>> ExtensiveErrorreports? >>>>>>>> >>>>>>>> That flag has a different purpose: _verbose distinguishes the two >>>>>>>> calls to VMError::report(): the first one to print a small report to >>>>>>>> stderr, the second print a large report to the hs-err file. We only >>>>>>>> want to print to the hs-err file, so _verbose is still needed. >>>>>>>> >>>>>>>>> >>>>>>>>> Also, ideally the STEP would be inside the guard as otherwise we >> will >>>>>>>>> just print the step description followed by nothing. If the macro >>>>>>>>> expansion makes that impossible then we should have an else >> clause >>>>>> that >>>>>>>>> prints something like: >>>>>>>>> >>>>>>>>> - disabled (use -XX:+ExtensiveErrorReports to see these details) >>>>>>>>> >>>>>>>> >>>>>>>> We may even go a step further (but in a separate patch) and make >>>>>>>> "ExtensiveErrorReports" a property of the STEP, to be given as >>>>>>>> parameter of the STEP macro. >>>>>>>> >>>>>>>> Thanks, Thomas >>>>>>>> >>>>>>> From gg5070 at gmail.com Thu Nov 15 10:42:37 2018 From: gg5070 at gmail.com (Gidon Gershinsky) Date: Thu, 15 Nov 2018 12:42:37 +0200 Subject: Problems with AES-GCM native acceleration In-Reply-To: <864a763d-0d67-d894-234c-fbbef78e0320@oracle.com> References: <92a16009b9e047eacf102b006bc0612bac4a2cfb.camel@redhat.com> <1b1fe9e4-ad71-7d48-13eb-48dbe267962e@oracle.com> <864a763d-0d67-d894-234c-fbbef78e0320@oracle.com> Message-ID: Hi all, Thanks for the prompt feedback on this stuff, appreciated. 1. Analytic queries are often interactive or one-off. A data scientist would get an on-demand notebook with a Spark cluster (spawned as a K8s pod), and run a number of queries. The cluster will be then closed either explicitly, or after a timeout. This is done both for a better resource utilization, and for security reasons. Re-using JVM for another user/tenant might leak the sensitive data and encryption keys, kept in the JVM memory. I'm not saying its the only way to solve this, there are architectures based on a long running service. But this short-lived approach is real and needs to be addressed. Even if the data scientist keeps the cluster alive for a few hours - having to wait a long time for the results of the first few queries (because the decryption is not warmed up yet) is a problem, since the things are interactive and expected to be done in real time. 2. Analytics and AI workloads work with ~ 64MB blocks; sometimes, they are broken in ~1MB pieces (like in Parquet). Still, taking even the minimal size of 1MB, and waiting the 10,000 rounds to get the decryption acceleration, means we process the first ~10GB at a slow rate. Sounds harsh. Both in absolute numbers, and in comparison to ENcryption, which kicks in after warming up with say 1KB chunks (created by breaking 1MB blocks into many update calls) - meaning ~1,000x faster than DEcryption. 3. Adam has mentioned an approach of "modifying the decryption operation (to decrypt immediately and buffer plaintext)" (in a negative context, though :). To me, it looks like a sound solution. However, I don't know how much effort does it require (?) - but it makes decryption implementation similar to encryption, and solves the problem at hand. Maybe there are other options, though. 4. AOT sounds interesting, I'll check it out. But its experimental for now. Moreover, both AOT and command line options require extra care in production, as correctly pointed out below. They will be a hard sell in real production environments. The same is true (or even worse) for manual warm-up with a repeated decryption of small blocks. This is indeed a benchmarking hack, I don't see it been used in production. Having the decryption optimized in the HotSpot engine would be ideal. Cheers, Gidon. On Thu, Nov 15, 2018 at 3:33 AM Anthony Scarpino < anthony.scarpino at oracle.com> wrote: > I agree with Adam that this is more of a tuning issue and not a problem > with the crypto. Sending multiple updates is a hack. > > I've been aware of this bug for a while and I do not understand why this > is a significant problem. The stackoverflow comments say it takes 50 > seconds to trigger the intrinsic. If this is a long running server > application slowness for the first 50 seconds is trivial. For smaller > operations, those are commonly small transactions, not decrypting a 3GB > file. > > If it cannot be resolved by commandline options and this is occurring in > a real world situation, please explain it fully. If this is only for > benchmarking, then that's not a real world situation. > > Tony > > On 11/14/18 8:41 AM, Adam Petcher wrote: > > I'm adding back in hotspot-dev, because this is a somewhat tricky topic > > related to intrinsics and JIT. Hopefully, a Hotspot expert can correct > > anything that I say below that is wrong, and suggest any solutions that > > I missed. > > > > The AES acceleration is implemented in a HotSpot intrinsic. In order for > > it to kick in, the code must be JIT compiled by the VM. As I understand > > it, this only happens to some particular method after it has been called > > a certain number of times. The rules that determine this number are > > somewhat complicated, but I think you can guarantee JIT in the default > > configuration by calling a method 10,000 times. > > > > The doFinal method calls the update method, so either one should trigger > > the acceleration as long as you call it enough. Breaking the message up > > into smaller chunks and calling update on each one works only because it > > ends up calling the update method more. You should be able to trigger > > the acceleration by calling doFinal more, too. > > > > The reason why the workaround doesn't work with decryption is that the > > decryption routine buffers the ciphertext and then decrypts it all at > > the end. So calling update multiple times and then calling doFinal at > > the end is essentially the same as calling doFinal once with the entire > > ciphertext. > > > > So here are some solutions that you may want to try: > > > > 1) In your benchmark, run at least 10,000 "warmup" iterations of > > whatever you are trying to do at the beginning, without timing it. This > > is a good idea for benchmarks, anyway. If it helps, you can try using > > smaller buffers in your "warmup" phase in order to get it to complete > > faster. > > > > 2) Try -XX:CompileThreshold=(some number smaller than 10000) as an > > argument to java. This will make JIT kick in sooner across the board. > > Obviously, this should be done carefully in production, since it will > > impact the performance of the entire program. > > > > 3) I haven't tried this, but running with an AOTed java.base module may > > also help. See the section titled "Steps to generate and use an AOT > > library for the java.base module" in the AOT JEP[1]. > > > > "Fixing" this issue in the JDK is non-trivial, because it gets into the > > behavior of the VM and JIT. I don't really like the idea of modifying > > doFinal (to break up the operation into multiple update calls) or > > modifying the decryption operation (to decrypt immediately and buffer > > plaintext) in order to work around this issue. Perhaps there is a better > > way for the VM to handle cases like this, in which a method is not > > called often, but the interpreted execution takes a long time to > > complete when it is called. Perhaps a VM expert will have some > > additional thoughts here. > > > > [1] https://openjdk.java.net/jeps/295 > > > > On 11/14/2018 9:49 AM, Severin Gehwolf wrote: > >> Dropping hotspot-dev and adding security-dev. > >> > >> On Wed, 2018-11-14 at 14:39 +0200, Gidon Gershinsky wrote: > >>> Hi, > >>> > >>> We are working on an encryption mechanism at the Apache Parquet - > >>> that will enable efficient analytics on encrypted data by frameworks > >>> such as Apache Spark. > >>> https://github.com/apache/parquet-format/blob/encryption/Encryption.md > >>> > https://www.slideshare.net/databricks/efficient-spark-analytics-on-encrypted-data-with-gidon-gershinsky > >>> > >>> > >>> We came across an AES-related issue in the Java HostSpot engine that > >>> looks like a substantial problem for us in both Spark and Parquet > >>> workloads. The bug report had been accepted a while ago: > >>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8201633 > >>> > >>> The fix should hopefully be rather straightforward though. > >>> Could you help us with that? I have a couple of small samples > >>> reproducing the problem. > >>> > >>> (If I'm writing to a wrong mailing list - I apologize, please point > >>> me in the right direction). > >>> > >>> Cheers, Gidon. > > From matthias.baesken at sap.com Thu Nov 15 16:26:13 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 15 Nov 2018 16:26:13 +0000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: <5ddb8e00-37e6-ee29-aa51-b0241e36d5ab@oracle.com> References: <6bc52eda-1ab5-e412-5eb0-f3e74b603d77@oracle.com> <5ddb8e00-37e6-ee29-aa51-b0241e36d5ab@oracle.com> Message-ID: Hi Thomas and David, new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.4/ This includes David's comment > My only quibble is about the treatment of uid_t > and gid_t as being the same. It would be marginally cleaner to me if the > cast to unsigned was done upfront on the library call: > and adjusts the STEP-coding (removed else ... part ) . David - can I add you as reviewer (Thomas was fine with the change already) ? Thanks, Matthias > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 15. November 2018 14:38 > To: Baesken, Matthias ; Thomas St?fe > > Cc: Langer, Christoph ; Volker Simonis > ; HotSpot Open Source Developers dev at openjdk.java.net> > Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > > On 15/11/2018 11:20 pm, Baesken, Matthias wrote: > > I'll remove the else - but I think I need a second reviewer, isn?t it ? > > I hadn't reviewed the actual printing of the user info but now I have > had a good look at it. My only quibble is about the treatment of uid_t > and gid_t as being the same. It would be marginally cleaner to me if the > cast to unsigned was done upfront on the library call: > > unsigned id = (unsigned) ::getuid(); > st->print("uid : %u ", id); > id = (unsigned) ::geteuid(); > ... > id = (unsigned) ::getgid(); > ... > > Thanks, > David > ----- > > > Best regards, Matthias > > > >> -----Original Message----- > >> From: Thomas St?fe > >> Sent: Donnerstag, 15. November 2018 14:15 > >> To: David Holmes > >> Cc: Baesken, Matthias ; Langer, Christoph > >> ; Volker Simonis > ; > >> HotSpot Open Source Developers > >> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file > >> > >> On Thu, Nov 15, 2018 at 1:43 PM David Holmes > > >> wrote: > >>> > >>> On 15/11/2018 10:33 pm, Thomas St?fe wrote: > >>>> I'm fine with this, Matthias. > >>>> > >>>> BTW, I looked and the STEP header line is only printed when secondary > >>>> exceptions happen inside the step, not always as David assumed (?). So > >>> > >>> Yes my bad - sorry. > >> > >> no problem. I did not see it right away either. > >> > >>> > >>>> your first form > >>>> > >>>> STEP("blabla") > >>>> if (ExtensiveErrorReports && _verbose) { > >>>> blub > >>>> } > >>>> > >>>> would have been fine too I think. But I do not want to send you > >>>> through another review iteration, so I am fine with this version. > >>> > >>> Deleting the else doesn't need another round of review. > >> > >> Same from me. Remove the else path and ship it. > >> > >> ..Thomas > >> From sgehwolf at redhat.com Thu Nov 15 19:09:38 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 15 Nov 2018 20:09:38 +0100 Subject: [11u] RFR: 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader Message-ID: Hi, Please review this 11u backport for JDK-8212937. While the JDK 12 patch applies cleanly it doesn't quite do the right thing. The JDK 12 test would still crash the JVM due to the lack of the changes in src/hotspot/share/classfile/javaClasses.cpp as describe_external is being used in JDK 11. That function has been removed with JDK- 8205611[1] in JDK 12. In addition, the exception messages aren't the same between JDK 12 and JDK 11, so Test.java needed an update to account for that. Also due to lack of JDK-8205611[1] in JDK 11. Everything else is the same as for JDK 12. Bug: https://bugs.openjdk.java.net/browse/JDK-8212937 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8212937/jdk11/webrev.01/ Testing: Added regression test, internal test. Thoughts? Thanks, Severin [1] https://bugs.openjdk.java.net/browse/JDK-8205611 From coleen.phillimore at oracle.com Thu Nov 15 22:50:30 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 15 Nov 2018 17:50:30 -0500 Subject: RFR 8213751: ClassLoaderDataGraph::cld_do() should sometimes require CLDG_lock In-Reply-To: <4c0d62ed-da1c-f07e-0b37-beb14c80323c@oracle.com> References: <4c0d62ed-da1c-f07e-0b37-beb14c80323c@oracle.com> Message-ID: <3f553075-406c-8ec2-59e8-ec68aac3c520@oracle.com> ErikO and I had some offline conversation about the function naming, and decided on cld_do() to follow all CLDs and loaded_cld_do() to only follow the loaded CLDS, so I've reverted some of the GC cld_oops_do() changes.? Also always_strong_loaded_cld_do() doesn't make sense, so I removed that. Incremental (somewhat of a revert) http://cr.openjdk.java.net/~coleenp/8213751.02.incr/webrev/index.html Full: http://cr.openjdk.java.net/~coleenp/8213751.02/webrev/index.html Reran serviceability tests and JFR tests. Thanks, Coleen On 11/14/18 7:44 AM, coleen.phillimore at oracle.com wrote: > Summary: Give GC own version of cld_oops_do and use cld_do for runtime > calls. > > This also adds a NoSafepointVerifier in the CLDG iterator. > > Tested with hs-tier1-3, vmTestbase/nsk, gc, runtime and JFR tests > locally. > > open webrev at http://cr.openjdk.java.net/~coleenp/8213751.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8213751 > > Thanks, > Coleen From david.holmes at oracle.com Thu Nov 15 23:32:16 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 16 Nov 2018 09:32:16 +1000 Subject: RFR : 8211326 : add OS user related information to hs_err file In-Reply-To: References: <5ddb8e00-37e6-ee29-aa51-b0241e36d5ab@oracle.com> Message-ID: <1a906ebb-2b65-1718-dbf4-e8676afce9e9@oracle.com> Reviewed! Thanks for your patience Matthias! David On 16/11/2018 2:26 am, Baesken, Matthias wrote: > Hi Thomas and David, > > new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.4/ > > This includes David's comment > >> My only quibble is about the treatment of uid_t >> and gid_t as being the same. It would be marginally cleaner to me if the >> cast to unsigned was done upfront on the library call: >> > > and adjusts the STEP-coding (removed else ... part ) . > > David - can I add you as reviewer (Thomas was fine with the change already) ? > > Thanks, Matthias > > >> -----Original Message----- >> From: David Holmes >> Sent: Donnerstag, 15. November 2018 14:38 >> To: Baesken, Matthias ; Thomas St?fe >> >> Cc: Langer, Christoph ; Volker Simonis >> ; HotSpot Open Source Developers > dev at openjdk.java.net> >> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >> >> On 15/11/2018 11:20 pm, Baesken, Matthias wrote: >>> I'll remove the else - but I think I need a second reviewer, isn?t it ? >> >> I hadn't reviewed the actual printing of the user info but now I have >> had a good look at it. My only quibble is about the treatment of uid_t >> and gid_t as being the same. It would be marginally cleaner to me if the >> cast to unsigned was done upfront on the library call: >> >> unsigned id = (unsigned) ::getuid(); >> st->print("uid : %u ", id); >> id = (unsigned) ::geteuid(); >> ... >> id = (unsigned) ::getgid(); >> ... >> >> Thanks, >> David >> ----- >> >>> Best regards, Matthias >>> >>>> -----Original Message----- >>>> From: Thomas St?fe >>>> Sent: Donnerstag, 15. November 2018 14:15 >>>> To: David Holmes >>>> Cc: Baesken, Matthias ; Langer, Christoph >>>> ; Volker Simonis >> ; >>>> HotSpot Open Source Developers >>>> Subject: Re: RFR : 8211326 : add OS user related information to hs_err file >>>> >>>> On Thu, Nov 15, 2018 at 1:43 PM David Holmes >> >>>> wrote: >>>>> >>>>> On 15/11/2018 10:33 pm, Thomas St?fe wrote: >>>>>> I'm fine with this, Matthias. >>>>>> >>>>>> BTW, I looked and the STEP header line is only printed when secondary >>>>>> exceptions happen inside the step, not always as David assumed (?). So >>>>> >>>>> Yes my bad - sorry. >>>> >>>> no problem. I did not see it right away either. >>>> >>>>> >>>>>> your first form >>>>>> >>>>>> STEP("blabla") >>>>>> if (ExtensiveErrorReports && _verbose) { >>>>>> blub >>>>>> } >>>>>> >>>>>> would have been fine too I think. But I do not want to send you >>>>>> through another review iteration, so I am fine with this version. >>>>> >>>>> Deleting the else doesn't need another round of review. >>>> >>>> Same from me. Remove the else path and ship it. >>>> >>>> ..Thomas >>>> > From kim.barrett at oracle.com Fri Nov 16 02:31:45 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 15 Nov 2018 21:31:45 -0500 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: > On Oct 3, 2018, at 3:13 PM, Kim Barrett wrote: > > I've submitted a JEP for > > (1) enabling the use of C++14 Language Features when building the JDK, > > (2) define a process for deciding and documenting which new features > can be used or are forbidden in HotSpot code, > > (3) provide an initial list of permitted and forbidden new features. > > https://bugs.openjdk.java.net/browse/JDK-8208089 While there has been some discussion of this JEP here, so far only Mikael has officially reviewed or endorsed it. Regarding the timing of this JEP, integration in time for JDK 12 seems unlikely. From erik.osterlund at oracle.com Fri Nov 16 06:48:34 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 16 Nov 2018 07:48:34 +0100 Subject: RFR 8213751: ClassLoaderDataGraph::cld_do() should sometimes require CLDG_lock In-Reply-To: <3f553075-406c-8ec2-59e8-ec68aac3c520@oracle.com> References: <4c0d62ed-da1c-f07e-0b37-beb14c80323c@oracle.com> <3f553075-406c-8ec2-59e8-ec68aac3c520@oracle.com> Message-ID: <5BEE6842.30003@oracle.com> Hi Coleen, Looks good. Thanks, /Erik On 2018-11-15 23:50, coleen.phillimore at oracle.com wrote: > > ErikO and I had some offline conversation about the function naming, > and decided on cld_do() to follow all CLDs and loaded_cld_do() to only > follow the loaded CLDS, so I've reverted some of the GC cld_oops_do() > changes. Also always_strong_loaded_cld_do() doesn't make sense, so I > removed that. > > Incremental (somewhat of a revert) > http://cr.openjdk.java.net/~coleenp/8213751.02.incr/webrev/index.html > Full: http://cr.openjdk.java.net/~coleenp/8213751.02/webrev/index.html > > Reran serviceability tests and JFR tests. > > Thanks, > Coleen > > On 11/14/18 7:44 AM, coleen.phillimore at oracle.com wrote: >> Summary: Give GC own version of cld_oops_do and use cld_do for >> runtime calls. >> >> This also adds a NoSafepointVerifier in the CLDG iterator. >> >> Tested with hs-tier1-3, vmTestbase/nsk, gc, runtime and JFR tests >> locally. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8213751.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8213751 >> >> Thanks, >> Coleen > From robbin.ehn at oracle.com Fri Nov 16 08:59:48 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 16 Nov 2018 09:59:48 +0100 Subject: RFR 8213751: ClassLoaderDataGraph::cld_do() should sometimes require CLDG_lock In-Reply-To: <5BEE6842.30003@oracle.com> References: <4c0d62ed-da1c-f07e-0b37-beb14c80323c@oracle.com> <3f553075-406c-8ec2-59e8-ec68aac3c520@oracle.com> <5BEE6842.30003@oracle.com> Message-ID: On 11/16/18 7:48 AM, Erik ?sterlund wrote: > Hi Coleen, > > Looks good. +1 /Robbin > > Thanks, > /Erik > > On 2018-11-15 23:50, coleen.phillimore at oracle.com wrote: >> >> ErikO and I had some offline conversation about the function naming, and >> decided on cld_do() to follow all CLDs and loaded_cld_do() to only follow the >> loaded CLDS, so I've reverted some of the GC cld_oops_do() changes.? Also >> always_strong_loaded_cld_do() doesn't make sense, so I removed that. >> >> Incremental (somewhat of a revert) >> http://cr.openjdk.java.net/~coleenp/8213751.02.incr/webrev/index.html >> Full: http://cr.openjdk.java.net/~coleenp/8213751.02/webrev/index.html >> >> Reran serviceability tests and JFR tests. >> >> Thanks, >> Coleen >> >> On 11/14/18 7:44 AM, coleen.phillimore at oracle.com wrote: >>> Summary: Give GC own version of cld_oops_do and use cld_do for runtime calls. >>> >>> This also adds a NoSafepointVerifier in the CLDG iterator. >>> >>> Tested with hs-tier1-3, vmTestbase/nsk, gc, runtime and JFR tests locally. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8213751.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8213751 >>> >>> Thanks, >>> Coleen >> > From coleen.phillimore at oracle.com Fri Nov 16 11:42:02 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 16 Nov 2018 06:42:02 -0500 Subject: RFR 8213751: ClassLoaderDataGraph::cld_do() should sometimes require CLDG_lock In-Reply-To: References: <4c0d62ed-da1c-f07e-0b37-beb14c80323c@oracle.com> <3f553075-406c-8ec2-59e8-ec68aac3c520@oracle.com> <5BEE6842.30003@oracle.com> Message-ID: <12c9f123-8872-4861-dd5b-ea7728679869@oracle.com> Thanks Erik and Robbin! Coleen On 11/16/18 3:59 AM, Robbin Ehn wrote: > On 11/16/18 7:48 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Looks good. > > +1 > > /Robbin > >> >> Thanks, >> /Erik >> >> On 2018-11-15 23:50, coleen.phillimore at oracle.com wrote: >>> >>> ErikO and I had some offline conversation about the function naming, >>> and decided on cld_do() to follow all CLDs and loaded_cld_do() to >>> only follow the loaded CLDS, so I've reverted some of the GC >>> cld_oops_do() changes.? Also always_strong_loaded_cld_do() doesn't >>> make sense, so I removed that. >>> >>> Incremental (somewhat of a revert) >>> http://cr.openjdk.java.net/~coleenp/8213751.02.incr/webrev/index.html >>> Full: http://cr.openjdk.java.net/~coleenp/8213751.02/webrev/index.html >>> >>> Reran serviceability tests and JFR tests. >>> >>> Thanks, >>> Coleen >>> >>> On 11/14/18 7:44 AM, coleen.phillimore at oracle.com wrote: >>>> Summary: Give GC own version of cld_oops_do and use cld_do for >>>> runtime calls. >>>> >>>> This also adds a NoSafepointVerifier in the CLDG iterator. >>>> >>>> Tested with hs-tier1-3, vmTestbase/nsk, gc, runtime and JFR tests >>>> locally. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8213751.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8213751 >>>> >>>> Thanks, >>>> Coleen >>> >> From bsrbnd at gmail.com Fri Nov 16 14:23:25 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 16 Nov 2018 15:23:25 +0100 Subject: Comments about JDK-8213473: Replace testB_mem_imm matcher with testUB_mem_imm Message-ID: Hi, The current fix [1] for JDK-8213473 [2] replaces 'testB_mem_imm' with 'testUB_mem_imm' in 'x86_64.ad' which is fine for unsigned bytes, for example: if ((flags & 0x80) != 0) ... But per JLS ?5.6.2 & ?5.1.2, if we have something like [3]: if ((flags & (byte)0x80) != 0) ... the mask is 32-bit sign-extended and 'LoadB' isn't converted to 'LoadUB' in 'AndINode::Ideal' [4] which is fine. Unfortunately 'testUB_mem_imm' won't be matched in this case, so I guess we'd need both 'testB_mem_imm' & 'testUB_mem_imm' patterns. If we all agree, I can create a JBS issue and send out a RFR for this? Thanks, Bernard [1] http://hg.openjdk.java.net/jdk/jdk/rev/656d2f222b42 [2] https://bugs.openjdk.java.net/browse/JDK-8213473 [3] http://hg.openjdk.java.net/jdk/jdk/file/74109912c738/src/jdk.charsets/share/classes/sun/nio/cs/ext/ISO2022_CN.java#l165 [4] http://hg.openjdk.java.net/jdk/jdk/file/74109912c738/src/hotspot/share/opto/mulnode.cpp#l493 From rkennke at redhat.com Fri Nov 16 14:33:18 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 16 Nov 2018 15:33:18 +0100 Subject: Comments about JDK-8213473: Replace testB_mem_imm matcher with testUB_mem_imm In-Reply-To: References: Message-ID: <92e9e7b5-1566-ae77-5561-ae05a489c021@redhat.com> Hi Bernard, Right... So either we reinstate the testB pattern, or we make the conversion check for sign-extended mask too? Not sure if the latter would be 100% correct though. Roman > Hi, > > The current fix [1] for JDK-8213473 [2] replaces 'testB_mem_imm' with > 'testUB_mem_imm' in 'x86_64.ad' which is fine for unsigned bytes, for > example: > if ((flags & 0x80) != 0) ... > > But per JLS ?5.6.2 & ?5.1.2, if we have something like [3]: > if ((flags & (byte)0x80) != 0) ... > > the mask is 32-bit sign-extended and 'LoadB' isn't converted to > 'LoadUB' in 'AndINode::Ideal' [4] which is fine. > > Unfortunately 'testUB_mem_imm' won't be matched in this case, so I > guess we'd need both 'testB_mem_imm' & 'testUB_mem_imm' patterns. > > If we all agree, I can create a JBS issue and send out a RFR for this? > > Thanks, > Bernard > > [1] http://hg.openjdk.java.net/jdk/jdk/rev/656d2f222b42 > [2] https://bugs.openjdk.java.net/browse/JDK-8213473 > [3] http://hg.openjdk.java.net/jdk/jdk/file/74109912c738/src/jdk.charsets/share/classes/sun/nio/cs/ext/ISO2022_CN.java#l165 > [4] http://hg.openjdk.java.net/jdk/jdk/file/74109912c738/src/hotspot/share/opto/mulnode.cpp#l493 > From sgehwolf at redhat.com Fri Nov 16 15:09:01 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 16 Nov 2018 16:09:01 +0100 Subject: [11u] RFR: 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: On Thu, 2018-11-15 at 20:09 +0100, Severin Gehwolf wrote: > Hi, > > Please review this 11u backport for JDK-8212937. While the JDK 12 patch > applies cleanly it doesn't quite do the right thing. The JDK 12 test > would still crash the JVM due to the lack of the changes in > src/hotspot/share/classfile/javaClasses.cpp as describe_external is > being used in JDK 11. That function has been removed with JDK- > 8205611[1] in JDK 12. > > In addition, the exception messages aren't the same between JDK 12 and > JDK 11, so Test.java needed an update to account for that. Also due to > lack of JDK-8205611[1] in JDK 11. Everything else is the same as for > JDK 12. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212937 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8212937/jdk11/webrev.01/ > > Testing: Added regression test, internal test. Also: tier1 testing showed no regressions with this on linux x86_64. Thanks, Severin > Thoughts? > > Thanks, > Severin > > [1] https://bugs.openjdk.java.net/browse/JDK-8205611 From lois.foltan at oracle.com Fri Nov 16 15:11:17 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 16 Nov 2018 10:11:17 -0500 Subject: [11u] RFR: 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: References: Message-ID: <5cd90d21-5fa8-4289-c242-5cca6f9ead8e@oracle.com> On 11/16/2018 10:09 AM, Severin Gehwolf wrote: > On Thu, 2018-11-15 at 20:09 +0100, Severin Gehwolf wrote: >> Hi, >> >> Please review this 11u backport for JDK-8212937. While the JDK 12 patch >> applies cleanly it doesn't quite do the right thing. The JDK 12 test >> would still crash the JVM due to the lack of the changes in >> src/hotspot/share/classfile/javaClasses.cpp as describe_external is >> being used in JDK 11. That function has been removed with JDK- >> 8205611[1] in JDK 12. >> >> In addition, the exception messages aren't the same between JDK 12 and >> JDK 11, so Test.java needed an update to account for that. Also due to >> lack of JDK-8205611[1] in JDK 11. Everything else is the same as for >> JDK 12. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8212937 >> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8212937/jdk11/webrev.01/ >> >> Testing: Added regression test, internal test. > Also: > > tier1 testing showed no regressions with this on linux x86_64. Hi Severin, Thank you for doing this testing.? I think your change looks good! Thanks, Lois > > Thanks, > Severin > >> Thoughts? >> >> Thanks, >> Severin >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8205611 From shade at redhat.com Fri Nov 16 16:18:29 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 16 Nov 2018 17:18:29 +0100 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option Message-ID: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8213992 While the original option was apparently conceived to aid debugging during the VM development, it is also useful in product builds to terminate VM when it is stuck on safepoint sync. This is sometimes asked by external people who have fail-overs set up to deal with VM crashes: making VM fail predictably on errors like this is a plus for them. David suggested making the option "product", but I don't like it very much: "product" option suggests this is a production-grade feature and it comes with expectations of support, which is interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode available in product builds without the attached notion of support. Users that want fail-fast VM crash can then use that option on "we know what we are doing" basis. The name of the option is hopefully aligned with: diagnostic AbortVMOnException (existing) diagnostic AbortVMOnSafepointTimeout (this one) diagnostic AbortVMOnVMOperationTimeout (JDK-8181143) diagnostic AbortVMOnCompilationFailure (JDK-8212070) Fix: diff -r 9ad663e63da5 -r 132db6e99f77 src/hotspot/share/runtime/globals.hpp --- a/src/hotspot/share/runtime/globals.hpp Fri Nov 16 12:02:08 2018 +0100 +++ b/src/hotspot/share/runtime/globals.hpp Fri Nov 16 13:35:16 2018 +0100 @@ -498,7 +498,7 @@ "Time out and warn or fail after SafepointTimeoutDelay " \ "milliseconds if failed to reach safepoint") \ \ - develop(bool, DieOnSafepointTimeout, false, \ + diagnostic(bool, AbortVMOnSafepointTimeout, false, \ "Die upon failure to reach safepoint (see SafepointTimeout)") \ \ /* 50 retries * (5 * current_retry_count) millis = ~6.375 seconds */ \ diff -r 9ad663e63da5 -r 132db6e99f77 src/hotspot/share/runtime/safepoint.cpp --- a/src/hotspot/share/runtime/safepoint.cpp Fri Nov 16 12:02:08 2018 +0100 +++ b/src/hotspot/share/runtime/safepoint.cpp Fri Nov 16 13:35:16 2018 +0100 @@ -978,9 +978,9 @@ } } - // To debug the long safepoint, specify both DieOnSafepointTimeout & + // To debug the long safepoint, specify both AbortVMOnSafepointTimeout & // ShowMessageBoxOnError. - if (DieOnSafepointTimeout) { + if (AbortVMOnSafepointTimeout) { fatal("Safepoint sync time longer than " INTX_FORMAT "ms detected when executing %s.", SafepointTimeoutDelay, VMThread::vm_safepoint_description()); } Testing: hotspot/tier1, grepping for DieOnSafepointTimeout usages, jdk-submit (running) Thanks, -Aleksey From shade at redhat.com Fri Nov 16 16:30:34 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 16 Nov 2018 17:30:34 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations Message-ID: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8181143 Webrev: http://cr.openjdk.java.net/~shade/8181143/webrev.03/ SafepointTimeout is nice to discover long/stuck safepoint syncs. But it is as important to discover long/stuck VM operations. This patch complements the timeout machinery with tracking VM operation themselves. Among other things, this allows to terminate the VM when very long VM operation is blocking progress. High-availability users would enjoy fail-fast JVM -- in fact, the original prototype was done as request from Apache Ignite developers. Example with -XX:+VMOperationTimeout -XX:VMOperationTimeoutDelay=100 -XX:+AbortVMOnVMOperationTimeout: [3.117s][info][gc,start] GC(2) Pause Young (Normal) (G1 Evacuation Pause) [3.224s][warning][vmthread] VM Operation G1CollectForAllocation took longer than 100 ms # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/home/sh/jdk-jdk/src/hotspot/share/runtime/vmThread.cpp:218), pid=2536, tid=2554 # fatal error: VM Operation G1CollectForAllocation took longer than 100 ms # Testing: hotspot/tier1, ad-hoc tests, jdk-submit (pending) Thanks, -Aleksey From vladimir.kozlov at oracle.com Fri Nov 16 16:33:36 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 16 Nov 2018 08:33:36 -0800 Subject: Comments about JDK-8213473: Replace testB_mem_imm matcher with testUB_mem_imm In-Reply-To: <92e9e7b5-1566-ae77-5561-ae05a489c021@redhat.com> References: <92e9e7b5-1566-ae77-5561-ae05a489c021@redhat.com> Message-ID: <3ff5512c-7405-afc0-56cc-8910ed58e322@oracle.com> Thank you, Bernard For signed bytes we should restore signed testB_mem_imm. We can't use unsigned load in such case I think. Vladimir On 11/16/18 6:33 AM, Roman Kennke wrote: > Hi Bernard, > > Right... > > So either we reinstate the testB pattern, or we make the conversion > check for sign-extended mask too? Not sure if the latter would be 100% > correct though. > > Roman > >> Hi, >> >> The current fix [1] for JDK-8213473 [2] replaces 'testB_mem_imm' with >> 'testUB_mem_imm' in 'x86_64.ad' which is fine for unsigned bytes, for >> example: >> if ((flags & 0x80) != 0) ... >> >> But per JLS ?5.6.2 & ?5.1.2, if we have something like [3]: >> if ((flags & (byte)0x80) != 0) ... >> >> the mask is 32-bit sign-extended and 'LoadB' isn't converted to >> 'LoadUB' in 'AndINode::Ideal' [4] which is fine. >> >> Unfortunately 'testUB_mem_imm' won't be matched in this case, so I >> guess we'd need both 'testB_mem_imm' & 'testUB_mem_imm' patterns. >> >> If we all agree, I can create a JBS issue and send out a RFR for this? >> >> Thanks, >> Bernard >> >> [1] http://hg.openjdk.java.net/jdk/jdk/rev/656d2f222b42 >> [2] https://bugs.openjdk.java.net/browse/JDK-8213473 >> [3] http://hg.openjdk.java.net/jdk/jdk/file/74109912c738/src/jdk.charsets/share/classes/sun/nio/cs/ext/ISO2022_CN.java#l165 >> [4] http://hg.openjdk.java.net/jdk/jdk/file/74109912c738/src/hotspot/share/opto/mulnode.cpp#l493 >> > From shade at redhat.com Fri Nov 16 17:06:09 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 16 Nov 2018 18:06:09 +0100 Subject: RFR (T) 8214004: Missing space between compiler thread name and task info in hs_err Message-ID: <794f5f5d-c79d-733e-7b22-8f22a9a36fe2@redhat.com> RFE: https://bugs.openjdk.java.net/browse/JDK-8214004 Compiler thread name is actually "CompilerThread0" here: Threads with active compile tasks: C2 CompilerThread01391 b sun.nio.cs.StandardCharsets$Aliases::init (6328 bytes) Trivial fix: diff -r dd2e91ddf44d -r 85d67a2961a2 src/hotspot/share/runtime/thread.cpp --- a/src/hotspot/share/runtime/thread.cpp Fri Nov 16 17:23:08 2018 +0100 +++ b/src/hotspot/share/runtime/thread.cpp Fri Nov 16 18:05:06 2018 +0100 @@ -4691,6 +4691,7 @@ CompileTask* task = ct->task(); if (task != NULL) { thread->print_name_on_error(st, buf, buflen); + st->print(" "); task->print(st, NULL, true, true); } } Testing: eyeballing hs_errs Thanks, -Aleksey From daniel.daugherty at oracle.com Fri Nov 16 17:31:33 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 16 Nov 2018 12:31:33 -0500 Subject: RFR (T) 8214004: Missing space between compiler thread name and task info in hs_err In-Reply-To: <794f5f5d-c79d-733e-7b22-8f22a9a36fe2@redhat.com> References: <794f5f5d-c79d-733e-7b22-8f22a9a36fe2@redhat.com> Message-ID: <12342963-9cca-8522-d330-9ab0a6fa3623@oracle.com> Thumbs up! And agree that this is a trivial fix. Dan On 11/16/18 12:06 PM, Aleksey Shipilev wrote: > RFE: > https://bugs.openjdk.java.net/browse/JDK-8214004 > > Compiler thread name is actually "CompilerThread0" here: > > Threads with active compile tasks: > C2 CompilerThread01391 b sun.nio.cs.StandardCharsets$Aliases::init (6328 bytes) > > Trivial fix: > > diff -r dd2e91ddf44d -r 85d67a2961a2 src/hotspot/share/runtime/thread.cpp > --- a/src/hotspot/share/runtime/thread.cpp Fri Nov 16 17:23:08 2018 +0100 > +++ b/src/hotspot/share/runtime/thread.cpp Fri Nov 16 18:05:06 2018 +0100 > @@ -4691,6 +4691,7 @@ > CompileTask* task = ct->task(); > if (task != NULL) { > thread->print_name_on_error(st, buf, buflen); > + st->print(" "); > task->print(st, NULL, true, true); > } > } > > Testing: eyeballing hs_errs > > Thanks, > -Aleksey > From erik.osterlund at oracle.com Fri Nov 16 18:12:01 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 16 Nov 2018 19:12:01 +0100 Subject: RFR: 8213209: [REDO] Allow Klass::_subklass and _next_sibling to have unloaded classes Message-ID: Hi, This is a redo of Coleen's earlier patch to clean weak metadata links using the Compile_lock. We noticed the proposed solution could cause deadlocks. At least one of the reasons for that is that the Compile_lock was taken during printing and verification in safepoints. And the Compile_lock is sometimes held while issuing safepoints. The latter problem also forces us to take the Compile_lock before joining the suspendible thread set, to avoid another deadlock situation. But that is really nasty for other reasons. And there have been a few more problems as well. So in this case, we think it is both safer, less error prone, and better performing (although less important) to perform this cleaning in a lock-free fashion rather than trying to dodge all the issues related to the locking protocol. Inserts still require a lock. Therefore, we are dealing with lock-free reads, and lock-free cleaning. The lock-free cleaning is interacting with multiple lock-free readers and a single writer. The single inserter always helps with cleaning the subclass list head before prepending new nodes to the chain. That allows an invariant that the siblings link is never inserted pointing at a Klass that is unloading, which simplifies things a lot. The head is inserted in a classic load + CAS in a loop. Readers require load_acquire when reading the head, due to competing inserts putting new Klass pointers there. The siblings, however, only need relaxed consistency, because they are linked to data strictly older than the head, which has already been acquired. Unlinked entries are all inserted into a purge list, because freeing them immediately is not safe. A global handshake operation is performed, and after that ClassLoaderDataGraph::purge() will go through the list and delete the entries safely. Webrev: http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213209 Thanks, /Erik From zgu at redhat.com Fri Nov 16 20:02:29 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 16 Nov 2018 15:02:29 -0500 Subject: RFR (T) 8214004: Missing space between compiler thread name and task info in hs_err In-Reply-To: <794f5f5d-c79d-733e-7b22-8f22a9a36fe2@redhat.com> References: <794f5f5d-c79d-733e-7b22-8f22a9a36fe2@redhat.com> Message-ID: <31fd5b64-cfde-1841-0579-0f7d614e7e13@redhat.com> Looks good and trivial. Thanks, -Zhengyu On 11/16/18 12:06 PM, Aleksey Shipilev wrote: > RFE: > https://bugs.openjdk.java.net/browse/JDK-8214004 > > Compiler thread name is actually "CompilerThread0" here: > > Threads with active compile tasks: > C2 CompilerThread01391 b sun.nio.cs.StandardCharsets$Aliases::init (6328 bytes) > > Trivial fix: > > diff -r dd2e91ddf44d -r 85d67a2961a2 src/hotspot/share/runtime/thread.cpp > --- a/src/hotspot/share/runtime/thread.cpp Fri Nov 16 17:23:08 2018 +0100 > +++ b/src/hotspot/share/runtime/thread.cpp Fri Nov 16 18:05:06 2018 +0100 > @@ -4691,6 +4691,7 @@ > CompileTask* task = ct->task(); > if (task != NULL) { > thread->print_name_on_error(st, buf, buflen); > + st->print(" "); > task->print(st, NULL, true, true); > } > } > > Testing: eyeballing hs_errs > > Thanks, > -Aleksey > From coleen.phillimore at oracle.com Fri Nov 16 21:08:52 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 16 Nov 2018 16:08:52 -0500 Subject: RFR: 8213209: [REDO] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: References: Message-ID: Erik, thank you for taking this change and making it lock-free. On 11/16/18 1:12 PM, Erik ?sterlund wrote: > Hi, > > This is a redo of Coleen's earlier patch to clean weak metadata links > using the Compile_lock. > > We noticed the proposed solution could cause deadlocks. At least one > of the reasons for that is that the Compile_lock was taken during > printing and verification in safepoints. And the Compile_lock is > sometimes held while issuing safepoints. The latter problem also > forces us to take the Compile_lock before joining the suspendible > thread set, to avoid another deadlock situation. But that is really > nasty for other reasons. And there have been a few more problems as well. > > So in this case, we think it is both safer, less error prone, and > better performing (although less important) to perform this cleaning > in a lock-free fashion rather than trying to dodge all the issues > related to the locking protocol. > > Inserts still require a lock. Therefore, we are dealing with lock-free > reads, and lock-free cleaning. The lock-free cleaning is interacting > with multiple lock-free readers and a single writer. > > The single inserter always helps with cleaning the subclass list head > before prepending new nodes to the chain. That allows an invariant > that the siblings link is never inserted pointing at a Klass that is > unloading, which simplifies things a lot. The head is inserted in a > classic load + CAS in a loop. > > Readers require load_acquire when reading the head, due to competing > inserts putting new Klass pointers there. The siblings, however, only > need relaxed consistency, because they are linked to data strictly > older than the head, which has already been acquired. > > Unlinked entries are all inserted into a purge list, because freeing > them immediately is not safe. A global handshake operation is > performed, and after that ClassLoaderDataGraph::purge() will go > through the list and delete the entries safely. I don't see the purge list in your change, and it's not needed since this is unlinking in place.? I think this is your other change you're thinking of. Should the Compile_lock be removed in ciInstanceKlass.cpp ciInstanceKlass* ciInstanceKlass::implementor() { and in jvmciCompilerToVM.cpp C2V_VMENTRY(jobject, getImplementor, (JNIEnv *, jobject, jobject jvmci_type)) The former does a bit more than calling implementor().? Or do you want to leave these locks to be conservative? ? JVMCIKlassHandle handle(THREAD); ? { ??? // Need Compile_lock around implementor() ??? MutexLocker locker(Compile_lock); ??? handle = iklass->implementor(); ? } http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/oops/instanceKlass.cpp.udiff.html nof_implementors() has an assert_locked_or_safepoint(Compile_lock); which it no longer needs.? You'll hit this from printing because you've removed the locking here: 3093 if (is_interface()) { 3094 MutexLocker ml(Compile_lock); 3095 st->print_cr(BULLET"nof implementors: %d", nof_implementors()); 3096 if (nof_implementors() == 1) { 3097 st->print_cr(BULLET"implementor: "); 3098 st->print(" "); 3099 implementor()->print_value_on(st); 3100 st->cr(); 3101 } We don't test printing very well. http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/utilities/vmError.cpp.udiff.html This file is unchanged except a line removed. This change looks good.? I've run through the tests with it and didn't find any problems that doomed my earlier version. Thanks!! Coleen > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213209 > > Thanks, > /Erik From coleen.phillimore at oracle.com Fri Nov 16 21:36:19 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 16 Nov 2018 16:36:19 -0500 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> Message-ID: <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> Hi Ioi,? I really like this new wrapper on the old hashtable to not have to write the boilerplate code! http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html + MEMFLAGS F = mtInternal, I think you should require the mt type though and not make it a default parameter. mtInternal is not very useful to finding memory leaks. Apart from this (which I don't need to see another version), your change looks good and nice to get good performance benefits from this. thanks, Coleen On 11/15/18 12:31 AM, Ioi Lam wrote: > Coleen pointed out to me off-line that the good old (and ugly) > BasicHashtable already supports resizing. I think that might be a > better starting point for this RFE: > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ > > > I wrote a new template class called "KVHashtable" (copying the style > from ResourceHashtable). That way, you can instantiate different (Key > -> Value) mappings without writing tons of boilerplate code. The > performance is similar to my previous version, and the code is much > cleaner. > > I also renamed the RFE title, as well as the subject line of this RFR > e-mail. > > Thanks > > - Ioi > > > On 11/14/18 5:14 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> On 11/14/18 9:09 AM, Ioi Lam wrote: >>> >>> >>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> The change looks reasonable to me in general. It would be helpful >>>> to see the performance difference with the expendable table. Do you >>>> have any data when large number of classes are loaded (>20000)? How >>>> much saving does it provide? >>>> >>> >>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>> >>> BEFORE: 93.971 sec >>> AFTER:? 34.761 sec >> >> Thanks for the data! That's about 2.6x improvement with large set of >> classes. >> >> Thanks, >> Jiangli >>> >>> Thanks >>> - Ioi >>> >>>> Thanks, >>>> >>>> Jiangli >>>> >>>> >>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>> >>>>> >>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table >>>>> size to be >>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>> >>>>> >>>>> ? ?? ?? *** C++ template guru alert *** >>>>> >>>>> >>>>> I don't know much about C++ templates, so my attempt on doing this >>>>> may be >>>>> ill-advised. >>>>> >>>>> I *think* that with my patch, the performance of existing code, >>>>> which uses >>>>> a statically-defined SIZE,? should not be affected, as the C++ >>>>> compiler >>>>> should be able to constant-propagate and reduce the new code: >>>>> >>>>> ? ALWAYSINLINE unsigned size() const { >>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>> ????? return SIZE; >>>>> ??? } else { >>>>> ????? return _configured_table_size; >>>>> ??? } >>>>> ? } >>>>> >>>>> ? ALWAYSINLINE Node** get_table() const { >>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>> ????? return (Node**)(&_static_table[0]); >>>>> ??? } else { >>>>> ????? return _configured_table; >>>>> ??? } >>>>> ? } >>>>> >>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>> ??? unsigned index = hash % size();??? <----- >>>>> ??? Node** table = get_table(); >>>>> ??? Node** ptr = &table[index]; ? <----- >>>>> >>>>> back to the old code: >>>>> >>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>> ??? unsigned index = hash % SIZE;????? <----- >>>>> ??? Node** ptr = &_table[index];?????? <----- >>>>> >>>>> >>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>> >>>>> Thanks! >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> >>>> >>> >> > From zgu at redhat.com Fri Nov 16 21:38:29 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 16 Nov 2018 16:38:29 -0500 Subject: RFR(XXS): 8214001: DependencyContext::remove_dependent_nmethod() deletion in place is unnecessary Message-ID: Hi, Please review this small cleanup. It is not necessary to delete stale entry in place, since it calls expunge_stale_entries() to purge all stale entries at the end anyway. Bug: https://bugs.openjdk.java.net/browse/JDK-8214001 Webrev: http://cr.openjdk.java.net/~zgu/8214001/webrev.00/ Test: hotspot_compiler on Linux x64 (fastdebug and release) Thanks, -Zhengyu From coleen.phillimore at oracle.com Fri Nov 16 21:46:14 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 16 Nov 2018 16:46:14 -0500 Subject: RFR(XXS): 8214001: DependencyContext::remove_dependent_nmethod() deletion in place is unnecessary In-Reply-To: References: Message-ID: <7522d7e6-bdf5-222e-b778-5ccdbd534cac@oracle.com> Hi Zhengyu, your change conflicts with one that ErikO has out for review: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/src/hotspot/share/code/dependencyContext.cpp.udiff.html All he needs is someone who understands this code to review it! Thanks, Coleen On 11/16/18 4:38 PM, Zhengyu Gu wrote: > Hi, > > Please review this small cleanup. > > It is not necessary to delete stale entry in place, since it calls > expunge_stale_entries() to purge all stale entries at the end anyway. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214001 > Webrev: http://cr.openjdk.java.net/~zgu/8214001/webrev.00/ > > Test: > > ?hotspot_compiler on Linux x64 (fastdebug and release) > > Thanks, > > -Zhengyu > > > > From zgu at redhat.com Fri Nov 16 22:25:27 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 16 Nov 2018 17:25:27 -0500 Subject: RFR(XXS): 8214001: DependencyContext::remove_dependent_nmethod() deletion in place is unnecessary In-Reply-To: <7522d7e6-bdf5-222e-b778-5ccdbd534cac@oracle.com> References: <7522d7e6-bdf5-222e-b778-5ccdbd534cac@oracle.com> Message-ID: <5aa4d76f-b6ec-402e-2422-db16faa01e4f@redhat.com> Hi Coleen, On 11/16/18 4:46 PM, coleen.phillimore at oracle.com wrote: > > Hi Zhengyu, your change conflicts with one that ErikO has out for review: > > RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod > Thanks for pointing out. We did see some weird behavior here (attached below), ErikO's patch probably has something to do with it. I would like to retract my RFR. Thanks, -Zhengyu # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/home/jenkins/workspace/nightly/shenandoah-jdk/src/hotspot/share/code/dependencyContext.cpp:164), pid=134111, tid=134153 # assert(!find_stale_entries()) failed: inconsistent info # # JRE version: OpenJDK Runtime Environment (12.0) (fastdebug build 12-internal+0-adhoc.jenkins.shenandoah-jdk) # Java VM: OpenJDK 64-Bit Server VM (fastdebug 12-internal+0-adhoc.jenkins.shenandoah-jdk, mixed mode, sharing, tiered, compressed oops, shenandoah gc, linux-amd64) # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # --------------- S U M M A R Y ------------ Command Line: -Xmx1g -Xms1g -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC -XX:+UnlockDiagnosticVMOptions -XX:ShenandoahGCHeuristics=aggressive -XX:+VerifyObjectEquals -XX:+ShenandoahStoreCheck -XX:+ShenandoahVerifyOptoBarriers -XX:+ShenandoahOOMDuringEvacALot -XX:CompileCommandFile=/tmp/jmh2643637278243232044compilecommand org.openjdk.jmh.runner.ForkedMain 127.0.0.1 32974 Host: polwarth.rhts.eng.bos.redhat.com, Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, 56 cores, 251G, Red Hat Enterprise Linux Server release 7.4 (Maipo) Time: Thu Nov 15 11:24:33 2018 EST elapsed time: 196 seconds (0d 0h 3m 16s) --------------- T H R E A D --------------- Current thread (0x00007fdd5c03d000): GCTaskThread "Shenandoah GC Threads#29" [stack: 0x00007fdd594d6000,0x00007fdd595d6000] [id=134153] Stack: [0x00007fdd594d6000,0x00007fdd595d6000], sp=0x00007fdd595d49d0, free space=1018k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x19b223d] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x15d V [libjvm.so+0x19b3107] VMError::report_and_die(Thread*, void*, char const*, int, char const*, char const*, __va_list_tag*)+0x47 V [libjvm.so+0xb2f289] report_vm_error(char const*, int, char const*, char const*, ...)+0x109 V [libjvm.so+0xb7889b] DependencyContext::expunge_stale_entries()+0x9b V [libjvm.so+0xe52eab] InstanceKlass::clean_weak_instanceklass_links()+0xeb V [libjvm.so+0x152d847] KlassCleaningTask::work()+0x137 V [libjvm.so+0x152dc08] ParallelCleaningTask::work(unsigned int)+0xf8 V [libjvm.so+0x1a33736] GangWorker::run_task(WorkData)+0x66 V [libjvm.so+0x1a33848] GangWorker::loop()+0x48 V [libjvm.so+0x18ee032] Thread::call_run()+0x72 V [libjvm.so+0x14e5ff0] thread_native_entry(Thread*)+0x110 > http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/src/hotspot/share/code/dependencyContext.cpp.udiff.html > > > All he needs is someone who understands this code to review it! > > Thanks, > Coleen > > > On 11/16/18 4:38 PM, Zhengyu Gu wrote: >> Hi, >> >> Please review this small cleanup. >> >> It is not necessary to delete stale entry in place, since it calls >> expunge_stale_entries() to purge all stale entries at the end anyway. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214001 >> Webrev: http://cr.openjdk.java.net/~zgu/8214001/webrev.00/ >> >> Test: >> >> ?hotspot_compiler on Linux x64 (fastdebug and release) >> >> Thanks, >> >> -Zhengyu >> >> >> >> > From zgu at redhat.com Fri Nov 16 22:31:35 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 16 Nov 2018 17:31:35 -0500 Subject: RFR(XXS): 8214001: DependencyContext::remove_dependent_nmethod() deletion in place is unnecessary In-Reply-To: <7522d7e6-bdf5-222e-b778-5ccdbd534cac@oracle.com> References: <7522d7e6-bdf5-222e-b778-5ccdbd534cac@oracle.com> Message-ID: <7fbc2258-2a21-bcea-4b32-c99b2bcbb52a@redhat.com> Hi Coleen, On 11/16/18 4:46 PM, coleen.phillimore at oracle.com wrote: > > Hi Zhengyu, your change conflicts with one that ErikO has out for review: > > RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod > > http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/src/hotspot/share/code/dependencyContext.cpp.udiff.html > Could you please point me the review thread? I don't remember I see it, Thanks, -Zhengyu > > All he needs is someone who understands this code to review it! > > Thanks, > Coleen > > > On 11/16/18 4:38 PM, Zhengyu Gu wrote: >> Hi, >> >> Please review this small cleanup. >> >> It is not necessary to delete stale entry in place, since it calls >> expunge_stale_entries() to purge all stale entries at the end anyway. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214001 >> Webrev: http://cr.openjdk.java.net/~zgu/8214001/webrev.00/ >> >> Test: >> >> ?hotspot_compiler on Linux x64 (fastdebug and release) >> >> Thanks, >> >> -Zhengyu >> >> >> >> > From coleen.phillimore at oracle.com Fri Nov 16 23:21:39 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 16 Nov 2018 18:21:39 -0500 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> Message-ID: How about instead of two bool arguments that are hard to read and error prone and need comments, like: 76 CompiledMethodIterator iter(true /* only_alive */, false /* only_not_unloading */); enum FilterAlive { all_blobs, only_alive }; enum FilterUnloading { all_blobs, only_not_unloading }; Then it can be something like: ?? CompiledMethodIterator iter(only_alive, all_blobs); Don't know if you can repeate all_blobs like that. http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html I'm not sure if metadata_do() should filter out unloading.? It's looking for "old" methods to stop from deallocating memory for them (but it's in a safepoint).? If you're unloading, the IC caches are already cleaned out? http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html 1108 // The release is only needed for compile-time ordering, as accesses 1109 // into the nmethod after the store is not safe, due to the sweeper 1110 // being allowed to free it then, in the case of concurrent nmethod 1111 // unloading. Therefore, there is no need for acquire on the loader side. 1112 OrderAccess::release_store(&_state, (signed char)unloaded); I tried to make sense out of this first sentence, but couldn't really.? After the store to unloaded, can the sweeper free the nmethod?? maybe remove "then, " and it would make more sense? Thanks, Coleen On 11/12/18 5:46 PM, Erik ?sterlund wrote: > ..put in bug number in subject to make mail filters happy. > > /Erik > > On 2018-11-12 23:44, Erik ?sterlund wrote: >> Hi, >> >> All current GCs perform code cache unloading in safepoints. >> Therefore, nmethods that are alive but is_unloading() are never >> observed outside of safepoints. With concurrent class unloading, >> nmethods that are alive but is_unloading() will become observable >> outside of safepoints. This must be handled appropriately. >> >> In this patch I changed the nmethod/compiled method iterators to >> accept parameters describing whether they should filter out not >> is_alive() or is_unloading() methods. Since there is no obvious >> default (all combinations are used depending on call site), you have >> to explicitly set what iteration mode you want. >> >> Other than that, I make sure that the sweeper stays away from >> is_unloading() nmethods that are not yet is_unloaded(). To make the >> interactions between the sweeper and concurrent GC threads safe, I >> had to move down the store that sets the state to unloaded, and use a >> release_store there, to make sure no accesses float below it at >> compile-time. Once that store is observed, nmethods may be deleted. >> >> In the IC miss handler, I also need to lazily clean stale IC caches >> due to calling is_unloading nmethods using nmethod entry barriers. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8213755 >> >> Thanks, >> /Erik From kim.barrett at oracle.com Sat Nov 17 01:40:06 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 16 Nov 2018 20:40:06 -0500 Subject: RFR (S) 8211926: Catastrophic size_t underflow in BitMap::*_large methods In-Reply-To: References: <45625231-E026-4A26-99AA-B35FFA2B5535@oracle.com> <8A15BCB8-C0BE-472F-98EF-7686D095E8BC@oracle.com> <56020dbe-7a11-00b5-bcca-c3b4943732f4@redhat.com> <253769FF-97DD-468A-AD8C-13F5DAA50639@oracle.com> <567c0b34-6e22-4284-f08c-0197eba954a5@redhat.com> Message-ID: > On Nov 12, 2018, at 2:14 AM, Thomas St?fe wrote: > > Hi Kim, > > On Sat, Nov 10, 2018 at 6:55 PM Kim Barrett wrote: >> >>> On Nov 10, 2018, at 7:58 AM, Thomas St?fe wrote: >>> - I find it confusing that we use the same type (idx_t) both for word- >>> and bit-indices. It would make the code a bit more readable if those >>> were separate types (even if they both were to alias to size_t, >>> ultimately). >> >> I dislike this too. I?d be interested in seeing a proposal to fix that, though >> I worry about the fanout. I *think* clients mostly operate in bits and the >> use of words is mostly limited to the implementation, but I haven?t done >> the searches needed to check that. > > Should I have the time I'll give this a try. Not sure when I will find > the time though. Same here. >>> - Generally, I find putting the decision between "large" and >>> "non-large" APIs off to the caller a bit strange. As a user of bitmap, >>> I cannot know at which point which variant would be better. I would >>> prefer the class itself handling that. >> >> I was thinking much the same thing while reviewing this change. >> Maybe some of the public API in this area should be revisited. >> In particular, I was surprised that one might call ?large? functions >> directly, rather than calling hinted functions with a ?large? hint. >> Perhaps the ?large? functions should be private helpers that just >> do what they are told (maybe with an assert as before), and the >> calls to them should be based on the hint and the runtime size >> check they are now performing. Or maybe the whole hint thing >> should be reconsidered. I?d be interested in a proposal in this >> area too. > > I would consider switching between loops and memset to be an internal > tuning parameter which the caller should not have to bother with. I > think hinting makes sense when I know something the code cannot know. > But there are no hidden information here beside the range size. Unless > I misunderstand the problem. I mostly agree. The thing a hint could be used for is to order a sequence of tests. For example, if hinted large, then check for large first, while if hinted small, check for the single bit or all bits in one word cases first. From ioi.lam at oracle.com Sat Nov 17 03:55:58 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 16 Nov 2018 19:55:58 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> Message-ID: Hi Coleen, I deleted the default value for MEMFLAGS as you suggested. For my instantiated templates, I still use mtInternal, though, since I can't find anything better for using at CDS dump time. Also, Jiangli noted that there's a memory leak, because I allocate and free the KVHashtable dynamically. So I added a destructor to BasicHashtable to free the buckets and the block-allocated entries. http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ This comment around Hashtable::allocate_new_entry() is wrong now -- "The allocator in blocks is preferable but doesn't have free semantics". Maybe I should change it to this? "The block allocator in BasicHashtable has less fragmentation, but the memory is not freed until the whole table is freed. Use allocate_new_entry() if you want to immediately free the memory used by each entry". I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested the solaris/x64 build since it seems to break every time you do something with templates :-( Thanks! - Ioi On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: > > Hi Ioi,? I really like this new wrapper on the old hashtable to not > have to write the boilerplate code! > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html > > > + MEMFLAGS F = mtInternal, > > > I think you should require the mt type though and not make it a > default parameter. mtInternal is not very useful to finding memory leaks. > > Apart from this (which I don't need to see another version), your > change looks good and nice to get good performance benefits from this. > > thanks, > Coleen > > On 11/15/18 12:31 AM, Ioi Lam wrote: >> Coleen pointed out to me off-line that the good old (and ugly) >> BasicHashtable already supports resizing. I think that might be a >> better starting point for this RFE: >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >> >> >> I wrote a new template class called "KVHashtable" (copying the style >> from ResourceHashtable). That way, you can instantiate different (Key >> -> Value) mappings without writing tons of boilerplate code. The >> performance is similar to my previous version, and the code is much >> cleaner. >> >> I also renamed the RFE title, as well as the subject line of this RFR >> e-mail. >> >> Thanks >> >> - Ioi >> >> >> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>> >>>> >>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> The change looks reasonable to me in general. It would be helpful >>>>> to see the performance difference with the expendable table. Do >>>>> you have any data when large number of classes are loaded >>>>> (>20000)? How much saving does it provide? >>>>> >>>> >>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>> >>>> BEFORE: 93.971 sec >>>> AFTER:? 34.761 sec >>> >>> Thanks for the data! That's about 2.6x improvement with large set of >>> classes. >>> >>> Thanks, >>> Jiangli >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Thanks, >>>>> >>>>> Jiangli >>>>> >>>>> >>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>> >>>>>> >>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table >>>>>> size to be >>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>> >>>>>> >>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>> >>>>>> >>>>>> I don't know much about C++ templates, so my attempt on doing >>>>>> this may be >>>>>> ill-advised. >>>>>> >>>>>> I *think* that with my patch, the performance of existing code, >>>>>> which uses >>>>>> a statically-defined SIZE,? should not be affected, as the C++ >>>>>> compiler >>>>>> should be able to constant-propagate and reduce the new code: >>>>>> >>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>> ????? return SIZE; >>>>>> ??? } else { >>>>>> ????? return _configured_table_size; >>>>>> ??? } >>>>>> ? } >>>>>> >>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>> ????? return (Node**)(&_static_table[0]); >>>>>> ??? } else { >>>>>> ????? return _configured_table; >>>>>> ??? } >>>>>> ? } >>>>>> >>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>> ??? unsigned index = hash % size();??? <----- >>>>>> ??? Node** table = get_table(); >>>>>> ??? Node** ptr = &table[index]; ? <----- >>>>>> >>>>>> back to the old code: >>>>>> >>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>> ??? unsigned index = hash % SIZE;????? <----- >>>>>> ??? Node** ptr = &_table[index];?????? <----- >>>>>> >>>>>> >>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>> >>>>>> Thanks! >>>>>> - Ioi >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > From jiangli.zhou at oracle.com Sat Nov 17 21:54:11 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Sat, 17 Nov 2018 13:54:11 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> Message-ID: <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> Hi Ioi, This looks good. Glad to see the change that utilizes the existing BasicHashtable. Thanks Coleen for the suggestion! To avoid changing the existing assumptions about BasicHashtable and Hashtable, how about adding a destructor to the new KVHashtable instead of BasicHashtable? Just did a quick test, which seems to work properly. Thanks, Jiangli On 11/16/18 7:55 PM, Ioi Lam wrote: > Hi Coleen, > > I deleted the default value for MEMFLAGS as you suggested. For my > instantiated templates, I still use mtInternal, though, since I can't > find anything better for using at CDS dump time. > > Also, Jiangli noted that there's a memory leak, because I allocate and > free the KVHashtable dynamically. So I added a destructor to > BasicHashtable to free the buckets and the block-allocated entries. > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ > > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ > > > This comment around Hashtable::allocate_new_entry() is wrong now -- > "The allocator in blocks is preferable but doesn't have free > semantics". Maybe I should change it to this? > > "The block allocator in BasicHashtable has less fragmentation, but the > memory is not freed until the whole table is freed. Use > allocate_new_entry() if you want to immediately free the memory used > by each entry". > > I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested > the solaris/x64 build since it seems to break every time you do > something with templates :-( > > Thanks! > - Ioi > > On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >> >> Hi Ioi,? I really like this new wrapper on the old hashtable to not >> have to write the boilerplate code! >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >> >> >> + MEMFLAGS F = mtInternal, >> >> >> I think you should require the mt type though and not make it a >> default parameter. mtInternal is not very useful to finding memory >> leaks. >> >> Apart from this (which I don't need to see another version), your >> change looks good and nice to get good performance benefits from this. >> >> thanks, >> Coleen >> >> On 11/15/18 12:31 AM, Ioi Lam wrote: >>> Coleen pointed out to me off-line that the good old (and ugly) >>> BasicHashtable already supports resizing. I think that might be a >>> better starting point for this RFE: >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>> >>> >>> I wrote a new template class called "KVHashtable" (copying the style >>> from ResourceHashtable). That way, you can instantiate different >>> (Key -> Value) mappings without writing tons of boilerplate code. >>> The performance is similar to my previous version, and the code is >>> much cleaner. >>> >>> I also renamed the RFE title, as well as the subject line of this >>> RFR e-mail. >>> >>> Thanks >>> >>> - Ioi >>> >>> >>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>> >>>>> >>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> The change looks reasonable to me in general. It would be helpful >>>>>> to see the performance difference with the expendable table. Do >>>>>> you have any data when large number of classes are loaded >>>>>> (>20000)? How much saving does it provide? >>>>>> >>>>> >>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>> >>>>> BEFORE: 93.971 sec >>>>> AFTER:? 34.761 sec >>>> >>>> Thanks for the data! That's about 2.6x improvement with large set >>>> of classes. >>>> >>>> Thanks, >>>> Jiangli >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Thanks, >>>>>> >>>>>> Jiangli >>>>>> >>>>>> >>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>> >>>>>>> >>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table >>>>>>> size to be >>>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>>> >>>>>>> >>>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>>> >>>>>>> >>>>>>> I don't know much about C++ templates, so my attempt on doing >>>>>>> this may be >>>>>>> ill-advised. >>>>>>> >>>>>>> I *think* that with my patch, the performance of existing code, >>>>>>> which uses >>>>>>> a statically-defined SIZE,? should not be affected, as the C++ >>>>>>> compiler >>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>> >>>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>> ????? return SIZE; >>>>>>> ??? } else { >>>>>>> ????? return _configured_table_size; >>>>>>> ??? } >>>>>>> ? } >>>>>>> >>>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>> ????? return (Node**)(&_static_table[0]); >>>>>>> ??? } else { >>>>>>> ????? return _configured_table; >>>>>>> ??? } >>>>>>> ? } >>>>>>> >>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>> ??? unsigned index = hash % size();??? <----- >>>>>>> ??? Node** table = get_table(); >>>>>>> ??? Node** ptr = &table[index]; ? <----- >>>>>>> >>>>>>> back to the old code: >>>>>>> >>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>> ??? unsigned index = hash % SIZE;????? <----- >>>>>>> ??? Node** ptr = &_table[index];?????? <----- >>>>>>> >>>>>>> >>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>> >>>>>>> Thanks! >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Sun Nov 18 02:48:59 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 17 Nov 2018 18:48:59 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> Message-ID: <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> On 11/17/18 1:54 PM, Jiangli Zhou wrote: > Hi Ioi, > > This looks good. Glad to see the change that utilizes the existing > BasicHashtable. Thanks Coleen for the suggestion! > > To avoid changing the existing assumptions about BasicHashtable and > Hashtable, how about adding a destructor to the new KVHashtable > instead of BasicHashtable? Just did a quick test, which seems to work > properly. > Hi Jiangli, Thanks for the review. Not freeing the entries when a BasicHashtable is destructed is a bug. None of the existing BasicHashtables (and subclasses thereof) are ever destructed, so I don't think my code would impact them. Also, if any code depends on the entries not freed even if the table is destructed, that's clearly a bug in that code, and it should be fixed. If I don't add the destructor to BasicHashtable, the next person who wants to allocate/free BasicHashtables will run in to the same issue. Thanks - Ioi > Thanks, > > Jiangli > > > On 11/16/18 7:55 PM, Ioi Lam wrote: >> Hi Coleen, >> >> I deleted the default value for MEMFLAGS as you suggested. For my >> instantiated templates, I still use mtInternal, though, since I can't >> find anything better for using at CDS dump time. >> >> Also, Jiangli noted that there's a memory leak, because I allocate >> and free the KVHashtable dynamically. So I added a destructor to >> BasicHashtable to free the buckets and the block-allocated entries. >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >> >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >> >> >> This comment around Hashtable::allocate_new_entry() is wrong now -- >> "The allocator in blocks is preferable but doesn't have free >> semantics". Maybe I should change it to this? >> >> "The block allocator in BasicHashtable has less fragmentation, but >> the memory is not freed until the whole table is freed. Use >> allocate_new_entry() if you want to immediately free the memory used >> by each entry". >> >> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested >> the solaris/x64 build since it seems to break every time you do >> something with templates :-( >> >> Thanks! >> - Ioi >> >> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>> >>> Hi Ioi,? I really like this new wrapper on the old hashtable to not >>> have to write the boilerplate code! >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>> >>> >>> + MEMFLAGS F = mtInternal, >>> >>> >>> I think you should require the mt type though and not make it a >>> default parameter. mtInternal is not very useful to finding memory >>> leaks. >>> >>> Apart from this (which I don't need to see another version), your >>> change looks good and nice to get good performance benefits from this. >>> >>> thanks, >>> Coleen >>> >>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>> Coleen pointed out to me off-line that the good old (and ugly) >>>> BasicHashtable already supports resizing. I think that might be a >>>> better starting point for this RFE: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>> >>>> >>>> I wrote a new template class called "KVHashtable" (copying the >>>> style from ResourceHashtable). That way, you can instantiate >>>> different (Key -> Value) mappings without writing tons of >>>> boilerplate code. The performance is similar to my previous >>>> version, and the code is much cleaner. >>>> >>>> I also renamed the RFE title, as well as the subject line of this >>>> RFR e-mail. >>>> >>>> Thanks >>>> >>>> - Ioi >>>> >>>> >>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>> >>>>>> >>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>> Hi Ioi, >>>>>>> >>>>>>> The change looks reasonable to me in general. It would be >>>>>>> helpful to see the performance difference with the expendable >>>>>>> table. Do you have any data when large number of classes are >>>>>>> loaded (>20000)? How much saving does it provide? >>>>>>> >>>>>> >>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>> >>>>>> BEFORE: 93.971 sec >>>>>> AFTER:? 34.761 sec >>>>> >>>>> Thanks for the data! That's about 2.6x improvement with large set >>>>> of classes. >>>>> >>>>> Thanks, >>>>> Jiangli >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jiangli >>>>>>> >>>>>>> >>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>> >>>>>>>> >>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the >>>>>>>> table size to be >>>>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>>>> >>>>>>>> >>>>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>>>> >>>>>>>> >>>>>>>> I don't know much about C++ templates, so my attempt on doing >>>>>>>> this may be >>>>>>>> ill-advised. >>>>>>>> >>>>>>>> I *think* that with my patch, the performance of existing code, >>>>>>>> which uses >>>>>>>> a statically-defined SIZE,? should not be affected, as the C++ >>>>>>>> compiler >>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>> >>>>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>> ????? return SIZE; >>>>>>>> ??? } else { >>>>>>>> ????? return _configured_table_size; >>>>>>>> ??? } >>>>>>>> ? } >>>>>>>> >>>>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>> ????? return (Node**)(&_static_table[0]); >>>>>>>> ??? } else { >>>>>>>> ????? return _configured_table; >>>>>>>> ??? } >>>>>>>> ? } >>>>>>>> >>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>> ??? unsigned index = hash % size();??? <----- >>>>>>>> ??? Node** table = get_table(); >>>>>>>> ??? Node** ptr = &table[index]; ? <----- >>>>>>>> >>>>>>>> back to the old code: >>>>>>>> >>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>> ??? unsigned index = hash % SIZE;????? <----- >>>>>>>> ??? Node** ptr = &_table[index];?????? <----- >>>>>>>> >>>>>>>> >>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>> >>>>>>>> Thanks! >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From jiangli.zhou at oracle.com Sun Nov 18 06:28:17 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Sat, 17 Nov 2018 22:28:17 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> Message-ID: <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> Hi Ioi, On 11/17/18 6:48 PM, Ioi Lam wrote: > > > On 11/17/18 1:54 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> This looks good. Glad to see the change that utilizes the existing >> BasicHashtable. Thanks Coleen for the suggestion! >> >> To avoid changing the existing assumptions about BasicHashtable and >> Hashtable, how about adding a destructor to the new KVHashtable >> instead of BasicHashtable? Just did a quick test, which seems to work >> properly. >> > Hi Jiangli, > > Thanks for the review. > > Not freeing the entries when a BasicHashtable is destructed is a bug. > None of the existing BasicHashtables (and subclasses thereof) are ever > destructed, so I don't think my code would impact them. Also, if any > code depends on the entries not freed even if the table is destructed, > that's clearly a bug in that code, and it should be fixed. > > If I don't add the destructor to BasicHashtable, the next person who > wants to allocate/free BasicHashtables will run in to the same issue. Dictionary, PackageEntryTable, ModuleEntryTable and G1CodeRootSetTable are derived from Hashtable and they call free_buckets() in their destructors. So free_buckets() will be called twice when those tables are freed although it's harmless. If you want to avoid the duplicated call, you could remove the free_buckets() calls from those classes' destructors and keep the current BasicHashtables changes. I don't have a strong opinion on this, I'll leave it to you to decide. Thanks, Jiangli > > Thanks > - Ioi > > >> Thanks, >> >> Jiangli >> >> >> On 11/16/18 7:55 PM, Ioi Lam wrote: >>> Hi Coleen, >>> >>> I deleted the default value for MEMFLAGS as you suggested. For my >>> instantiated templates, I still use mtInternal, though, since I >>> can't find anything better for using at CDS dump time. >>> >>> Also, Jiangli noted that there's a memory leak, because I allocate >>> and free the KVHashtable dynamically. So I added a destructor to >>> BasicHashtable to free the buckets and the block-allocated entries. >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>> >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>> >>> >>> This comment around Hashtable::allocate_new_entry() is wrong now -- >>> "The allocator in blocks is preferable but doesn't have free >>> semantics". Maybe I should change it to this? >>> >>> "The block allocator in BasicHashtable has less fragmentation, but >>> the memory is not freed until the whole table is freed. Use >>> allocate_new_entry() if you want to immediately free the memory used >>> by each entry". >>> >>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested >>> the solaris/x64 build since it seems to break every time you do >>> something with templates :-( >>> >>> Thanks! >>> - Ioi >>> >>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>> >>>> Hi Ioi,? I really like this new wrapper on the old hashtable to not >>>> have to write the boilerplate code! >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>> >>>> >>>> + MEMFLAGS F = mtInternal, >>>> >>>> >>>> I think you should require the mt type though and not make it a >>>> default parameter. mtInternal is not very useful to finding memory >>>> leaks. >>>> >>>> Apart from this (which I don't need to see another version), your >>>> change looks good and nice to get good performance benefits from this. >>>> >>>> thanks, >>>> Coleen >>>> >>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>> Coleen pointed out to me off-line that the good old (and ugly) >>>>> BasicHashtable already supports resizing. I think that might be a >>>>> better starting point for this RFE: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>> >>>>> >>>>> I wrote a new template class called "KVHashtable" (copying the >>>>> style from ResourceHashtable). That way, you can instantiate >>>>> different (Key -> Value) mappings without writing tons of >>>>> boilerplate code. The performance is similar to my previous >>>>> version, and the code is much cleaner. >>>>> >>>>> I also renamed the RFE title, as well as the subject line of this >>>>> RFR e-mail. >>>>> >>>>> Thanks >>>>> >>>>> - Ioi >>>>> >>>>> >>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>> >>>>>>> >>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> The change looks reasonable to me in general. It would be >>>>>>>> helpful to see the performance difference with the expendable >>>>>>>> table. Do you have any data when large number of classes are >>>>>>>> loaded (>20000)? How much saving does it provide? >>>>>>>> >>>>>>> >>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>> >>>>>>> BEFORE: 93.971 sec >>>>>>> AFTER:? 34.761 sec >>>>>> >>>>>> Thanks for the data! That's about 2.6x improvement with large set >>>>>> of classes. >>>>>> >>>>>> Thanks, >>>>>> Jiangli >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Jiangli >>>>>>>> >>>>>>>> >>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>> >>>>>>>>> >>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the >>>>>>>>> table size to be >>>>>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>>>>> >>>>>>>>> >>>>>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>>>>> >>>>>>>>> >>>>>>>>> I don't know much about C++ templates, so my attempt on doing >>>>>>>>> this may be >>>>>>>>> ill-advised. >>>>>>>>> >>>>>>>>> I *think* that with my patch, the performance of existing >>>>>>>>> code, which uses >>>>>>>>> a statically-defined SIZE,? should not be affected, as the C++ >>>>>>>>> compiler >>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>> >>>>>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>> ????? return SIZE; >>>>>>>>> ??? } else { >>>>>>>>> ????? return _configured_table_size; >>>>>>>>> ??? } >>>>>>>>> ? } >>>>>>>>> >>>>>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>> ????? return (Node**)(&_static_table[0]); >>>>>>>>> ??? } else { >>>>>>>>> ????? return _configured_table; >>>>>>>>> ??? } >>>>>>>>> ? } >>>>>>>>> >>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>> ??? unsigned index = hash % size();??? <----- >>>>>>>>> ??? Node** table = get_table(); >>>>>>>>> ??? Node** ptr = &table[index]; ? <----- >>>>>>>>> >>>>>>>>> back to the old code: >>>>>>>>> >>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>> ??? unsigned index = hash % SIZE;????? <----- >>>>>>>>> ??? Node** ptr = &_table[index]; <----- >>>>>>>>> >>>>>>>>> >>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Mon Nov 19 03:50:11 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Sun, 18 Nov 2018 19:50:11 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> Message-ID: <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> On 11/17/18 10:28 PM, Jiangli Zhou wrote: > Hi Ioi, > > On 11/17/18 6:48 PM, Ioi Lam wrote: > >> >> >> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> This looks good. Glad to see the change that utilizes the existing >>> BasicHashtable. Thanks Coleen for the suggestion! >>> >>> To avoid changing the existing assumptions about BasicHashtable and >>> Hashtable, how about adding a destructor to the new KVHashtable >>> instead of BasicHashtable? Just did a quick test, which seems to >>> work properly. >>> >> Hi Jiangli, >> >> Thanks for the review. >> >> Not freeing the entries when a BasicHashtable is destructed is a bug. >> None of the existing BasicHashtables (and subclasses thereof) are >> ever destructed, so I don't think my code would impact them. Also, if >> any code depends on the entries not freed even if the table is >> destructed, that's clearly a bug in that code, and it should be fixed. >> >> If I don't add the destructor to BasicHashtable, the next person who >> wants to allocate/free BasicHashtables will run in to the same issue. > > Dictionary, PackageEntryTable, ModuleEntryTable and G1CodeRootSetTable > are derived from Hashtable and they call free_buckets() in their > destructors. So free_buckets() will be called twice when those tables > are freed although it's harmless. If you want to avoid the duplicated > call, you could remove the free_buckets() calls from those classes' > destructors and keep the current BasicHashtables changes. I don't have > a strong opinion on this, I'll leave it to you to decide. > > Hi Jiangli, Good catch. I've remove the call to free_buckets() in the destructor of those 4 classes. Here's the delta from the last webrev: http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ Also, I found 2 problems while doing this. Hashtables in HotSpot are really a mess. https://bugs.openjdk.java.net/browse/JDK-8214029 https://bugs.openjdk.java.net/browse/JDK-8214030 Thanks - Ioi > Thanks, > > Jiangli >> >> Thanks >> - Ioi >> >> >>> Thanks, >>> >>> Jiangli >>> >>> >>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>> Hi Coleen, >>>> >>>> I deleted the default value for MEMFLAGS as you suggested. For my >>>> instantiated templates, I still use mtInternal, though, since I >>>> can't find anything better for using at CDS dump time. >>>> >>>> Also, Jiangli noted that there's a memory leak, because I allocate >>>> and free the KVHashtable dynamically. So I added a destructor to >>>> BasicHashtable to free the buckets and the block-allocated entries. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>> >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>> >>>> >>>> This comment around Hashtable::allocate_new_entry() is wrong now -- >>>> "The allocator in blocks is preferable but doesn't have free >>>> semantics". Maybe I should change it to this? >>>> >>>> "The block allocator in BasicHashtable has less fragmentation, but >>>> the memory is not freed until the whole table is freed. Use >>>> allocate_new_entry() if you want to immediately free the memory >>>> used by each entry". >>>> >>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested >>>> the solaris/x64 build since it seems to break every time you do >>>> something with templates :-( >>>> >>>> Thanks! >>>> - Ioi >>>> >>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>> >>>>> Hi Ioi,? I really like this new wrapper on the old hashtable to >>>>> not have to write the boilerplate code! >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>> >>>>> >>>>> + MEMFLAGS F = mtInternal, >>>>> >>>>> >>>>> I think you should require the mt type though and not make it a >>>>> default parameter. mtInternal is not very useful to finding memory >>>>> leaks. >>>>> >>>>> Apart from this (which I don't need to see another version), your >>>>> change looks good and nice to get good performance benefits from >>>>> this. >>>>> >>>>> thanks, >>>>> Coleen >>>>> >>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>> Coleen pointed out to me off-line that the good old (and ugly) >>>>>> BasicHashtable already supports resizing. I think that might be a >>>>>> better starting point for this RFE: >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>> >>>>>> >>>>>> I wrote a new template class called "KVHashtable" (copying the >>>>>> style from ResourceHashtable). That way, you can instantiate >>>>>> different (Key -> Value) mappings without writing tons of >>>>>> boilerplate code. The performance is similar to my previous >>>>>> version, and the code is much cleaner. >>>>>> >>>>>> I also renamed the RFE title, as well as the subject line of this >>>>>> RFR e-mail. >>>>>> >>>>>> Thanks >>>>>> >>>>>> - Ioi >>>>>> >>>>>> >>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>> Hi Ioi, >>>>>>> >>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>> Hi Ioi, >>>>>>>>> >>>>>>>>> The change looks reasonable to me in general. It would be >>>>>>>>> helpful to see the performance difference with the expendable >>>>>>>>> table. Do you have any data when large number of classes are >>>>>>>>> loaded (>20000)? How much saving does it provide? >>>>>>>>> >>>>>>>> >>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>> >>>>>>>> BEFORE: 93.971 sec >>>>>>>> AFTER:? 34.761 sec >>>>>>> >>>>>>> Thanks for the data! That's about 2.6x improvement with large >>>>>>> set of classes. >>>>>>> >>>>>>> Thanks, >>>>>>> Jiangli >>>>>>>> >>>>>>>> Thanks >>>>>>>> - Ioi >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Jiangli >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the >>>>>>>>>> table size to be >>>>>>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I don't know much about C++ templates, so my attempt on doing >>>>>>>>>> this may be >>>>>>>>>> ill-advised. >>>>>>>>>> >>>>>>>>>> I *think* that with my patch, the performance of existing >>>>>>>>>> code, which uses >>>>>>>>>> a statically-defined SIZE,? should not be affected, as the >>>>>>>>>> C++ compiler >>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>> >>>>>>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>> ????? return SIZE; >>>>>>>>>> ??? } else { >>>>>>>>>> ????? return _configured_table_size; >>>>>>>>>> ??? } >>>>>>>>>> ? } >>>>>>>>>> >>>>>>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>> ????? return (Node**)(&_static_table[0]); >>>>>>>>>> ??? } else { >>>>>>>>>> ????? return _configured_table; >>>>>>>>>> ??? } >>>>>>>>>> ? } >>>>>>>>>> >>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>> ??? unsigned index = hash % size();??? <----- >>>>>>>>>> ??? Node** table = get_table(); >>>>>>>>>> ??? Node** ptr = &table[index]; ? <----- >>>>>>>>>> >>>>>>>>>> back to the old code: >>>>>>>>>> >>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>> ??? unsigned index = hash % SIZE;????? <----- >>>>>>>>>> ??? Node** ptr = &_table[index]; <----- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From david.holmes at oracle.com Mon Nov 19 05:08:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 19 Nov 2018 15:08:12 +1000 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> Message-ID: Hi Aleksey, On 17/11/2018 2:18 am, Aleksey Shipilev wrote: > RFE: > https://bugs.openjdk.java.net/browse/JDK-8213992 > > While the original option was apparently conceived to aid debugging during the VM development, it is > also useful in product builds to terminate VM when it is stuck on safepoint sync. This is sometimes > asked by external people who have fail-overs set up to deal with VM crashes: making VM fail > predictably on errors like this is a plus for them. > > David suggested making the option "product", but I don't like it very much: "product" option > suggests this is a production-grade feature and it comes with expectations of support, which is > interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode available in > product builds without the attached notion of support. Users that want fail-fast VM crash can then > use that option on "we know what we are doing" basis. I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported product". If you really want to fail-fast in production then you're looking for a production flag not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. > The name of the option is hopefully aligned with: > diagnostic AbortVMOnException (existing) > diagnostic AbortVMOnSafepointTimeout (this one) > diagnostic AbortVMOnVMOperationTimeout (JDK-8181143) > diagnostic AbortVMOnCompilationFailure (JDK-8212070) That's just crying out for an AbortVMOn: type of interface. :) But I won't force that on you. > Fix: > > diff -r 9ad663e63da5 -r 132db6e99f77 src/hotspot/share/runtime/globals.hpp > --- a/src/hotspot/share/runtime/globals.hpp Fri Nov 16 12:02:08 2018 +0100 > +++ b/src/hotspot/share/runtime/globals.hpp Fri Nov 16 13:35:16 2018 +0100 > @@ -498,7 +498,7 @@ > "Time out and warn or fail after SafepointTimeoutDelay " \ > "milliseconds if failed to reach safepoint") \ > \ > - develop(bool, DieOnSafepointTimeout, false, \ > + diagnostic(bool, AbortVMOnSafepointTimeout, false, \ > "Die upon failure to reach safepoint (see SafepointTimeout)") \ s/Die/Abort Thanks. \ > /* 50 retries * (5 * current_retry_count) millis = ~6.375 seconds */ \ > diff -r 9ad663e63da5 -r 132db6e99f77 src/hotspot/share/runtime/safepoint.cpp > --- a/src/hotspot/share/runtime/safepoint.cpp Fri Nov 16 12:02:08 2018 +0100 > +++ b/src/hotspot/share/runtime/safepoint.cpp Fri Nov 16 13:35:16 2018 +0100 > @@ -978,9 +978,9 @@ > } > } > > - // To debug the long safepoint, specify both DieOnSafepointTimeout & > + // To debug the long safepoint, specify both AbortVMOnSafepointTimeout & > // ShowMessageBoxOnError. > - if (DieOnSafepointTimeout) { > + if (AbortVMOnSafepointTimeout) { > fatal("Safepoint sync time longer than " INTX_FORMAT "ms detected when executing %s.", > SafepointTimeoutDelay, VMThread::vm_safepoint_description()); > } > > Testing: hotspot/tier1, grepping for DieOnSafepointTimeout usages, jdk-submit (running) No further comments. Thanks, David > Thanks, > -Aleksey > From kim.barrett at oracle.com Mon Nov 19 06:14:09 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 19 Nov 2018 01:14:09 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot Message-ID: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> Please review this fix of the debug-only copy constructor and assignment operator for ResourceObj, along with adding some missing uses of the copy constructor. The missing copy constructor uses were found by enabling -Wextra. The ResourceObj debug-only copy constructor should behave exactly the same as the corresponding default constructor. That is, the setup for checking the allocation state in the destructor and operator delete should be the same whether in the normal or copy constructor. That previously wasn't true, resulting in assert failures. The ResourceObj assignment operator also should not be assuming the target of the assignment was stack/embeded allocated; that's just not a valid assumption. The corresponding assertion has been removed. Note that the ResourceObj allocation/construction/deletion code is oddly indented, using an initial 4 space indentation and 2 spaces for further indenting, unlike the usual consistent 2 space indentation used elsewhere in HotSpot, including elsewhere in allocation.cpp. I've left that indentation intact to minimize diffs in some views for review. Unless there are objections, I plan to fix the indentation. CR: https://bugs.openjdk.java.net/browse/JDK-8213481 Webrev: http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ Testing: mach5 tier1-3. There were many failures in these tiers with just the addition of the missing copy constructor calls (JDK-8213414). From david.holmes at oracle.com Mon Nov 19 06:13:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 19 Nov 2018 16:13:53 +1000 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: Hi Aleksey, First the synopsis is not accurate: "Introduce diagnostic flag to abort VM on too long VM operations" You're not just introducing one diagnostic flag, your introducing the entire VM operation timeout mechanism, including two product flags and one diagnostic. So the CR needs to reflect that clearly and you will need a CSR request to add the two product flags. And they will need to be documented. Three flags just for this makes me cringe. (Yes it mirrors the safepoint timeout flags but if that were proposed today I'd have the same reaction.) On 17/11/2018 2:30 am, Aleksey Shipilev wrote: > RFE: > https://bugs.openjdk.java.net/browse/JDK-8181143 > > Webrev: > http://cr.openjdk.java.net/~shade/8181143/webrev.03/ > > SafepointTimeout is nice to discover long/stuck safepoint syncs. But it is as important to discover > long/stuck VM operations. This patch complements the timeout machinery with tracking VM operation > themselves. Among other things, this allows to terminate the VM when very long VM operation is > blocking progress. High-availability users would enjoy fail-fast JVM -- in fact, the original > prototype was done as request from Apache Ignite developers. > > Example with -XX:+VMOperationTimeout -XX:VMOperationTimeoutDelay=100 -XX:+AbortVMOnVMOperationTimeout: > > [3.117s][info][gc,start] GC(2) Pause Young (Normal) (G1 Evacuation Pause) > [3.224s][warning][vmthread] VM Operation G1CollectForAllocation took longer than 100 ms > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error (/home/sh/jdk-jdk/src/hotspot/share/runtime/vmThread.cpp:218), pid=2536, tid=2554 > # fatal error: VM Operation G1CollectForAllocation took longer than 100 ms > # It's not safe to access vm_safepoint_description() outside the VMThread as the _cur_vm_operation could be deleted while you're trying to access its name. Initially I thought this might be useful for tracking down excessively long VM ops, but with a global timeout it can't do that. And a per-op timeout would be rather tricky to pass through from the command-line (but easy enough to use once you had it). And as we don't have a general timer mechanism this has to use polling so you pay for a 10ms task wakeup regardless of how long the timeout is. Given the limitations of the global timeout I'm not sure I see a use for the non-aborting form. This could just reduce down to: -XX:AbortVMOpsAfter:NN otherwise I don't really think this carries its weight. Of course that's just my opinion. Interested to hear others. Cheers, David > Testing: hotspot/tier1, ad-hoc tests, jdk-submit (pending) > > Thanks, > -Aleksey > From david.holmes at oracle.com Mon Nov 19 06:31:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 19 Nov 2018 16:31:36 +1000 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> Message-ID: <2310d10b-64ec-5680-a780-71138252a259@oracle.com> Hi Kim, On 16/11/2018 12:31 pm, Kim Barrett wrote: >> On Oct 3, 2018, at 3:13 PM, Kim Barrett wrote: >> >> I've submitted a JEP for >> >> (1) enabling the use of C++14 Language Features when building the JDK, >> >> (2) define a process for deciding and documenting which new features >> can be used or are forbidden in HotSpot code, >> >> (3) provide an initial list of permitted and forbidden new features. >> >> https://bugs.openjdk.java.net/browse/JDK-8208089 > > While there has been some discussion of this JEP here, so far only Mikael has officially > reviewed or endorsed it. I added myself as a Reviewer. > Regarding the timing of this JEP, integration in time for JDK 12 seems unlikely. This doesn't strike me as a JEP that actually integrates anything. It wants to establish a process by which future C++ features are accepted for use with hotspot. That's just documentation on the OpenJDK wiki as far as I can see. The actual set of features and any "vote" on them can happen any time after the JEP is approved. Any actual use of those features can follow any time after that. I think it is important that all the port owners buy into this. Thanks, David From kim.barrett at oracle.com Mon Nov 19 07:04:14 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 19 Nov 2018 02:04:14 -0500 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <2310d10b-64ec-5680-a780-71138252a259@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <2310d10b-64ec-5680-a780-71138252a259@oracle.com> Message-ID: > On Nov 19, 2018, at 1:31 AM, David Holmes wrote: > > Hi Kim, > > On 16/11/2018 12:31 pm, Kim Barrett wrote: >>> On Oct 3, 2018, at 3:13 PM, Kim Barrett wrote: >>> >>> I've submitted a JEP for >>> >>> (1) enabling the use of C++14 Language Features when building the JDK, >>> >>> (2) define a process for deciding and documenting which new features >>> can be used or are forbidden in HotSpot code, >>> >>> (3) provide an initial list of permitted and forbidden new features. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8208089 >> While there has been some discussion of this JEP here, so far only Mikael has officially >> reviewed or endorsed it. > > I added myself as a Reviewer. Thanks. >> Regarding the timing of this JEP, integration in time for JDK 12 seems unlikely. > > This doesn't strike me as a JEP that actually integrates anything. It wants to establish a process by which future C++ features are accepted for use with hotspot. That's just documentation on the OpenJDK wiki as far as I can see. The actual set of features and any "vote" on them can happen any time after the JEP is approved. Any actual use of those features can follow any time after that. There is the integration of the build changes needed to enable the use of C++14. Presently we explicitly specify C++98 (actually, gnu++98) for gcc. We also presently permit the use of older versions of compilers that don?t support C++14 (or even C++11) at all, or only partially. Those need to be dropped. > I think it is important that all the port owners buy into this. At least one port (aix_ppc) presently seems to have no way to support this change, because the compiler being used is seriously deficient and appears to be languishing. (It doesn?t even support C++11, let alone C++14.) I think the community could (and in my opinion, should) chose to move ahead despite that. If a new and adequate compiler is expected ?soon? then the community might choose to wait, or might proceed and let that port languish until the new compiler is available. I think that?s all part of the discussion that should happen around the targeting of this JEP. I hope this inadequate compiler on a relatively niche platform won?t be an indefinite blocker in this area. From Yang.Zhang at arm.com Mon Nov 19 07:53:39 2018 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Mon, 19 Nov 2018 07:53:39 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 Message-ID: Hi, When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. compiler/c2/cr6340864/TestByteVect.java compiler/c2/cr6340864/TestIntVect.java compiler/c2/cr6340864/TestShortVect.java compiler/codegen/TestCharVect2.java Regards Yang From kim.barrett at oracle.com Mon Nov 19 07:58:20 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 19 Nov 2018 02:58:20 -0500 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <2310d10b-64ec-5680-a780-71138252a259@oracle.com> Message-ID: <0C0B3EA2-7DF1-480F-8280-190168A401C6@oracle.com> > On Nov 19, 2018, at 2:04 AM, Kim Barrett wrote: > >> On Nov 19, 2018, at 1:31 AM, David Holmes wrote: >> I think it is important that all the port owners buy into this. > > At least one port (aix_ppc) presently seems to have no way to support this change, because > the compiler being used is seriously deficient and appears to be languishing. (It doesn?t even > support C++11, let alone C++14.) I think the community could (and in my opinion, should) > chose to move ahead despite that. If a new and adequate compiler is expected ?soon? then > the community might choose to wait, or might proceed and let that port languish until the new > compiler is available. I think that?s all part of the discussion that should happen around the > targeting of this JEP. I hope this inadequate compiler on a relatively niche platform won?t be > an indefinite blocker in this area. Here?s what Volker said on the build-dev list, 2017-07-19, subject ?C++11/C++14 support in XLC ?? - SAP is currently maintaining the AIX port in the OpenJDK and we're willing to do that in the future. But we're not IBM and we can not decide about the XLC feature list. If Oracle and the OpenJDK community finally decide to use C++11/14 features which are not available in XLC we have to live with that. We can either escalate the XLC deficiencies to IBM and suspend the port until the compiler gets fixed. Or we can switch the port to use the GCC tool chain with all the pros (bigger compatibility with Linux platforms) and cons (porting effort, testing, compatibility with other AIX software compiled with XLC, compiler support). While the GCC alternative sounds very appealing at a first glance it really isn't that perfect in reality, especially not for our commercial SAP JVM version of OpenJDK. One problem is the fact that there's no official support for GCC on AIX, the other is compatibility. Just think you had to replace Solaris Studio by GCC on Solaris :) He had more to say on the general topic at the time. I don?t know if anything has changed in the intervening nearly 1 1/2 years. From volker.simonis at gmail.com Mon Nov 19 08:21:08 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 19 Nov 2018 09:21:08 +0100 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <0C0B3EA2-7DF1-480F-8280-190168A401C6@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <2310d10b-64ec-5680-a780-71138252a259@oracle.com> <0C0B3EA2-7DF1-480F-8280-190168A401C6@oracle.com> Message-ID: On Mon, Nov 19, 2018 at 9:00 AM Kim Barrett wrote: > > > On Nov 19, 2018, at 2:04 AM, Kim Barrett wrote: > > > >> On Nov 19, 2018, at 1:31 AM, David Holmes wrote: > >> I think it is important that all the port owners buy into this. > > > > At least one port (aix_ppc) presently seems to have no way to support this change, because > > the compiler being used is seriously deficient and appears to be languishing. (It doesn?t even > > support C++11, let alone C++14.) I think the community could (and in my opinion, should) > > chose to move ahead despite that. If a new and adequate compiler is expected ?soon? then > > the community might choose to wait, or might proceed and let that port languish until the new > > compiler is available. I think that?s all part of the discussion that should happen around the > > targeting of this JEP. I hope this inadequate compiler on a relatively niche platform won?t be > > an indefinite blocker in this area. > > Here?s what Volker said on the build-dev list, 2017-07-19, subject ?C++11/C++14 support in XLC ?? > > > - SAP is currently maintaining the AIX port in the OpenJDK and we're > willing to do that in the future. But we're not IBM and we can not > decide about the XLC feature list. If Oracle and the OpenJDK community > finally decide to use C++11/14 features which are not available in XLC > we have to live with that. We can either escalate the XLC deficiencies > to IBM and suspend the port until the compiler gets fixed. Or we can > switch the port to use the GCC tool chain with all the pros (bigger > compatibility with Linux platforms) and cons (porting effort, testing, > compatibility with other AIX software compiled with XLC, compiler > support). While the GCC alternative sounds very appealing at a first > glance it really isn't that perfect in reality, especially not for our > commercial SAP JVM version of OpenJDK. One problem is the fact that > there's no official support for GCC on AIX, the other is > compatibility. Just think you had to replace Solaris Studio by GCC on > Solaris :) > > > He had more to say on the general topic at the time. I don?t know if anything > has changed in the intervening nearly 1 1/2 years. > Not really. IBM is working on a new compiler which is in Beta now. But neither have I tested it nor do I know exactly if it will have full C++11/14 support. As I wrote before, this is a chicken/egg problem which I can not solve and I agree that it shouldn't be a blocker for OpenJDK. Therefore, please go ahead and use whichever feature the OpenJDK community agrees upon. Regards, Volker From robbin.ehn at oracle.com Mon Nov 19 08:35:10 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 19 Nov 2018 09:35:10 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: <8f52ae9c-e2e3-68ad-a56b-786af93ccee6@oracle.com> Hi Aleksey, You patch seems not to be against jdk/jdk (jdk12). Without the actual core file, I don't see the hs file being very useful containing the watcherthread stacktrace. It would be good if the 'killer' signaled the VM thread and we do the error reporting from the signal handler instead in VM thread context. If you are only looking at VM ops, it seem useless to wake-up when there is no VM op. You should start the timer when we start the safepoint and stop the timer when it ends. I had a patch which did: - 'killer thread' waits on semaphore. - VM thread post to 'killer thread' in ::begin. - 'killer thread' do a timewait on a the semaphore. - If VM thread manage to post on that semaphore in ::end before the timeout: - 'killer thread' do a normal wait again. - Else if timeout occurs: - 'killer thread' starts error reporting. You get gist of my thinking. /Robbin On 11/16/18 5:30 PM, Aleksey Shipilev wrote: > RFE: > https://bugs.openjdk.java.net/browse/JDK-8181143 > > Webrev: > http://cr.openjdk.java.net/~shade/8181143/webrev.03/ > > SafepointTimeout is nice to discover long/stuck safepoint syncs. But it is as important to discover > long/stuck VM operations. This patch complements the timeout machinery with tracking VM operation > themselves. Among other things, this allows to terminate the VM when very long VM operation is > blocking progress. High-availability users would enjoy fail-fast JVM -- in fact, the original > prototype was done as request from Apache Ignite developers. > > Example with -XX:+VMOperationTimeout -XX:VMOperationTimeoutDelay=100 -XX:+AbortVMOnVMOperationTimeout: > > [3.117s][info][gc,start] GC(2) Pause Young (Normal) (G1 Evacuation Pause) > [3.224s][warning][vmthread] VM Operation G1CollectForAllocation took longer than 100 ms > # > # A fatal error has been detected by the Java Runtime Environment: > # > # Internal Error (/home/sh/jdk-jdk/src/hotspot/share/runtime/vmThread.cpp:218), pid=2536, tid=2554 > # fatal error: VM Operation G1CollectForAllocation took longer than 100 ms > # > > Testing: hotspot/tier1, ad-hoc tests, jdk-submit (pending) > > Thanks, > -Aleksey > From shade at redhat.com Mon Nov 19 11:23:43 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 19 Nov 2018 12:23:43 +0100 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> Message-ID: <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> On 11/19/18 6:08 AM, David Holmes wrote: >> David suggested making the option "product", but I don't like it very much: "product" option >> suggests this is a production-grade feature and it comes with expectations of support, which is >> interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode available in >> product builds without the attached notion of support. Users that want fail-fast VM crash can then >> use that option on "we know what we are doing" basis. > > I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported > product". If you really want to fail-fast in production then you're looking for a production flag > not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. Yup, let's hear it. I don't want to bikeshed this too much :) To re-iterate: I don't really want to introduce the hard-to-retract product option. Rather, I would like to expose the flag that we are using for debugging into release builds, so it could both used by us for our release-mode debugging, and by power users who know what they are doing. It seems "diagnostic" hits the spot here. Also, symmetry against other Abort* flags... >> Fix: >> >> diff -r 9ad663e63da5 -r 132db6e99f77 src/hotspot/share/runtime/globals.hpp >> --- a/src/hotspot/share/runtime/globals.hpp??? Fri Nov 16 12:02:08 2018 +0100 >> +++ b/src/hotspot/share/runtime/globals.hpp??? Fri Nov 16 13:35:16 2018 +0100 >> @@ -498,7 +498,7 @@ >> ??????????? "Time out and warn or fail after SafepointTimeoutDelay "????????? \ >> ??????????? "milliseconds if failed to reach safepoint")????????????????????? \ >> ????????????????????????????????????????????????????????????????????????????? \ >> -? develop(bool, DieOnSafepointTimeout, false,?????????????????????????????? \ >> +? diagnostic(bool, AbortVMOnSafepointTimeout, false,??????????????????????? \ >> ??????????? "Die upon failure to reach safepoint (see SafepointTimeout)")???? \ > s/Die/Abort > > Thanks. Done in my local patch queue, thanks! -Aleksey From erik.osterlund at oracle.com Mon Nov 19 12:31:30 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Nov 2018 13:31:30 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> Message-ID: <5BF2AD22.70100@oracle.com> Hi Coleen, Thanks for having a look at this. New full webrev: http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ Incremental: http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: > > How about instead of two bool arguments that are hard to read and > error prone and need comments, like: > > 76 CompiledMethodIterator iter(true /* only_alive */, false /* > only_not_unloading */); > > > enum FilterAlive { all_blobs, only_alive }; > enum FilterUnloading { all_blobs, only_not_unloading }; > > Then it can be something like: > > CompiledMethodIterator iter(only_alive, all_blobs); > > Don't know if you can repeate all_blobs like that. You can't really have all_blobs in both. I tried out a variant with a single enum though, hope you like it. The observation was that we only need 3 different variations: all_blobs, only_alive or only_alive_and_not_unloading. So I made an enum out of that. In fact, these three modes are the only ones I support right now, so they are also the only valid options, which made the single enum look even more reasonable. > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html > > > I'm not sure if metadata_do() should filter out unloading. It's > looking for "old" methods to stop from deallocating memory for them > (but it's in a safepoint). If you're unloading, the IC caches are > already cleaned out? IC caches to unloading nmethods are not necessarily cleaned out. The concurrent GC threads will clean the IC caches concurrently, but a JavaThread can call the IC cache before the GC has gotten around to cleaning the IC cache. But any such call into an is_unloading() nmethod will be trapped by nmethod entry barriers, which will call the IC miss handler, which sorts things out lazily. So you can view this as the IC caches to is_unloading() nmethods being logically cleared, but not necessarily physically cleared yet. The effect is the same - the is_unloading() nmethods may not be entered from anywhere, including IC caches. So if you were to not filter is_unloading(), you would get a whole bunch of nmethods that are not really reachable by the application, i.e. they are not on any stacks, and may not be entered by the application. Therefore, I think it's perfectly fine to filter them out here, unless I missed something. And this makes the behaviour closer to as-if the nmethods were indeed unloaded in the safepoint. > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html > > > 1108 // The release is only needed for compile-time ordering, as accesses > 1109 // into the nmethod after the store is not safe, due to the sweeper > 1110 // being allowed to free it then, in the case of concurrent nmethod > 1111 // unloading. Therefore, there is no need for acquire on the > loader side. > 1112 OrderAccess::release_store(&_state, (signed char)unloaded); > > I tried to make sense out of this first sentence, but couldn't > really. After the store to unloaded, can the sweeper free the > nmethod? maybe remove "then, " and it would make more sense? Right, after the store is observed, the sweeper thread may concurrently observe the nmethod to be unloaded, and then it may immediately flip it over to zombie. And I don't want any concurrent make_unloaded code in that nmethod to still be racing after the sweeper turns it into zombie; a very unnecessary race to deal with. I updated the comment - hopefully it reads better now. Thanks, /Erik > Thanks, > Coleen > > On 11/12/18 5:46 PM, Erik ?sterlund wrote: >> ..put in bug number in subject to make mail filters happy. >> >> /Erik >> >> On 2018-11-12 23:44, Erik ?sterlund wrote: >>> Hi, >>> >>> All current GCs perform code cache unloading in safepoints. >>> Therefore, nmethods that are alive but is_unloading() are never >>> observed outside of safepoints. With concurrent class unloading, >>> nmethods that are alive but is_unloading() will become observable >>> outside of safepoints. This must be handled appropriately. >>> >>> In this patch I changed the nmethod/compiled method iterators to >>> accept parameters describing whether they should filter out not >>> is_alive() or is_unloading() methods. Since there is no obvious >>> default (all combinations are used depending on call site), you have >>> to explicitly set what iteration mode you want. >>> >>> Other than that, I make sure that the sweeper stays away from >>> is_unloading() nmethods that are not yet is_unloaded(). To make the >>> interactions between the sweeper and concurrent GC threads safe, I >>> had to move down the store that sets the state to unloaded, and use >>> a release_store there, to make sure no accesses float below it at >>> compile-time. Once that store is observed, nmethods may be deleted. >>> >>> In the IC miss handler, I also need to lazily clean stale IC caches >>> due to calling is_unloading nmethods using nmethod entry barriers. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>> >>> Thanks, >>> /Erik > From erik.osterlund at oracle.com Mon Nov 19 12:52:02 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Nov 2018 13:52:02 +0100 Subject: RFR: 8213209: [REDO] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: References: Message-ID: <5BF2B1F2.8070004@oracle.com> Hi Coleen, Thanks for the review. Full webrev: http://cr.openjdk.java.net/~eosterlund/8213209/webrev.01/ Incremental webrev: http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00_01/ On 2018-11-16 22:08, coleen.phillimore at oracle.com wrote: > > Erik, thank you for taking this change and making it lock-free. > > On 11/16/18 1:12 PM, Erik ?sterlund wrote: >> Hi, >> >> This is a redo of Coleen's earlier patch to clean weak metadata links >> using the Compile_lock. >> >> We noticed the proposed solution could cause deadlocks. At least one >> of the reasons for that is that the Compile_lock was taken during >> printing and verification in safepoints. And the Compile_lock is >> sometimes held while issuing safepoints. The latter problem also >> forces us to take the Compile_lock before joining the suspendible >> thread set, to avoid another deadlock situation. But that is really >> nasty for other reasons. And there have been a few more problems as >> well. >> >> So in this case, we think it is both safer, less error prone, and >> better performing (although less important) to perform this cleaning >> in a lock-free fashion rather than trying to dodge all the issues >> related to the locking protocol. >> >> Inserts still require a lock. Therefore, we are dealing with >> lock-free reads, and lock-free cleaning. The lock-free cleaning is >> interacting with multiple lock-free readers and a single writer. >> >> The single inserter always helps with cleaning the subclass list head >> before prepending new nodes to the chain. That allows an invariant >> that the siblings link is never inserted pointing at a Klass that is >> unloading, which simplifies things a lot. The head is inserted in a >> classic load + CAS in a loop. >> >> Readers require load_acquire when reading the head, due to competing >> inserts putting new Klass pointers there. The siblings, however, only >> need relaxed consistency, because they are linked to data strictly >> older than the head, which has already been acquired. >> >> Unlinked entries are all inserted into a purge list, because freeing >> them immediately is not safe. A global handshake operation is >> performed, and after that ClassLoaderDataGraph::purge() will go >> through the list and delete the entries safely. > > I don't see the purge list in your change, and it's not needed since > this is unlinking in place. I think this is your other change you're > thinking of. Yes you are right, of course. It's the unloading list and not a separate purge list... > Should the Compile_lock be removed in ciInstanceKlass.cpp > ciInstanceKlass* ciInstanceKlass::implementor() { > > and in jvmciCompilerToVM.cpp > C2V_VMENTRY(jobject, getImplementor, (JNIEnv *, jobject, jobject > jvmci_type)) > > The former does a bit more than calling implementor(). Or do you want > to leave these locks to be conservative? While I think your analysis is correct, I would indeed like to leave them in this RFE to be conservative. > JVMCIKlassHandle handle(THREAD); > { > // Need Compile_lock around implementor() > MutexLocker locker(Compile_lock); > handle = iklass->implementor(); > } > > > http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/oops/instanceKlass.cpp.udiff.html > > > nof_implementors() has an assert_locked_or_safepoint(Compile_lock); > which it no longer needs. You'll hit this from printing because > you've removed the locking here: > > 3093 if (is_interface()) { > 3094 MutexLocker ml(Compile_lock); > 3095 st->print_cr(BULLET"nof implementors: %d", nof_implementors()); > 3096 if (nof_implementors() == 1) { > 3097 st->print_cr(BULLET"implementor: "); > 3098 st->print(" "); > 3099 implementor()->print_value_on(st); > 3100 st->cr(); > 3101 } > > > We don't test printing very well. > Ahh yes - good catch. I removed the assert as it is no longer required. > http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/utilities/vmError.cpp.udiff.html > > > This file is unchanged except a line removed. I put it back the way it was before. > This change looks good. I've run through the tests with it and didn't > find any problems that doomed my earlier version. Thank you for running additional tests on this patch! Thanks, /Erik > Thanks!! > Coleen >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8213209 >> >> Thanks, >> /Erik > From david.holmes at oracle.com Mon Nov 19 12:56:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 19 Nov 2018 22:56:54 +1000 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <2310d10b-64ec-5680-a780-71138252a259@oracle.com> Message-ID: <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com> On 19/11/2018 5:04 pm, Kim Barrett wrote: >> On Nov 19, 2018, at 1:31 AM, David Holmes wrote: >> >> Hi Kim, >> >> On 16/11/2018 12:31 pm, Kim Barrett wrote: >>>> On Oct 3, 2018, at 3:13 PM, Kim Barrett wrote: >>>> >>>> I've submitted a JEP for >>>> >>>> (1) enabling the use of C++14 Language Features when building the JDK, >>>> >>>> (2) define a process for deciding and documenting which new features >>>> can be used or are forbidden in HotSpot code, >>>> >>>> (3) provide an initial list of permitted and forbidden new features. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8208089 >>> While there has been some discussion of this JEP here, so far only Mikael has officially >>> reviewed or endorsed it. >> >> I added myself as a Reviewer. > > Thanks. > >>> Regarding the timing of this JEP, integration in time for JDK 12 seems unlikely. >> >> This doesn't strike me as a JEP that actually integrates anything. It wants to establish a process by which future C++ features are accepted for use with hotspot. That's just documentation on the OpenJDK wiki as far as I can see. The actual set of features and any "vote" on them can happen any time after the JEP is approved. Any actual use of those features can follow any time after that. > > There is the integration of the build changes needed to enable the use of C++14. > Presently we explicitly specify C++98 (actually, gnu++98) for gcc. > We also presently permit the use of older versions of compilers that don?t support C++14 > (or even C++11) at all, or only partially. Those need to be dropped. Even so those changes don't need to be put in place until we actually need to start using a new feature. So I would still think we can finalize the JEP independently of the subsequent code changes. >> I think it is important that all the port owners buy into this. > > At least one port (aix_ppc) presently seems to have no way to support this change, because > the compiler being used is seriously deficient and appears to be languishing. (It doesn?t even > support C++11, let alone C++14.) I think the community could (and in my opinion, should) > chose to move ahead despite that. Okay I've seen the mail on this - thanks. Cheers, David > If a new and adequate compiler is expected ?soon? then > the community might choose to wait, or might proceed and let that port languish until the new > compiler is available. I think that?s all part of the discussion that should happen around the > targeting of this JEP. I hope this inadequate compiler on a relatively niche platform won?t be > an indefinite blocker in this area. > > > From erik.osterlund at oracle.com Mon Nov 19 14:09:08 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Nov 2018 15:09:08 +0100 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <55e11251-f8da-aeed-8372-6a26efbb7822@oracle.com> References: <5BCF3766.3050504@oracle.com> <5BD032C7.1000807@oracle.com> <55e11251-f8da-aeed-8372-6a26efbb7822@oracle.com> Message-ID: <5BF2C404.9030104@oracle.com> Hi Dean, May I interpret your commentary as Reviewed? Thanks, /Erik On 2018-10-24 22:20, dean.long at oracle.com wrote: > On 10/24/18 1:52 AM, Erik ?sterlund wrote: >> Hi Dean, >> >> On 2018-10-23 22:27, dean.long at oracle.com wrote: >>> Can allocate_objArray_klass() end up calling >>> SystemDictionary::add_to_hierarchy()? If so, then you'll end up >>> locking Compile_lock anway, but in the wrong order with respect to >>> MultiArray_lock. >> >> No, I can't see that allocate_objArray_klass() ever calls >> SystemDictionary::add_to_hierarchy(). If it did, we would assert that >> the Compile_lock is held; it never re-acquires the lock. >> > > OK, I forgot that mutex's can't be locked recursively. > > I found a clue to the reason for original 1998 change. This was also > added at the same time, to klassVtable::initialize_vtable(): > > // We need the compile lock in order to force the super vtables and > // methodOop code pointers to stay constant. > // (Race scenario: compiler thread updates a vtable entry while > we're copying > // down entries from the superclass vtable.) > assert_lock(Compile_lock); > > By 1999, the vtable code changed a lot and the assert above went away, > so it was probably just an oversight that the extra uses of > Compile_lock weren't removed at the same time. > > dl > >> Thanks, >> /Erik >> >>> dl >>> >>> On 10/23/18 7:59 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> We occasionally blockingly wait for GC in allocations in >>>> ObjArrayKlass::allocate_objArray_klass(), while holding the >>>> Compile_lock. >>>> This is problematic for concurrent class unloading that needs to >>>> hold that lock in the unloading path. This introduces a potential >>>> deadlock situation. >>>> >>>> After staring enough at this code together with help from Coleen, I >>>> have convinced myself that the Compile_lock is in fact not needed >>>> in this path, and there is nothing to my knowledge it actually >>>> protects here. The vague comment next to the lock suggests its >>>> purpose is to protect vtable stubs. But it doesn't. There is a >>>> VtableStubs_lock for that, and I find no traces if the Compile_lock >>>> having anything to do with that. >>>> >>>> Coleen helped me trying to trace down where this locking code came >>>> from. It takes us back to before anyone was around, and does not >>>> seem to have changed in the past 2 decades, and its origins are a >>>> bit unclear. The theory is that perhaps vtable stubs used to be >>>> protected by the Compile_lock in ancient times, and the locking >>>> code is still around because nobody dared to touch it. >>>> >>>> Since both code analysis and mach5 suggest there is no reason for >>>> this lock to be held here, I propose to remove it with this patch. >>>> >>>> If anyone that has been around for more than 2 decades in HotSpot >>>> happens to know the reason why this locking was introduced, any >>>> commentary around that would be very appreciated. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8212682 >>>> >>>> Thanks, >>>> /Erik >>> >> > From daniel.daugherty at oracle.com Mon Nov 19 14:43:27 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 19 Nov 2018 09:43:27 -0500 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> Message-ID: On 11/19/18 6:23 AM, Aleksey Shipilev wrote: > On 11/19/18 6:08 AM, David Holmes wrote: >>> David suggested making the option "product", but I don't like it very much: "product" option >>> suggests this is a production-grade feature and it comes with expectations of support, which is >>> interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode available in >>> product builds without the attached notion of support. Users that want fail-fast VM crash can then >>> use that option on "we know what we are doing" basis. >> I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported >> product". If you really want to fail-fast in production then you're looking for a production flag >> not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. > Yup, let's hear it. I don't want to bikeshed this too much :) If I was doing the work on this bug, I would have made it a diagnostic option also. I have a bias towards adding options as diagnostic for two reasons: 1) Easier to get rid of the option if it's a bad idea. 2) You have to use the '-XX:UnlockDiagnosticVMOption' option to use it ?? which makes unintentional use more difficult. Dan > To re-iterate: I don't really want to introduce the hard-to-retract product option. Rather, I would > like to expose the flag that we are using for debugging into release builds, so it could both used > by us for our release-mode debugging, and by power users who know what they are doing. It seems > "diagnostic" hits the spot here. Also, symmetry against other Abort* flags... > >>> Fix: >>> >>> diff -r 9ad663e63da5 -r 132db6e99f77 src/hotspot/share/runtime/globals.hpp >>> --- a/src/hotspot/share/runtime/globals.hpp??? Fri Nov 16 12:02:08 2018 +0100 >>> +++ b/src/hotspot/share/runtime/globals.hpp??? Fri Nov 16 13:35:16 2018 +0100 >>> @@ -498,7 +498,7 @@ >>> ??????????? "Time out and warn or fail after SafepointTimeoutDelay "????????? \ >>> ??????????? "milliseconds if failed to reach safepoint")????????????????????? \ >>> ????????????????????????????????????????????????????????????????????????????? \ >>> -? develop(bool, DieOnSafepointTimeout, false,?????????????????????????????? \ >>> +? diagnostic(bool, AbortVMOnSafepointTimeout, false,??????????????????????? \ >>> ??????????? "Die upon failure to reach safepoint (see SafepointTimeout)")???? \ >> s/Die/Abort >> >> Thanks. > Done in my local patch queue, thanks! > > -Aleksey > > From erik.osterlund at oracle.com Mon Nov 19 15:27:52 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Nov 2018 16:27:52 +0100 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod Message-ID: <5BF2D678.3030208@oracle.com> Hi, A GC might want extra GC-related information attached to a CompiledMethod. I propose to introduce a type-erased _gc_data member, with accessors. This allows ZGC to stick in some data we need to keep track of concurrent class unloading. Webrev: http://cr.openjdk.java.net/~eosterlund/8214056/webrev.00/ Bug ID: https://bugs.openjdk.java.net/browse/JDK-8214056 Thanks, /Erik From coleen.phillimore at oracle.com Mon Nov 19 15:31:27 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 19 Nov 2018 10:31:27 -0500 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> Message-ID: <94d8c4ea-3cc6-e2d5-8cb9-df621b353014@oracle.com> Hi, Jiangli - Good find that the blocks leak!? None of the tables that used blocks were ever deallocated, before this one.? It might make sense for all the tables to use blocks with this change, although the blocks might be too big for some of the smaller hashtables. Ioi, his change and the deltas look really good. Thanks, Coleen On 11/18/18 10:50 PM, Ioi Lam wrote: > > > On 11/17/18 10:28 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> On 11/17/18 6:48 PM, Ioi Lam wrote: >> >>> >>> >>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> This looks good. Glad to see the change that utilizes the existing >>>> BasicHashtable. Thanks Coleen for the suggestion! >>>> >>>> To avoid changing the existing assumptions about BasicHashtable and >>>> Hashtable, how about adding a destructor to the new KVHashtable >>>> instead of BasicHashtable? Just did a quick test, which seems to >>>> work properly. >>>> >>> Hi Jiangli, >>> >>> Thanks for the review. >>> >>> Not freeing the entries when a BasicHashtable is destructed is a >>> bug. None of the existing BasicHashtables (and subclasses thereof) >>> are ever destructed, so I don't think my code would impact them. >>> Also, if any code depends on the entries not freed even if the table >>> is destructed, that's clearly a bug in that code, and it should be >>> fixed. >>> >>> If I don't add the destructor to BasicHashtable, the next person who >>> wants to allocate/free BasicHashtables will run in to the same issue. >> >> Dictionary, PackageEntryTable, ModuleEntryTable and >> G1CodeRootSetTable are derived from Hashtable and they call >> free_buckets() in their destructors. So free_buckets() will be called >> twice when those tables are freed although it's harmless. If you want >> to avoid the duplicated call, you could remove the free_buckets() >> calls from those classes' destructors and keep the current >> BasicHashtables changes. I don't have a strong opinion on this, I'll >> leave it to you to decide. >> >> > > Hi Jiangli, > > Good catch. I've remove the call to free_buckets() in the destructor > of those 4 classes. Here's the delta from the last webrev: > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ > > > Also, I found 2 problems while doing this. Hashtables in HotSpot are > really a mess. > > https://bugs.openjdk.java.net/browse/JDK-8214029 > https://bugs.openjdk.java.net/browse/JDK-8214030 > > Thanks > - Ioi > >> Thanks, >> >> Jiangli >>> >>> Thanks >>> - Ioi >>> >>> >>>> Thanks, >>>> >>>> Jiangli >>>> >>>> >>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> I deleted the default value for MEMFLAGS as you suggested. For my >>>>> instantiated templates, I still use mtInternal, though, since I >>>>> can't find anything better for using at CDS dump time. >>>>> >>>>> Also, Jiangli noted that there's a memory leak, because I allocate >>>>> and free the KVHashtable dynamically. So I added a destructor to >>>>> BasicHashtable to free the buckets and the block-allocated entries. >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>> >>>>> >>>>> This comment around Hashtable::allocate_new_entry() is wrong now >>>>> -- "The allocator in blocks is preferable but doesn't have free >>>>> semantics". Maybe I should change it to this? >>>>> >>>>> "The block allocator in BasicHashtable has less fragmentation, but >>>>> the memory is not freed until the whole table is freed. Use >>>>> allocate_new_entry() if you want to immediately free the memory >>>>> used by each entry". >>>>> >>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also >>>>> tested the solaris/x64 build since it seems to break every time >>>>> you do something with templates :-( >>>>> >>>>> Thanks! >>>>> - Ioi >>>>> >>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> Hi Ioi,? I really like this new wrapper on the old hashtable to >>>>>> not have to write the boilerplate code! >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>> >>>>>> >>>>>> + MEMFLAGS F = mtInternal, >>>>>> >>>>>> >>>>>> I think you should require the mt type though and not make it a >>>>>> default parameter. mtInternal is not very useful to finding >>>>>> memory leaks. >>>>>> >>>>>> Apart from this (which I don't need to see another version), your >>>>>> change looks good and nice to get good performance benefits from >>>>>> this. >>>>>> >>>>>> thanks, >>>>>> Coleen >>>>>> >>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>> Coleen pointed out to me off-line that the good old (and ugly) >>>>>>> BasicHashtable already supports resizing. I think that might be >>>>>>> a better starting point for this RFE: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>> >>>>>>> >>>>>>> I wrote a new template class called "KVHashtable" (copying the >>>>>>> style from ResourceHashtable). That way, you can instantiate >>>>>>> different (Key -> Value) mappings without writing tons of >>>>>>> boilerplate code. The performance is similar to my previous >>>>>>> version, and the code is much cleaner. >>>>>>> >>>>>>> I also renamed the RFE title, as well as the subject line of >>>>>>> this RFR e-mail. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>> Hi Ioi, >>>>>>>>>> >>>>>>>>>> The change looks reasonable to me in general. It would be >>>>>>>>>> helpful to see the performance difference with the expendable >>>>>>>>>> table. Do you have any data when large number of classes are >>>>>>>>>> loaded (>20000)? How much saving does it provide? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>> >>>>>>>>> BEFORE: 93.971 sec >>>>>>>>> AFTER:? 34.761 sec >>>>>>>> >>>>>>>> Thanks for the data! That's about 2.6x improvement with large >>>>>>>> set of classes. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the >>>>>>>>>>> table size to be >>>>>>>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't know much about C++ templates, so my attempt on >>>>>>>>>>> doing this may be >>>>>>>>>>> ill-advised. >>>>>>>>>>> >>>>>>>>>>> I *think* that with my patch, the performance of existing >>>>>>>>>>> code, which uses >>>>>>>>>>> a statically-defined SIZE,? should not be affected, as the >>>>>>>>>>> C++ compiler >>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>> >>>>>>>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>> ????? return SIZE; >>>>>>>>>>> ??? } else { >>>>>>>>>>> ????? return _configured_table_size; >>>>>>>>>>> ??? } >>>>>>>>>>> ? } >>>>>>>>>>> >>>>>>>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>> ????? return (Node**)(&_static_table[0]); >>>>>>>>>>> ??? } else { >>>>>>>>>>> ????? return _configured_table; >>>>>>>>>>> ??? } >>>>>>>>>>> ? } >>>>>>>>>>> >>>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>> ??? unsigned index = hash % size(); <----- >>>>>>>>>>> ??? Node** table = get_table(); >>>>>>>>>>> ??? Node** ptr = &table[index]; <----- >>>>>>>>>>> >>>>>>>>>>> back to the old code: >>>>>>>>>>> >>>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>> ??? unsigned index = hash % SIZE; <----- >>>>>>>>>>> ??? Node** ptr = &_table[index]; <----- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> - Ioi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From shade at redhat.com Mon Nov 19 15:43:06 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 19 Nov 2018 16:43:06 +0100 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: <5BF2D678.3030208@oracle.com> References: <5BF2D678.3030208@oracle.com> Message-ID: <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> On 11/19/18 4:27 PM, Erik ?sterlund wrote: > Hi, > > A GC might want extra GC-related information attached to a CompiledMethod. > I propose to introduce a type-erased _gc_data member, with accessors. > This allows ZGC to stick in some data we need to keep track of concurrent class unloading. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8214056/webrev.00/ This looks generally useful. Patch looks good. Having the type-checked abstraction would be nicer, of course, but type-erased member would also do. -Aleksey From erik.osterlund at oracle.com Mon Nov 19 16:08:50 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Nov 2018 17:08:50 +0100 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> References: <5BF2D678.3030208@oracle.com> <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> Message-ID: <5BF2E012.5010701@oracle.com> Hi Aleksey, Thanks for the review. I liked your idea of having the casting done by the accessors, like we did for GC data on Thread. So here is a new webrev with that approach: http://cr.openjdk.java.net/~eosterlund/8214056/webrev.01/ Thanks, /Erik On 2018-11-19 16:43, Aleksey Shipilev wrote: > On 11/19/18 4:27 PM, Erik ?sterlund wrote: >> Hi, >> >> A GC might want extra GC-related information attached to a CompiledMethod. >> I propose to introduce a type-erased _gc_data member, with accessors. >> This allows ZGC to stick in some data we need to keep track of concurrent class unloading. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8214056/webrev.00/ > This looks generally useful. Patch looks good. Having the type-checked abstraction would be nicer, > of course, but type-erased member would also do. > > -Aleksey > From gerard.ziemski at oracle.com Mon Nov 19 16:11:23 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Mon, 19 Nov 2018 10:11:23 -0600 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> Message-ID: Hi Ioi, Very nice! I only have couple of questions. #1 Regarding ?maybe_grow()? in src/hotspot/share/utilities/hashtable.hpp: 235 // Grow the number of buckets if the average entries per bucket is over the load_factor 236 bool maybe_grow(int load_factor = 8) { 237 if (number_of_entries() / table_size() > load_factor) { 238 resize(table_size() * 2); 239 return true; 240 } else { 241 return false; 242 } 243 } a) A suggestion: from my personal experience this didn?t make a difference, but technically a hash map benefits from a size that is a prime number. Maybe you could make a comparison run, with the code as is vs with prime number when resizing and see if you get an even better performance? b) I am a little worried that we don?t have an upper bound here. In our other hash table we always have upper bounds (or don?t resize). c) Is this table only used from one thread? We are guaranteed here that no other thread will insert/delete anything while we resize? Cheers > On Nov 18, 2018, at 9:50 PM, Ioi Lam wrote: > > > > On 11/17/18 10:28 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> On 11/17/18 6:48 PM, Ioi Lam wrote: >> >>> >>> >>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> This looks good. Glad to see the change that utilizes the existing BasicHashtable. Thanks Coleen for the suggestion! >>>> >>>> To avoid changing the existing assumptions about BasicHashtable and Hashtable, how about adding a destructor to the new KVHashtable instead of BasicHashtable? Just did a quick test, which seems to work properly. >>>> >>> Hi Jiangli, >>> >>> Thanks for the review. >>> >>> Not freeing the entries when a BasicHashtable is destructed is a bug. None of the existing BasicHashtables (and subclasses thereof) are ever destructed, so I don't think my code would impact them. Also, if any code depends on the entries not freed even if the table is destructed, that's clearly a bug in that code, and it should be fixed. >>> >>> If I don't add the destructor to BasicHashtable, the next person who wants to allocate/free BasicHashtables will run in to the same issue. >> >> Dictionary, PackageEntryTable, ModuleEntryTable and G1CodeRootSetTable are derived from Hashtable and they call free_buckets() in their destructors. So free_buckets() will be called twice when those tables are freed although it's harmless. If you want to avoid the duplicated call, you could remove the free_buckets() calls from those classes' destructors and keep the current BasicHashtables changes. I don't have a strong opinion on this, I'll leave it to you to decide. >> >> > > Hi Jiangli, > > Good catch. I've remove the call to free_buckets() in the destructor of those 4 classes. Here's the delta from the last webrev: > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ > > Also, I found 2 problems while doing this. Hashtables in HotSpot are really a mess. > > https://bugs.openjdk.java.net/browse/JDK-8214029 > https://bugs.openjdk.java.net/browse/JDK-8214030 > > Thanks > - Ioi > >> Thanks, >> >> Jiangli >>> >>> Thanks >>> - Ioi >>> >>> >>>> Thanks, >>>> >>>> Jiangli >>>> >>>> >>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> I deleted the default value for MEMFLAGS as you suggested. For my instantiated templates, I still use mtInternal, though, since I can't find anything better for using at CDS dump time. >>>>> >>>>> Also, Jiangli noted that there's a memory leak, because I allocate and free the KVHashtable dynamically. So I added a destructor to BasicHashtable to free the buckets and the block-allocated entries. >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>> >>>>> This comment around Hashtable::allocate_new_entry() is wrong now -- "The allocator in blocks is preferable but doesn't have free semantics". Maybe I should change it to this? >>>>> >>>>> "The block allocator in BasicHashtable has less fragmentation, but the memory is not freed until the whole table is freed. Use allocate_new_entry() if you want to immediately free the memory used by each entry". >>>>> >>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested the solaris/x64 build since it seems to break every time you do something with templates :-( >>>>> >>>>> Thanks! >>>>> - Ioi >>>>> >>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> Hi Ioi, I really like this new wrapper on the old hashtable to not have to write the boilerplate code! >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>> >>>>>> + MEMFLAGS F = mtInternal, >>>>>> >>>>>> >>>>>> I think you should require the mt type though and not make it a default parameter. mtInternal is not very useful to finding memory leaks. >>>>>> >>>>>> Apart from this (which I don't need to see another version), your change looks good and nice to get good performance benefits from this. >>>>>> >>>>>> thanks, >>>>>> Coleen >>>>>> >>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>> Coleen pointed out to me off-line that the good old (and ugly) BasicHashtable already supports resizing. I think that might be a better starting point for this RFE: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>> >>>>>>> I wrote a new template class called "KVHashtable" (copying the style from ResourceHashtable). That way, you can instantiate different (Key -> Value) mappings without writing tons of boilerplate code. The performance is similar to my previous version, and the code is much cleaner. >>>>>>> >>>>>>> I also renamed the RFE title, as well as the subject line of this RFR e-mail. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>> Hi Ioi, >>>>>>>>>> >>>>>>>>>> The change looks reasonable to me in general. It would be helpful to see the performance difference with the expendable table. Do you have any data when large number of classes are loaded (>20000)? How much saving does it provide? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>> >>>>>>>>> BEFORE: 93.971 sec >>>>>>>>> AFTER: 34.761 sec >>>>>>>> >>>>>>>> Thanks for the data! That's about 2.6x improvement with large set of classes. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>> >>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table size to be >>>>>>>>>>> dynamically specified when the table is constructed. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *** C++ template guru alert *** >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't know much about C++ templates, so my attempt on doing this may be >>>>>>>>>>> ill-advised. >>>>>>>>>>> >>>>>>>>>>> I *think* that with my patch, the performance of existing code, which uses >>>>>>>>>>> a statically-defined SIZE, should not be affected, as the C++ compiler >>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>> >>>>>>>>>>> ALWAYSINLINE unsigned size() const { >>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>> return SIZE; >>>>>>>>>>> } else { >>>>>>>>>>> return _configured_table_size; >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> ALWAYSINLINE Node** get_table() const { >>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>> return (Node**)(&_static_table[0]); >>>>>>>>>>> } else { >>>>>>>>>>> return _configured_table; >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>> unsigned index = hash % size(); <----- >>>>>>>>>>> Node** table = get_table(); >>>>>>>>>>> Node** ptr = &table[index]; <----- >>>>>>>>>>> >>>>>>>>>>> back to the old code: >>>>>>>>>>> >>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>> unsigned index = hash % SIZE; <----- >>>>>>>>>>> Node** ptr = &_table[index]; <----- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> - Ioi From sgehwolf at redhat.com Mon Nov 19 16:23:43 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 19 Nov 2018 17:23:43 +0100 Subject: [11u] RFR: 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: <5cd90d21-5fa8-4289-c242-5cca6f9ead8e@oracle.com> References: <5cd90d21-5fa8-4289-c242-5cca6f9ead8e@oracle.com> Message-ID: <675c77930b9ba3c7f724c74f9372582a9080b81e.camel@redhat.com> On Fri, 2018-11-16 at 10:11 -0500, Lois Foltan wrote: > On 11/16/2018 10:09 AM, Severin Gehwolf wrote: > > > On Thu, 2018-11-15 at 20:09 +0100, Severin Gehwolf wrote: > > > Hi, > > > > > > Please review this 11u backport for JDK-8212937. While the JDK 12 patch > > > applies cleanly it doesn't quite do the right thing. The JDK 12 test > > > would still crash the JVM due to the lack of the changes in > > > src/hotspot/share/classfile/javaClasses.cpp as describe_external is > > > being used in JDK 11. That function has been removed with JDK- > > > 8205611[1] in JDK 12. > > > > > > In addition, the exception messages aren't the same between JDK 12 and > > > JDK 11, so Test.java needed an update to account for that. Also due to > > > lack of JDK-8205611[1] in JDK 11. Everything else is the same as for > > > JDK 12. > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212937 > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8212937/jdk11/webrev.01/ > > > > > > Testing: Added regression test, internal test. > > > > Also: > > > > tier1 testing showed no regressions with this on linux x86_64. > > Hi Severin, > > Thank you for doing this testing. I think your change looks good! Thanks, Lois. Could I get a second review for this, please? Thanks, Severin > > > > > Thanks, > > Severin > > > > > Thoughts? > > > > > > Thanks, > > > Severin > > > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8205611 > > From adinn at redhat.com Mon Nov 19 16:34:34 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 19 Nov 2018 16:34:34 +0000 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: <5BF2E012.5010701@oracle.com> References: <5BF2D678.3030208@oracle.com> <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> <5BF2E012.5010701@oracle.com> Message-ID: <9db0d59e-e8ef-1a60-f62a-4569177de23c@redhat.com> On 19/11/2018 16:08, Erik ?sterlund wrote: > Thanks for the review. I liked your idea of having the casting done by > the accessors, like we did for GC data on Thread. > So here is a new webrev with that approach: > http://cr.openjdk.java.net/~eosterlund/8214056/webrev.01/ Yes, that's a lot nicer. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From erik.osterlund at oracle.com Mon Nov 19 16:36:16 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Nov 2018 17:36:16 +0100 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: <9db0d59e-e8ef-1a60-f62a-4569177de23c@redhat.com> References: <5BF2D678.3030208@oracle.com> <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> <5BF2E012.5010701@oracle.com> <9db0d59e-e8ef-1a60-f62a-4569177de23c@redhat.com> Message-ID: <5BF2E680.8000509@oracle.com> Hi Andrew, Thanks for the review. /Erik On 2018-11-19 17:34, Andrew Dinn wrote: > On 19/11/2018 16:08, Erik ?sterlund wrote: >> Thanks for the review. I liked your idea of having the casting done by >> the accessors, like we did for GC data on Thread. >> So here is a new webrev with that approach: >> http://cr.openjdk.java.net/~eosterlund/8214056/webrev.01/ > Yes, that's a lot nicer. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From aph at redhat.com Mon Nov 19 17:50:14 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 19 Nov 2018 17:50:14 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: Message-ID: <7da0518c-f531-a91a-eb52-958bb7c40967@redhat.com> On 11/19/18 7:53 AM, Yang Zhang (Arm Technology China) wrote: > Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ I've got a test failure: rerun: cd /home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/test-support/jtreg_test_hotspot_jtreg_compiler_c2_cr6340864/scratch/2 && \ HOME=/home/aph \ JDK8_HOME=/home/aph/jdk-11.0.1+13 \ LANG=en_US.UTF-8 \ LC_ALL=en_US.UTF-8 \ LC_CTYPE=C \ PATH=/bin:/usr/bin \ TEST_IMAGE_GRAAL_DIR=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/images/test/hotspot/jtreg/graal \ XMODIFIERS=@im=none \ CLASSPATH=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/test-support/jtreg_test_hotspot_jtreg_compiler_c2_cr6340864/classes/2/compiler/c2/cr6340864/TestIntVect.d:/home/aph/jdk-jdk/test/hotspot/jtreg/compiler/c2/cr6340864:/home/aph/jtreg/lib/javatest.jar:/home/aph/jtreg/lib/jtreg.jar \ /home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/jdk/bin/java \ -Dtest.class.path.prefix=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/test-support/jtreg_test_hotspot_jtreg_compiler_c2_cr6340864/classes/2/compiler/c2/cr6340864/TestIntVect.d:/home/aph/jdk-jdk/test/hotspot/jtreg/compiler/c2/cr6340864 \ -Dtest.src=/home/aph/jdk-jdk/test/hotspot/jtreg/compiler/c2/cr6340864 \ -Dtest.src.path=/home/aph/jdk-jdk/test/hotspot/jtreg/compiler/c2/cr6340864 \ -Dtest.classes=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/test-support/jtreg_test_hotspot_jtreg_compiler_c2_cr6340864/classes/2/compiler/c2/cr6340864/TestIntVect.d \ -Dtest.class.path=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/test-support/jtreg_test_hotspot_jtreg_compiler_c2_cr6340864/classes/2/compiler/c2/cr6340864/TestIntVect.d \ -Dtest.vm.opts=-XX:MaxRAMPercentage=2 \ -Dtest.tool.vm.opts=-J-XX:MaxRAMPercentage=2 \ -Dtest.compiler.opts= \ -Dtest.java.opts= \ -Dtest.jdk=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/jdk \ -Dcompile.jdk=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/jdk \ -Dtest.timeout.factor=4.0 \ -Dtest.nativepath=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/images/test/hotspot/jtreg/native \ -XX:MaxRAMPercentage=2 \ -Djava.library.path=/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/images/test/hotspot/jtreg/native \ -Xbatch \ -Xmx128m \ com.sun.javatest.regtest.agent.MainWrapper /home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/test-support/jtreg_test_hotspot_jtreg_compiler_c2_cr6340864/compiler/c2/cr6340864/TestIntVect.d/main.0.jta TEST RESULT: Error. Program `/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/jdk/bin/java' timed out (timeout set to 1600000ms, elapsed time including timeout handling was 1663679ms). -------------------------------------------------- -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From jiangli.zhou at oracle.com Mon Nov 19 18:12:39 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 19 Nov 2018 10:12:39 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> Message-ID: Hi Ioi, The update looks good. Thanks, Jiangli On 11/18/18 7:50 PM, Ioi Lam wrote: > > > On 11/17/18 10:28 PM, Jiangli Zhou wrote: >> Hi Ioi, >> >> On 11/17/18 6:48 PM, Ioi Lam wrote: >> >>> >>> >>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> This looks good. Glad to see the change that utilizes the existing >>>> BasicHashtable. Thanks Coleen for the suggestion! >>>> >>>> To avoid changing the existing assumptions about BasicHashtable and >>>> Hashtable, how about adding a destructor to the new KVHashtable >>>> instead of BasicHashtable? Just did a quick test, which seems to >>>> work properly. >>>> >>> Hi Jiangli, >>> >>> Thanks for the review. >>> >>> Not freeing the entries when a BasicHashtable is destructed is a >>> bug. None of the existing BasicHashtables (and subclasses thereof) >>> are ever destructed, so I don't think my code would impact them. >>> Also, if any code depends on the entries not freed even if the table >>> is destructed, that's clearly a bug in that code, and it should be >>> fixed. >>> >>> If I don't add the destructor to BasicHashtable, the next person who >>> wants to allocate/free BasicHashtables will run in to the same issue. >> >> Dictionary, PackageEntryTable, ModuleEntryTable and >> G1CodeRootSetTable are derived from Hashtable and they call >> free_buckets() in their destructors. So free_buckets() will be called >> twice when those tables are freed although it's harmless. If you want >> to avoid the duplicated call, you could remove the free_buckets() >> calls from those classes' destructors and keep the current >> BasicHashtables changes. I don't have a strong opinion on this, I'll >> leave it to you to decide. >> >> > > Hi Jiangli, > > Good catch. I've remove the call to free_buckets() in the destructor > of those 4 classes. Here's the delta from the last webrev: > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ > > > Also, I found 2 problems while doing this. Hashtables in HotSpot are > really a mess. > > https://bugs.openjdk.java.net/browse/JDK-8214029 > https://bugs.openjdk.java.net/browse/JDK-8214030 > > Thanks > - Ioi > >> Thanks, >> >> Jiangli >>> >>> Thanks >>> - Ioi >>> >>> >>>> Thanks, >>>> >>>> Jiangli >>>> >>>> >>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>> Hi Coleen, >>>>> >>>>> I deleted the default value for MEMFLAGS as you suggested. For my >>>>> instantiated templates, I still use mtInternal, though, since I >>>>> can't find anything better for using at CDS dump time. >>>>> >>>>> Also, Jiangli noted that there's a memory leak, because I allocate >>>>> and free the KVHashtable dynamically. So I added a destructor to >>>>> BasicHashtable to free the buckets and the block-allocated entries. >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>> >>>>> >>>>> This comment around Hashtable::allocate_new_entry() is wrong now >>>>> -- "The allocator in blocks is preferable but doesn't have free >>>>> semantics". Maybe I should change it to this? >>>>> >>>>> "The block allocator in BasicHashtable has less fragmentation, but >>>>> the memory is not freed until the whole table is freed. Use >>>>> allocate_new_entry() if you want to immediately free the memory >>>>> used by each entry". >>>>> >>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also >>>>> tested the solaris/x64 build since it seems to break every time >>>>> you do something with templates :-( >>>>> >>>>> Thanks! >>>>> - Ioi >>>>> >>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> Hi Ioi,? I really like this new wrapper on the old hashtable to >>>>>> not have to write the boilerplate code! >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>> >>>>>> >>>>>> + MEMFLAGS F = mtInternal, >>>>>> >>>>>> >>>>>> I think you should require the mt type though and not make it a >>>>>> default parameter. mtInternal is not very useful to finding >>>>>> memory leaks. >>>>>> >>>>>> Apart from this (which I don't need to see another version), your >>>>>> change looks good and nice to get good performance benefits from >>>>>> this. >>>>>> >>>>>> thanks, >>>>>> Coleen >>>>>> >>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>> Coleen pointed out to me off-line that the good old (and ugly) >>>>>>> BasicHashtable already supports resizing. I think that might be >>>>>>> a better starting point for this RFE: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>> >>>>>>> >>>>>>> I wrote a new template class called "KVHashtable" (copying the >>>>>>> style from ResourceHashtable). That way, you can instantiate >>>>>>> different (Key -> Value) mappings without writing tons of >>>>>>> boilerplate code. The performance is similar to my previous >>>>>>> version, and the code is much cleaner. >>>>>>> >>>>>>> I also renamed the RFE title, as well as the subject line of >>>>>>> this RFR e-mail. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>> Hi Ioi, >>>>>>>>>> >>>>>>>>>> The change looks reasonable to me in general. It would be >>>>>>>>>> helpful to see the performance difference with the expendable >>>>>>>>>> table. Do you have any data when large number of classes are >>>>>>>>>> loaded (>20000)? How much saving does it provide? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>> >>>>>>>>> BEFORE: 93.971 sec >>>>>>>>> AFTER:? 34.761 sec >>>>>>>> >>>>>>>> Thanks for the data! That's about 2.6x improvement with large >>>>>>>> set of classes. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jiangli >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the >>>>>>>>>>> table size to be >>>>>>>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I don't know much about C++ templates, so my attempt on >>>>>>>>>>> doing this may be >>>>>>>>>>> ill-advised. >>>>>>>>>>> >>>>>>>>>>> I *think* that with my patch, the performance of existing >>>>>>>>>>> code, which uses >>>>>>>>>>> a statically-defined SIZE,? should not be affected, as the >>>>>>>>>>> C++ compiler >>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>> >>>>>>>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>> ????? return SIZE; >>>>>>>>>>> ??? } else { >>>>>>>>>>> ????? return _configured_table_size; >>>>>>>>>>> ??? } >>>>>>>>>>> ? } >>>>>>>>>>> >>>>>>>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>> ????? return (Node**)(&_static_table[0]); >>>>>>>>>>> ??? } else { >>>>>>>>>>> ????? return _configured_table; >>>>>>>>>>> ??? } >>>>>>>>>>> ? } >>>>>>>>>>> >>>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>> ??? unsigned index = hash % size(); <----- >>>>>>>>>>> ??? Node** table = get_table(); >>>>>>>>>>> ??? Node** ptr = &table[index]; <----- >>>>>>>>>>> >>>>>>>>>>> back to the old code: >>>>>>>>>>> >>>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>> ??? unsigned index = hash % SIZE; <----- >>>>>>>>>>> ??? Node** ptr = &_table[index]; <----- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> - Ioi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From aph at redhat.com Mon Nov 19 18:14:26 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 19 Nov 2018 18:14:26 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: <7da0518c-f531-a91a-eb52-958bb7c40967@redhat.com> References: <7da0518c-f531-a91a-eb52-958bb7c40967@redhat.com> Message-ID: On 11/19/18 5:50 PM, Andrew Haley wrote: > On 11/19/18 7:53 AM, Yang Zhang (Arm Technology China) wrote: >> Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ > > I've got a test failure: > > > TEST RESULT: Error. Program `/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/jdk/bin/java' timed out (timeout set to 1600000ms, elapsed time including timeout handling was 1663679ms). Ah, this is because of -Xbatch, running the compilers in slowdebug is just too slow because everything runs single-threaded. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From dean.long at oracle.com Mon Nov 19 18:26:26 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 19 Nov 2018 10:26:26 -0800 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <5BF2C404.9030104@oracle.com> References: <5BCF3766.3050504@oracle.com> <5BD032C7.1000807@oracle.com> <55e11251-f8da-aeed-8372-6a26efbb7822@oracle.com> <5BF2C404.9030104@oracle.com> Message-ID: <7657825c-b8c7-abdd-a71f-46828c776257@oracle.com> Yes, Reviewed! dl On 11/19/18 6:09 AM, Erik ?sterlund wrote: > Hi Dean, > > May I interpret your commentary as Reviewed? > > Thanks, > /Erik > > On 2018-10-24 22:20, dean.long at oracle.com wrote: >> On 10/24/18 1:52 AM, Erik ?sterlund wrote: >>> Hi Dean, >>> >>> On 2018-10-23 22:27, dean.long at oracle.com wrote: >>>> Can allocate_objArray_klass() end up calling >>>> SystemDictionary::add_to_hierarchy()?? If so, then you'll end up >>>> locking Compile_lock anway, but in the wrong order with respect to >>>> MultiArray_lock. >>> >>> No, I can't see that allocate_objArray_klass() ever calls >>> SystemDictionary::add_to_hierarchy(). If it did, we would assert >>> that the Compile_lock is held; it never re-acquires the lock. >>> >> >> OK, I forgot that mutex's can't be locked recursively. >> >> I found a clue to the reason for original 1998 change.? This was also >> added at the same time, to klassVtable::initialize_vtable(): >> >> ? // We need the compile lock in order to force the super vtables and >> ? // methodOop code pointers to stay constant. >> ? // (Race scenario: compiler thread updates a vtable entry while >> we're copying >> ? // down entries from the superclass vtable.) >> ? assert_lock(Compile_lock); >> >> By 1999, the vtable code changed a lot and the assert above went >> away, so it was probably just an oversight that the extra uses of >> Compile_lock weren't removed at the same time. >> >> dl >> >>> Thanks, >>> /Erik >>> >>>> dl >>>> >>>> On 10/23/18 7:59 AM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> We occasionally blockingly wait for GC in allocations in >>>>> ObjArrayKlass::allocate_objArray_klass(), while holding the >>>>> Compile_lock. >>>>> This is problematic for concurrent class unloading that needs to >>>>> hold that lock in the unloading path. This introduces a potential >>>>> deadlock situation. >>>>> >>>>> After staring enough at this code together with help from Coleen, >>>>> I have convinced myself that the Compile_lock is in fact not >>>>> needed in this path, and there is nothing to my knowledge it >>>>> actually protects here. The vague comment next to the lock >>>>> suggests its purpose is to protect vtable stubs. But it doesn't. >>>>> There is a VtableStubs_lock for that, and I find no traces if the >>>>> Compile_lock having anything to do with that. >>>>> >>>>> Coleen helped me trying to trace down where this locking code came >>>>> from. It takes us back to before anyone was around, and does not >>>>> seem to have changed in the past 2 decades, and its origins are a >>>>> bit unclear. The theory is that perhaps vtable stubs used to be >>>>> protected by the Compile_lock in ancient times, and the locking >>>>> code is still around because nobody dared to touch it. >>>>> >>>>> Since both code analysis and mach5 suggest there is no reason for >>>>> this lock to be held here, I propose to remove it with this patch. >>>>> >>>>> If anyone that has been around for more than 2 decades in HotSpot >>>>> happens to know the reason why this locking was introduced, any >>>>> commentary around that would be very appreciated. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8212682 >>>>> >>>>> Thanks, >>>>> /Erik >>>> >>> >> > From erik.osterlund at oracle.com Mon Nov 19 18:29:56 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 19 Nov 2018 19:29:56 +0100 Subject: RFR: 8212682: Avoid holding Compile_lock when blocking for GC in ObjArrayKlass::allocate_objArray_klass() In-Reply-To: <7657825c-b8c7-abdd-a71f-46828c776257@oracle.com> References: <5BCF3766.3050504@oracle.com> <5BD032C7.1000807@oracle.com> <55e11251-f8da-aeed-8372-6a26efbb7822@oracle.com> <5BF2C404.9030104@oracle.com> <7657825c-b8c7-abdd-a71f-46828c776257@oracle.com> Message-ID: <713796D9-DD6F-4F68-AF65-E1B62CE18AAF@oracle.com> Hi Dean, Thanks for the review! /Erik > On 19 Nov 2018, at 19:26, dean.long at oracle.com wrote: > > Yes, Reviewed! > > dl > >> On 11/19/18 6:09 AM, Erik ?sterlund wrote: >> Hi Dean, >> >> May I interpret your commentary as Reviewed? >> >> Thanks, >> /Erik >> >>> On 2018-10-24 22:20, dean.long at oracle.com wrote: >>>> On 10/24/18 1:52 AM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>>> On 2018-10-23 22:27, dean.long at oracle.com wrote: >>>>> Can allocate_objArray_klass() end up calling SystemDictionary::add_to_hierarchy()? If so, then you'll end up locking Compile_lock anway, but in the wrong order with respect to MultiArray_lock. >>>> >>>> No, I can't see that allocate_objArray_klass() ever calls SystemDictionary::add_to_hierarchy(). If it did, we would assert that the Compile_lock is held; it never re-acquires the lock. >>>> >>> >>> OK, I forgot that mutex's can't be locked recursively. >>> >>> I found a clue to the reason for original 1998 change. This was also added at the same time, to klassVtable::initialize_vtable(): >>> >>> // We need the compile lock in order to force the super vtables and >>> // methodOop code pointers to stay constant. >>> // (Race scenario: compiler thread updates a vtable entry while we're copying >>> // down entries from the superclass vtable.) >>> assert_lock(Compile_lock); >>> >>> By 1999, the vtable code changed a lot and the assert above went away, so it was probably just an oversight that the extra uses of Compile_lock weren't removed at the same time. >>> >>> dl >>> >>>> Thanks, >>>> /Erik >>>> >>>>> dl >>>>> >>>>>> On 10/23/18 7:59 AM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> We occasionally blockingly wait for GC in allocations in ObjArrayKlass::allocate_objArray_klass(), while holding the Compile_lock. >>>>>> This is problematic for concurrent class unloading that needs to hold that lock in the unloading path. This introduces a potential deadlock situation. >>>>>> >>>>>> After staring enough at this code together with help from Coleen, I have convinced myself that the Compile_lock is in fact not needed in this path, and there is nothing to my knowledge it actually protects here. The vague comment next to the lock suggests its purpose is to protect vtable stubs. But it doesn't. There is a VtableStubs_lock for that, and I find no traces if the Compile_lock having anything to do with that. >>>>>> >>>>>> Coleen helped me trying to trace down where this locking code came from. It takes us back to before anyone was around, and does not seem to have changed in the past 2 decades, and its origins are a bit unclear. The theory is that perhaps vtable stubs used to be protected by the Compile_lock in ancient times, and the locking code is still around because nobody dared to touch it. >>>>>> >>>>>> Since both code analysis and mach5 suggest there is no reason for this lock to be held here, I propose to remove it with this patch. >>>>>> >>>>>> If anyone that has been around for more than 2 decades in HotSpot happens to know the reason why this locking was introduced, any commentary around that would be very appreciated. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8212682/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8212682 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>> >>>> >>> >> > From kim.barrett at oracle.com Mon Nov 19 20:39:01 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 19 Nov 2018 15:39:01 -0500 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com> References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <2310d10b-64ec-5680-a780-71138252a259@oracle.com> <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com> Message-ID: > On Nov 19, 2018, at 7:56 AM, David Holmes wrote: > > On 19/11/2018 5:04 pm, Kim Barrett wrote: >>> On Nov 19, 2018, at 1:31 AM, David Holmes wrote: >>> >>> Hi Kim, >>> >>> On 16/11/2018 12:31 pm, Kim Barrett wrote: >>> >>> This doesn't strike me as a JEP that actually integrates anything. It wants to establish a process by which future C++ features are accepted for use with hotspot. That's just documentation on the OpenJDK wiki as far as I can see. The actual set of features and any "vote" on them can happen any time after the JEP is approved. Any actual use of those features can follow any time after that. >> There is the integration of the build changes needed to enable the use of C++14. >> Presently we explicitly specify C++98 (actually, gnu++98) for gcc. >> We also presently permit the use of older versions of compilers that don?t support C++14 >> (or even C++11) at all, or only partially. Those need to be dropped. > > Even so those changes don't need to be put in place until we actually need to start using a new feature. So I would still think we can finalize the JEP independently of the subsequent code changes. I don't see any benefit to making the first C++ code change that uses a new feature also incorporate the needed build system changes. Indeed, how does one develop that feature usage unless one has the build system changes already in hand? It seems to me that if the documentation says one can use some new language feature but the build system hasn't been updated to actually support doing so, then the documentation is wrong. And recall that this JEP includes making some new features available immediately. (That initial list might be modified as part of this JEP discussion.) From erik.osterlund at oracle.com Mon Nov 19 20:42:31 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 19 Nov 2018 21:42:31 +0100 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT Message-ID: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> Hi, To aid code unloading, I introduced a new member function on CompiledMethod called is_unloading(). It calls oops_do to find if there are broken oops or not. During unloading, if a CompiledMethod is_unloading(), then we make_unloading(), otherwise we clean caches. For this to work, it is assumed that all is_alive() CompiledMethods are visited during code cache unloading, so that later on, you can ask is_alive() CompiledMethods if they are is_unloading(). ...except it looks like for AOTCompiledMethods when running with tiered compilation, may first be is_alive(), then become !is_alive() for a while, and then get resurrected to is_alive() using make_entrant(). This is normally fine, because IC caches are cleaned between resurrections, and the oops are held strongly live. But it messes with the is_unloading() state since while it is temporarily dead, these methods are not visited by code cache unloading, and hence don't get is_unloading() evaluated, and the epoch counters get messed up. So once the method gets resurrected using make_entrant(), we incorrectly ask to re-evaluate the operation outside of safepoints even for STW-unloading collectors. Since for AOTCompiledMethods, we really just want to return false (these ones are not unloaded, and the oops are strong roots), I simply moved the epoched is_unloading() mechanism to nmethod, and let AOTCompiledMethod simply return false, allowing its wonky life cycle to not cause trouble. Webrev: http://cr.openjdk.java.net/~eosterlund/8213486/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213486 Thanks, /Erik From david.holmes at oracle.com Mon Nov 19 21:09:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 Nov 2018 07:09:05 +1000 Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features In-Reply-To: References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com> <2310d10b-64ec-5680-a780-71138252a259@oracle.com> <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com> Message-ID: <47a573c5-17fc-d2b5-3894-45e0714ed8e5@oracle.com> On 20/11/2018 6:39 am, Kim Barrett wrote: >> On Nov 19, 2018, at 7:56 AM, David Holmes wrote: >> >> On 19/11/2018 5:04 pm, Kim Barrett wrote: >>>> On Nov 19, 2018, at 1:31 AM, David Holmes wrote: >>>> >>>> Hi Kim, >>>> >>>> On 16/11/2018 12:31 pm, Kim Barrett wrote: >>>> >>>> This doesn't strike me as a JEP that actually integrates anything. It wants to establish a process by which future C++ features are accepted for use with hotspot. That's just documentation on the OpenJDK wiki as far as I can see. The actual set of features and any "vote" on them can happen any time after the JEP is approved. Any actual use of those features can follow any time after that. >>> There is the integration of the build changes needed to enable the use of C++14. >>> Presently we explicitly specify C++98 (actually, gnu++98) for gcc. >>> We also presently permit the use of older versions of compilers that don?t support C++14 >>> (or even C++11) at all, or only partially. Those need to be dropped. >> >> Even so those changes don't need to be put in place until we actually need to start using a new feature. So I would still think we can finalize the JEP independently of the subsequent code changes. > > I don't see any benefit to making the first C++ code change that uses > a new feature also incorporate the needed build system changes. > Indeed, how does one develop that feature usage unless one has the > build system changes already in hand? Just trying to distinguish the JEP from the build changes that will enable what is agreed upon by the JEP versus the first use of such changes. If you want it all tied together that's fine - you're right it likely won't make 12. Cheers, David ----- > > It seems to me that if the documentation says one can use some new > language feature but the build system hasn't been updated to actually > support doing so, then the documentation is wrong. And recall that > this JEP includes making some new features available immediately. > (That initial list might be modified as part of this JEP discussion.) > From coleen.phillimore at oracle.com Mon Nov 19 22:42:28 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 19 Nov 2018 17:42:28 -0500 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <5BF2AD22.70100@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> Message-ID: On 11/19/18 7:31 AM, Erik ?sterlund wrote: > Hi Coleen, > > Thanks for having a look at this. > > New full webrev: > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ > > On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >> >> How about instead of two bool arguments that are hard to read and >> error prone and need comments, like: >> >> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >> only_not_unloading */); >> >> >> enum FilterAlive { all_blobs, only_alive }; >> enum FilterUnloading { all_blobs, only_not_unloading }; >> >> Then it can be something like: >> >> ?? CompiledMethodIterator iter(only_alive, all_blobs); >> >> Don't know if you can repeate all_blobs like that. > > You can't really have all_blobs in both. I tried out a variant with a > single enum though, hope you like it. > The observation was that we only need 3 different variations: > all_blobs, only_alive or only_alive_and_not_unloading. So I made an > enum out of that. > In fact, these three modes are the only ones I support right now, so > they are also the only valid options, which made the single enum look > even more reasonable. I do like it! > >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >> >> >> I'm not sure if metadata_do() should filter out unloading.? It's >> looking for "old" methods to stop from deallocating memory for them >> (but it's in a safepoint).? If you're unloading, the IC caches are >> already cleaned out? > > IC caches to unloading nmethods are not necessarily cleaned out. The > concurrent GC threads will clean the IC caches concurrently, but a > JavaThread can call the IC cache before the GC has gotten around to > cleaning the IC cache. But any such call into an is_unloading() > nmethod will be trapped by nmethod entry barriers, which will call the > IC miss handler, which sorts things out lazily. So you can view this > as the IC caches to is_unloading() nmethods being logically cleared, > but not necessarily physically cleared yet. The effect is the same - > the is_unloading() nmethods may not be entered from anywhere, > including IC caches. > > So if you were to not filter is_unloading(), you would get a whole > bunch of nmethods that are not really reachable by the application, > i.e. they are not on any stacks, and may not be entered by the > application. Therefore, I think it's perfectly fine to filter them out > here, unless I missed something. And this makes the behaviour closer > to as-if the nmethods were indeed unloaded in the safepoint. Ok.? The thing I'm worried about is if we deallocate memory for a Method* in the is_unloading methods, and some other cleaning or sweeping thread tries to access that memory.? This is used by MetadataOnStackMark during safepoints to see what's safe to deallocate from redefinition. > >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >> >> >> 1108 // The release is only needed for compile-time ordering, as >> accesses >> 1109 // into the nmethod after the store is not safe, due to the sweeper >> 1110 // being allowed to free it then, in the case of concurrent nmethod >> 1111 // unloading. Therefore, there is no need for acquire on the >> loader side. >> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >> >> I tried to make sense out of this first sentence, but couldn't >> really.? After the store to unloaded, can the sweeper free the >> nmethod?? maybe remove "then, " and it would make more sense? > > Right, after the store is observed, the sweeper thread may > concurrently observe the nmethod to be unloaded, and then it may > immediately flip it over to zombie. And I don't want any concurrent > make_unloaded code in that nmethod to still be racing after the > sweeper turns it into zombie; a very unnecessary race to deal with. > > I updated the comment - hopefully it reads better now. > How about removing a comma changing it to accesses are not safe, or access is not safe. + // The release is only needed for compile-time ordering, as accesses + // into the nmethod after the store are not safe, due to the sweeper + // being allowed to free it when the store is observed during + // concurrent nmethod unloading. Therefore, there is no need for + // acquire on the loader side. http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/src/hotspot/share/code/codeCache.hpp.udiff.html Why is this copy constructor defined?? Isn't this what the default copy constructor would do? It looks almost perfect to me :) Coleen > Thanks, > /Erik > >> Thanks, >> Coleen >> >> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>> ..put in bug number in subject to make mail filters happy. >>> >>> /Erik >>> >>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> All current GCs perform code cache unloading in safepoints. >>>> Therefore, nmethods that are alive but is_unloading() are never >>>> observed outside of safepoints. With concurrent class unloading, >>>> nmethods that are alive but is_unloading() will become observable >>>> outside of safepoints. This must be handled appropriately. >>>> >>>> In this patch I changed the nmethod/compiled method iterators to >>>> accept parameters describing whether they should filter out not >>>> is_alive() or is_unloading() methods. Since there is no obvious >>>> default (all combinations are used depending on call site), you >>>> have to explicitly set what iteration mode you want. >>>> >>>> Other than that, I make sure that the sweeper stays away from >>>> is_unloading() nmethods that are not yet is_unloaded(). To make the >>>> interactions between the sweeper and concurrent GC threads safe, I >>>> had to move down the store that sets the state to unloaded, and use >>>> a release_store there, to make sure no accesses float below it at >>>> compile-time. Once that store is observed, nmethods may be deleted. >>>> >>>> In the IC miss handler, I also need to lazily clean stale IC caches >>>> due to calling is_unloading nmethods using nmethod entry barriers. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>> >>>> Thanks, >>>> /Erik >> > From coleen.phillimore at oracle.com Mon Nov 19 22:54:28 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 19 Nov 2018 17:54:28 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> Message-ID: <72021e4a-b5b6-f5c7-5525-89b55d67a095@oracle.com> Looks good to me. Coleen On 11/19/18 1:14 AM, Kim Barrett wrote: > Please review this fix of the debug-only copy constructor and > assignment operator for ResourceObj, along with adding some missing > uses of the copy constructor. > > The missing copy constructor uses were found by enabling -Wextra. > > The ResourceObj debug-only copy constructor should behave exactly the > same as the corresponding default constructor. That is, the setup for > checking the allocation state in the destructor and operator delete > should be the same whether in the normal or copy constructor. That > previously wasn't true, resulting in assert failures. > > The ResourceObj assignment operator also should not be assuming the > target of the assignment was stack/embeded allocated; that's just not > a valid assumption. The corresponding assertion has been removed. > > Note that the ResourceObj allocation/construction/deletion code is > oddly indented, using an initial 4 space indentation and 2 spaces for > further indenting, unlike the usual consistent 2 space indentation > used elsewhere in HotSpot, including elsewhere in allocation.cpp. > I've left that indentation intact to minimize diffs in some views for > review. Unless there are objections, I plan to fix the indentation. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8213481 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ > > Testing: > mach5 tier1-3. There were many failures in these tiers with just the > addition of the missing copy constructor calls (JDK-8213414). > From dean.long at oracle.com Mon Nov 19 23:00:13 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 19 Nov 2018 15:00:13 -0800 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> Message-ID: Hi Erik, On 11/19/18 12:42 PM, Erik ?sterlund wrote: > ...except it looks like for AOTCompiledMethods when running with > tiered compilation, may first be is_alive(), then become !is_alive() > for a while, and then get resurrected to is_alive() using make_entrant(). this doesn't sounds quite right.? AOTCompiledMethods aren't allowed to transition to zombie (!alive), only not_used or not_entrant, which should still have is_alive() returning true.? Maybe some code is using is_not_entrant() instead of !is_alive()? dl From kim.barrett at oracle.com Mon Nov 19 23:29:58 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 19 Nov 2018 18:29:58 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <72021e4a-b5b6-f5c7-5525-89b55d67a095@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <72021e4a-b5b6-f5c7-5525-89b55d67a095@oracle.com> Message-ID: <15CC77BB-7AB1-451C-8119-BB03F25E7701@oracle.com> > On Nov 19, 2018, at 5:54 PM, coleen.phillimore at oracle.com wrote: > > Looks good to me. > Coleen Thanks! > > On 11/19/18 1:14 AM, Kim Barrett wrote: >> Please review this fix of the debug-only copy constructor and >> assignment operator for ResourceObj, along with adding some missing >> uses of the copy constructor. >> >> The missing copy constructor uses were found by enabling -Wextra. >> >> The ResourceObj debug-only copy constructor should behave exactly the >> same as the corresponding default constructor. That is, the setup for >> checking the allocation state in the destructor and operator delete >> should be the same whether in the normal or copy constructor. That >> previously wasn't true, resulting in assert failures. >> >> The ResourceObj assignment operator also should not be assuming the >> target of the assignment was stack/embeded allocated; that's just not >> a valid assumption. The corresponding assertion has been removed. >> >> Note that the ResourceObj allocation/construction/deletion code is >> oddly indented, using an initial 4 space indentation and 2 spaces for >> further indenting, unlike the usual consistent 2 space indentation >> used elsewhere in HotSpot, including elsewhere in allocation.cpp. >> I've left that indentation intact to minimize diffs in some views for >> review. Unless there are objections, I plan to fix the indentation. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8213481 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ >> >> Testing: >> mach5 tier1-3. There were many failures in these tiers with just the >> addition of the missing copy constructor calls (JDK-8213414). From Yang.Zhang at arm.com Tue Nov 20 02:56:14 2018 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Tue, 20 Nov 2018 02:56:14 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: <7da0518c-f531-a91a-eb52-958bb7c40967@redhat.com> Message-ID: >> I've got a test failure: >> >> >> TEST RESULT: Error. Program `/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/jdk/bin/java' timed out (timeout set to 1600000ms, elapsed time including timeout handling was 1663679ms). > Ah, this is because of -Xbatch, running the compilers in slowdebug is just too slow because everything runs single-threaded. Do I need to remove this option? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From Nick.Gasson at arm.com Tue Nov 20 03:05:43 2018 From: Nick.Gasson at arm.com (Nick Gasson) Date: Tue, 20 Nov 2018 03:05:43 +0000 Subject: RFR: 8213845: ARM32: Interpreter doesn't call result handler after native calls Message-ID: Hi, Could someone help me review this patch please? Bug: https://bugs.openjdk.java.net/browse/JDK-8213845 Webrev: http://cr.openjdk.java.net/~njian/8213845/webrev.0/ This fixes a failure of the test runtime/BoolReturn/JNIBooleanTest.java caused by a native method returning a value outside 0..1 as a jboolean which is then interpreted as Java boolean false. Other platforms map all non-zero values to 1 when a native method returns boolean. This is basically the same as bug 8209637 on S390 but the fix for ARM32 is more complicated as this platform didn't previously use the "result handler" mechanism. Changes: * We now call the result handler in the interpreter. Restoring oop results from the stack is now handled here rather than directly in the native wrapper. Boolean normalisation is performed if the result type is T_BOOLEAN otherwise the handler does nothing as before. * Implemented MacroAssembler::c2bool to match other platforms. * Call c2bool at the end of SharedRuntime::generate_native_wrapper to handle boolean results in compiled code. I've checked that this causes no new Jtreg regressions. Thanks! Nick From ioi.lam at oracle.com Tue Nov 20 05:25:24 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 19 Nov 2018 21:25:24 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <94d8c4ea-3cc6-e2d5-8cb9-df621b353014@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> <94d8c4ea-3cc6-e2d5-8cb9-df621b353014@oracle.com> Message-ID: On 11/19/18 7:31 AM, coleen.phillimore at oracle.com wrote: > > Hi, Jiangli - Good find that the blocks leak!? None of the tables that > used blocks were ever deallocated, before this one.? It might make > sense for all the tables to use blocks with this change, although the > blocks might be too big for some of the smaller hashtables. > The blocks are deallocated only when the table is free, but some tables may want to free their entries immediately, or at least more eagerly. I've filed https://bugs.openjdk.java.net/browse/JDK-8214030. I think we should consolidate the code for allocating/freeing the entries. Currently every table does its own way and that's pretty fragile. Thanks - Ioi > Ioi, his change and the deltas look really good. > > Thanks, > Coleen > > On 11/18/18 10:50 PM, Ioi Lam wrote: >> >> >> On 11/17/18 10:28 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> On 11/17/18 6:48 PM, Ioi Lam wrote: >>> >>>> >>>> >>>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> This looks good. Glad to see the change that utilizes the existing >>>>> BasicHashtable. Thanks Coleen for the suggestion! >>>>> >>>>> To avoid changing the existing assumptions about BasicHashtable >>>>> and Hashtable, how about adding a destructor to the new >>>>> KVHashtable instead of BasicHashtable? Just did a quick test, >>>>> which seems to work properly. >>>>> >>>> Hi Jiangli, >>>> >>>> Thanks for the review. >>>> >>>> Not freeing the entries when a BasicHashtable is destructed is a >>>> bug. None of the existing BasicHashtables (and subclasses thereof) >>>> are ever destructed, so I don't think my code would impact them. >>>> Also, if any code depends on the entries not freed even if the >>>> table is destructed, that's clearly a bug in that code, and it >>>> should be fixed. >>>> >>>> If I don't add the destructor to BasicHashtable, the next person >>>> who wants to allocate/free BasicHashtables will run in to the same >>>> issue. >>> >>> Dictionary, PackageEntryTable, ModuleEntryTable and >>> G1CodeRootSetTable are derived from Hashtable and they call >>> free_buckets() in their destructors. So free_buckets() will be >>> called twice when those tables are freed although it's harmless. If >>> you want to avoid the duplicated call, you could remove the >>> free_buckets() calls from those classes' destructors and keep the >>> current BasicHashtables changes. I don't have a strong opinion on >>> this, I'll leave it to you to decide. >>> >>> >> >> Hi Jiangli, >> >> Good catch. I've remove the call to free_buckets() in the destructor >> of those 4 classes. Here's the delta from the last webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ >> >> >> Also, I found 2 problems while doing this. Hashtables in HotSpot are >> really a mess. >> >> https://bugs.openjdk.java.net/browse/JDK-8214029 >> https://bugs.openjdk.java.net/browse/JDK-8214030 >> >> Thanks >> - Ioi >> >>> Thanks, >>> >>> Jiangli >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>>> Thanks, >>>>> >>>>> Jiangli >>>>> >>>>> >>>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> I deleted the default value for MEMFLAGS as you suggested. For my >>>>>> instantiated templates, I still use mtInternal, though, since I >>>>>> can't find anything better for using at CDS dump time. >>>>>> >>>>>> Also, Jiangli noted that there's a memory leak, because I >>>>>> allocate and free the KVHashtable dynamically. So I added a >>>>>> destructor to BasicHashtable to free the buckets and the >>>>>> block-allocated entries. >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>>> >>>>>> >>>>>> This comment around Hashtable::allocate_new_entry() is wrong now >>>>>> -- "The allocator in blocks is preferable but doesn't have free >>>>>> semantics". Maybe I should change it to this? >>>>>> >>>>>> "The block allocator in BasicHashtable has less fragmentation, >>>>>> but the memory is not freed until the whole table is freed. Use >>>>>> allocate_new_entry() if you want to immediately free the memory >>>>>> used by each entry". >>>>>> >>>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also >>>>>> tested the solaris/x64 build since it seems to break every time >>>>>> you do something with templates :-( >>>>>> >>>>>> Thanks! >>>>>> - Ioi >>>>>> >>>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> Hi Ioi,? I really like this new wrapper on the old hashtable to >>>>>>> not have to write the boilerplate code! >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>>> >>>>>>> >>>>>>> + MEMFLAGS F = mtInternal, >>>>>>> >>>>>>> >>>>>>> I think you should require the mt type though and not make it a >>>>>>> default parameter. mtInternal is not very useful to finding >>>>>>> memory leaks. >>>>>>> >>>>>>> Apart from this (which I don't need to see another version), >>>>>>> your change looks good and nice to get good performance benefits >>>>>>> from this. >>>>>>> >>>>>>> thanks, >>>>>>> Coleen >>>>>>> >>>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>>> Coleen pointed out to me off-line that the good old (and ugly) >>>>>>>> BasicHashtable already supports resizing. I think that might be >>>>>>>> a better starting point for this RFE: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>>> >>>>>>>> >>>>>>>> I wrote a new template class called "KVHashtable" (copying the >>>>>>>> style from ResourceHashtable). That way, you can instantiate >>>>>>>> different (Key -> Value) mappings without writing tons of >>>>>>>> boilerplate code. The performance is similar to my previous >>>>>>>> version, and the code is much cleaner. >>>>>>>> >>>>>>>> I also renamed the RFE title, as well as the subject line of >>>>>>>> this RFR e-mail. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>>> Hi Ioi, >>>>>>>>> >>>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>>> Hi Ioi, >>>>>>>>>>> >>>>>>>>>>> The change looks reasonable to me in general. It would be >>>>>>>>>>> helpful to see the performance difference with the >>>>>>>>>>> expendable table. Do you have any data when large number of >>>>>>>>>>> classes are loaded (>20000)? How much saving does it provide? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>>> >>>>>>>>>> BEFORE: 93.971 sec >>>>>>>>>> AFTER:? 34.761 sec >>>>>>>>> >>>>>>>>> Thanks for the data! That's about 2.6x improvement with large >>>>>>>>> set of classes. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the >>>>>>>>>>>> table size to be >>>>>>>>>>>> ? ? ? ? ? dynamically specified when the table is constructed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ? ?? ?? *** C++ template guru alert *** >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I don't know much about C++ templates, so my attempt on >>>>>>>>>>>> doing this may be >>>>>>>>>>>> ill-advised. >>>>>>>>>>>> >>>>>>>>>>>> I *think* that with my patch, the performance of existing >>>>>>>>>>>> code, which uses >>>>>>>>>>>> a statically-defined SIZE,? should not be affected, as the >>>>>>>>>>>> C++ compiler >>>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>>> >>>>>>>>>>>> ? ALWAYSINLINE unsigned size() const { >>>>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>> ????? return SIZE; >>>>>>>>>>>> ??? } else { >>>>>>>>>>>> ????? return _configured_table_size; >>>>>>>>>>>> ??? } >>>>>>>>>>>> ? } >>>>>>>>>>>> >>>>>>>>>>>> ? ALWAYSINLINE Node** get_table() const { >>>>>>>>>>>> ??? if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>> ????? return (Node**)(&_static_table[0]); >>>>>>>>>>>> ??? } else { >>>>>>>>>>>> ????? return _configured_table; >>>>>>>>>>>> ??? } >>>>>>>>>>>> ? } >>>>>>>>>>>> >>>>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>> ??? unsigned index = hash % size(); <----- >>>>>>>>>>>> ??? Node** table = get_table(); >>>>>>>>>>>> ??? Node** ptr = &table[index]; <----- >>>>>>>>>>>> >>>>>>>>>>>> back to the old code: >>>>>>>>>>>> >>>>>>>>>>>> ? Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>> ??? unsigned index = hash % SIZE; <----- >>>>>>>>>>>> ??? Node** ptr = &_table[index]; <----- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> - Ioi >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From ioi.lam at oracle.com Tue Nov 20 06:01:45 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 19 Nov 2018 22:01:45 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> Message-ID: <6551274e-f1a0-7040-f425-b64b4610b9c1@oracle.com> Hi Gerard, Thanks for the review. Please see my comments in-line. On 11/19/18 8:11 AM, Gerard Ziemski wrote: > Hi Ioi, > > Very nice! I only have couple of questions. > > #1 Regarding ?maybe_grow()? in src/hotspot/share/utilities/hashtable.hpp: > > 235 // Grow the number of buckets if the average entries per bucket is over the load_factor > 236 bool maybe_grow(int load_factor = 8) { > 237 if (number_of_entries() / table_size() > load_factor) { > 238 resize(table_size() * 2); > 239 return true; > 240 } else { > 241 return false; > 242 } > 243 } > > a) A suggestion: from my personal experience this didn?t make a difference, but technically a hash map benefits from a size that is a prime number. Maybe you could make a comparison run, with the code as is vs with prime number when resizing and see if you get an even better performance? In my patch, all tables that call maybe_grow started with a table size of a prime number. E.g., in metaspaceShared.cpp: 1074?? typedef KVHashtable< 1075?????? address, address> RelocationTable; 1081???? _new_loc_table = new RelocationTable(8087); So hopefully this will at least be better than starting with a simple power of 2. Anyway, in the LotsOfClasses test, the two tables grew to 254224 and 258784 entries. I tried hard-coding them to start with a large prime number (254249 and? 258787 entries, respectively): ??? BEFORE: 24.004s ??? AFTER : 22.476s The AFTER run also saves all the costs of expanding the tables gradually. So I think doing the simple multiply-by-two is good enough. > b) I am a little worried that we don?t have an upper bound here. In our other hash table we always have upper bounds (or don?t resize). With 35,000 classes, the tables will have over 1.5 million entries and we expand the table to about 250,000. I think we should support dumping at least 100,000 classes. That means the upper bound for the table size should probably be at least 1,000,000. What do you think? > c) Is this table only used from one thread? We are guaranteed here that no other thread will insert/delete anything while we resize? Currently maybe_grow() is called only within a safepoint. resize() already has an assert for safepoint. I will add the same assert into maybe_grow: ? bool maybe_grow(int load_factor = 8) { +? assert(SafepointSynchronize::is_at_safepoint(), "must be at safepoint"); ??? if (number_of_entries() / table_size() > load_factor) { Thanks - Ioi > > Cheers > >> On Nov 18, 2018, at 9:50 PM, Ioi Lam wrote: >> >> >> >> On 11/17/18 10:28 PM, Jiangli Zhou wrote: >>> Hi Ioi, >>> >>> On 11/17/18 6:48 PM, Ioi Lam wrote: >>> >>>> >>>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> This looks good. Glad to see the change that utilizes the existing BasicHashtable. Thanks Coleen for the suggestion! >>>>> >>>>> To avoid changing the existing assumptions about BasicHashtable and Hashtable, how about adding a destructor to the new KVHashtable instead of BasicHashtable? Just did a quick test, which seems to work properly. >>>>> >>>> Hi Jiangli, >>>> >>>> Thanks for the review. >>>> >>>> Not freeing the entries when a BasicHashtable is destructed is a bug. None of the existing BasicHashtables (and subclasses thereof) are ever destructed, so I don't think my code would impact them. Also, if any code depends on the entries not freed even if the table is destructed, that's clearly a bug in that code, and it should be fixed. >>>> >>>> If I don't add the destructor to BasicHashtable, the next person who wants to allocate/free BasicHashtables will run in to the same issue. >>> Dictionary, PackageEntryTable, ModuleEntryTable and G1CodeRootSetTable are derived from Hashtable and they call free_buckets() in their destructors. So free_buckets() will be called twice when those tables are freed although it's harmless. If you want to avoid the duplicated call, you could remove the free_buckets() calls from those classes' destructors and keep the current BasicHashtables changes. I don't have a strong opinion on this, I'll leave it to you to decide. >>> >>> >> Hi Jiangli, >> >> Good catch. I've remove the call to free_buckets() in the destructor of those 4 classes. Here's the delta from the last webrev: >> >> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ >> >> Also, I found 2 problems while doing this. Hashtables in HotSpot are really a mess. >> >> https://bugs.openjdk.java.net/browse/JDK-8214029 >> https://bugs.openjdk.java.net/browse/JDK-8214030 >> >> Thanks >> - Ioi >> >>> Thanks, >>> >>> Jiangli >>>> Thanks >>>> - Ioi >>>> >>>> >>>>> Thanks, >>>>> >>>>> Jiangli >>>>> >>>>> >>>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> I deleted the default value for MEMFLAGS as you suggested. For my instantiated templates, I still use mtInternal, though, since I can't find anything better for using at CDS dump time. >>>>>> >>>>>> Also, Jiangli noted that there's a memory leak, because I allocate and free the KVHashtable dynamically. So I added a destructor to BasicHashtable to free the buckets and the block-allocated entries. >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>>> >>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>>> >>>>>> This comment around Hashtable::allocate_new_entry() is wrong now -- "The allocator in blocks is preferable but doesn't have free semantics". Maybe I should change it to this? >>>>>> >>>>>> "The block allocator in BasicHashtable has less fragmentation, but the memory is not freed until the whole table is freed. Use allocate_new_entry() if you want to immediately free the memory used by each entry". >>>>>> >>>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested the solaris/x64 build since it seems to break every time you do something with templates :-( >>>>>> >>>>>> Thanks! >>>>>> - Ioi >>>>>> >>>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>>> Hi Ioi, I really like this new wrapper on the old hashtable to not have to write the boilerplate code! >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>>> >>>>>>> + MEMFLAGS F = mtInternal, >>>>>>> >>>>>>> >>>>>>> I think you should require the mt type though and not make it a default parameter. mtInternal is not very useful to finding memory leaks. >>>>>>> >>>>>>> Apart from this (which I don't need to see another version), your change looks good and nice to get good performance benefits from this. >>>>>>> >>>>>>> thanks, >>>>>>> Coleen >>>>>>> >>>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>>> Coleen pointed out to me off-line that the good old (and ugly) BasicHashtable already supports resizing. I think that might be a better starting point for this RFE: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>>> >>>>>>>> I wrote a new template class called "KVHashtable" (copying the style from ResourceHashtable). That way, you can instantiate different (Key -> Value) mappings without writing tons of boilerplate code. The performance is similar to my previous version, and the code is much cleaner. >>>>>>>> >>>>>>>> I also renamed the RFE title, as well as the subject line of this RFR e-mail. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> - Ioi >>>>>>>> >>>>>>>> >>>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>>> Hi Ioi, >>>>>>>>> >>>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>>> >>>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>>> Hi Ioi, >>>>>>>>>>> >>>>>>>>>>> The change looks reasonable to me in general. It would be helpful to see the performance difference with the expendable table. Do you have any data when large number of classes are loaded (>20000)? How much saving does it provide? >>>>>>>>>>> >>>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>>> >>>>>>>>>> BEFORE: 93.971 sec >>>>>>>>>> AFTER: 34.761 sec >>>>>>>>> Thanks for the data! That's about 2.6x improvement with large set of classes. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Jiangli >>>>>>>>>> Thanks >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Jiangli >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>>> >>>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table size to be >>>>>>>>>>>> dynamically specified when the table is constructed. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> *** C++ template guru alert *** >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I don't know much about C++ templates, so my attempt on doing this may be >>>>>>>>>>>> ill-advised. >>>>>>>>>>>> >>>>>>>>>>>> I *think* that with my patch, the performance of existing code, which uses >>>>>>>>>>>> a statically-defined SIZE, should not be affected, as the C++ compiler >>>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>>> >>>>>>>>>>>> ALWAYSINLINE unsigned size() const { >>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>> return SIZE; >>>>>>>>>>>> } else { >>>>>>>>>>>> return _configured_table_size; >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> ALWAYSINLINE Node** get_table() const { >>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>> return (Node**)(&_static_table[0]); >>>>>>>>>>>> } else { >>>>>>>>>>>> return _configured_table; >>>>>>>>>>>> } >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>> unsigned index = hash % size(); <----- >>>>>>>>>>>> Node** table = get_table(); >>>>>>>>>>>> Node** ptr = &table[index]; <----- >>>>>>>>>>>> >>>>>>>>>>>> back to the old code: >>>>>>>>>>>> >>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>> unsigned index = hash % SIZE; <----- >>>>>>>>>>>> Node** ptr = &_table[index]; <----- >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> - Ioi From thomas.stuefe at gmail.com Tue Nov 20 07:07:44 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 20 Nov 2018 08:07:44 +0100 Subject: RFR (S) 8181143: Introduce diagnostic flag to abort VM on too long VM operations In-Reply-To: References: <81bae1ab-cd7a-85f3-be8b-42592da965d7@redhat.com> Message-ID: Hi, I like the idea in general. I don't know whether I need 10ms resolution though - if we limit the goal to catching just hanging VMOps - which would still pretty useful - 1s would even be enough. I think we can do with just two flags, since VMOperationTimeout and VMOperationTimeoutDelay are redundant. Just go with the delay, make -1 the default that does nothing. Same could go for SafepointTimeout vs SafepointTimeoutDelay. I also agree with others that it would make more sense were we to kill the VMThread instead (e.g. just with pthread_kill(SIGILL) or similar) - we do something like that in error handling (see ErrorLogTimeout). just my 2 cents. ..Thomas On Mon, Nov 19, 2018 at 7:16 AM David Holmes wrote: > > Hi Aleksey, > > First the synopsis is not accurate: > > "Introduce diagnostic flag to abort VM on too long VM operations" > > You're not just introducing one diagnostic flag, your introducing the > entire VM operation timeout mechanism, including two product flags and > one diagnostic. So the CR needs to reflect that clearly and you will > need a CSR request to add the two product flags. And they will need to > be documented. > > Three flags just for this makes me cringe. (Yes it mirrors the safepoint > timeout flags but if that were proposed today I'd have the same reaction.) > > On 17/11/2018 2:30 am, Aleksey Shipilev wrote: > > RFE: > > https://bugs.openjdk.java.net/browse/JDK-8181143 > > > > Webrev: > > http://cr.openjdk.java.net/~shade/8181143/webrev.03/ > > > > SafepointTimeout is nice to discover long/stuck safepoint syncs. But it is as important to discover > > long/stuck VM operations. This patch complements the timeout machinery with tracking VM operation > > themselves. Among other things, this allows to terminate the VM when very long VM operation is > > blocking progress. High-availability users would enjoy fail-fast JVM -- in fact, the original > > prototype was done as request from Apache Ignite developers. > > > > Example with -XX:+VMOperationTimeout -XX:VMOperationTimeoutDelay=100 -XX:+AbortVMOnVMOperationTimeout: > > > > [3.117s][info][gc,start] GC(2) Pause Young (Normal) (G1 Evacuation Pause) > > [3.224s][warning][vmthread] VM Operation G1CollectForAllocation took longer than 100 ms > > # > > # A fatal error has been detected by the Java Runtime Environment: > > # > > # Internal Error (/home/sh/jdk-jdk/src/hotspot/share/runtime/vmThread.cpp:218), pid=2536, tid=2554 > > # fatal error: VM Operation G1CollectForAllocation took longer than 100 ms > > # > > It's not safe to access vm_safepoint_description() outside the VMThread > as the _cur_vm_operation could be deleted while you're trying to access > its name. > > Initially I thought this might be useful for tracking down excessively > long VM ops, but with a global timeout it can't do that. And a per-op > timeout would be rather tricky to pass through from the command-line > (but easy enough to use once you had it). > > And as we don't have a general timer mechanism this has to use polling > so you pay for a 10ms task wakeup regardless of how long the timeout is. > > Given the limitations of the global timeout I'm not sure I see a use for > the non-aborting form. This could just reduce down to: > > -XX:AbortVMOpsAfter:NN > > otherwise I don't really think this carries its weight. Of course that's > just my opinion. Interested to hear others. > > Cheers, > David > > > Testing: hotspot/tier1, ad-hoc tests, jdk-submit (pending) > > > > Thanks, > > -Aleksey > > From aph at redhat.com Tue Nov 20 09:39:06 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 20 Nov 2018 09:39:06 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: <7da0518c-f531-a91a-eb52-958bb7c40967@redhat.com> Message-ID: On 11/20/18 2:56 AM, Yang Zhang (Arm Technology China) wrote: >>> I've got a test failure: >>> >>> >>> TEST RESULT: Error. Program `/home/aph/jdk-jdk/build/linux-aarch64-server-slowdebug/jdk/bin/java' timed out (timeout set to 1600000ms, elapsed time including timeout handling was 1663679ms). > >> Ah, this is because of -Xbatch, running the compilers in slowdebug is just too slow because everything runs single-threaded. > > Do I need to remove this option? No. The test needs -Xbatch because we have to be certain that the code is compiled before it is inspected. I'm still reviewing your patch. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From robbin.ehn at oracle.com Tue Nov 20 09:57:59 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Tue, 20 Nov 2018 10:57:59 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <5BF2AD22.70100@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> Message-ID: <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> Looks good! I gave a suggestion IRL about an is_dying method, which covers the is_alive and is_unloading query. If choose to take it or not I'll leave it up to you. Thanks, Robbin On 11/19/18 1:31 PM, Erik ?sterlund wrote: > Hi Coleen, > > Thanks for having a look at this. > > New full webrev: > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ > > On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >> >> How about instead of two bool arguments that are hard to read and error prone >> and need comments, like: >> >> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >> only_not_unloading */); >> >> >> enum FilterAlive { all_blobs, only_alive }; >> enum FilterUnloading { all_blobs, only_not_unloading }; >> >> Then it can be something like: >> >> ?? CompiledMethodIterator iter(only_alive, all_blobs); >> >> Don't know if you can repeate all_blobs like that. > > You can't really have all_blobs in both. I tried out a variant with a single > enum though, hope you like it. > The observation was that we only need 3 different variations: all_blobs, > only_alive or only_alive_and_not_unloading. So I made an enum out of that. > In fact, these three modes are the only ones I support right now, so they are > also the only valid options, which made the single enum look even more reasonable. > >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >> >> >> I'm not sure if metadata_do() should filter out unloading.? It's looking for >> "old" methods to stop from deallocating memory for them (but it's in a >> safepoint).? If you're unloading, the IC caches are already cleaned out? > > IC caches to unloading nmethods are not necessarily cleaned out. The concurrent > GC threads will clean the IC caches concurrently, but a JavaThread can call the > IC cache before the GC has gotten around to cleaning the IC cache. But any such > call into an is_unloading() nmethod will be trapped by nmethod entry barriers, > which will call the IC miss handler, which sorts things out lazily. So you can > view this as the IC caches to is_unloading() nmethods being logically cleared, > but not necessarily physically cleared yet. The effect is the same - the > is_unloading() nmethods may not be entered from anywhere, including IC caches. > > So if you were to not filter is_unloading(), you would get a whole bunch of > nmethods that are not really reachable by the application, i.e. they are not on > any stacks, and may not be entered by the application. Therefore, I think it's > perfectly fine to filter them out here, unless I missed something. And this > makes the behaviour closer to as-if the nmethods were indeed unloaded in the > safepoint. > >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >> >> >> 1108 // The release is only needed for compile-time ordering, as accesses >> 1109 // into the nmethod after the store is not safe, due to the sweeper >> 1110 // being allowed to free it then, in the case of concurrent nmethod >> 1111 // unloading. Therefore, there is no need for acquire on the loader side. >> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >> >> I tried to make sense out of this first sentence, but couldn't really.? After >> the store to unloaded, can the sweeper free the nmethod?? maybe remove "then, >> " and it would make more sense? > > Right, after the store is observed, the sweeper thread may concurrently observe > the nmethod to be unloaded, and then it may immediately flip it over to zombie. > And I don't want any concurrent make_unloaded code in that nmethod to still be > racing after the sweeper turns it into zombie; a very unnecessary race to deal > with. > > I updated the comment - hopefully it reads better now. > > Thanks, > /Erik > >> Thanks, >> Coleen >> >> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>> ..put in bug number in subject to make mail filters happy. >>> >>> /Erik >>> >>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> All current GCs perform code cache unloading in safepoints. Therefore, >>>> nmethods that are alive but is_unloading() are never observed outside of >>>> safepoints. With concurrent class unloading, nmethods that are alive but >>>> is_unloading() will become observable outside of safepoints. This must be >>>> handled appropriately. >>>> >>>> In this patch I changed the nmethod/compiled method iterators to accept >>>> parameters describing whether they should filter out not is_alive() or >>>> is_unloading() methods. Since there is no obvious default (all combinations >>>> are used depending on call site), you have to explicitly set what iteration >>>> mode you want. >>>> >>>> Other than that, I make sure that the sweeper stays away from is_unloading() >>>> nmethods that are not yet is_unloaded(). To make the interactions between >>>> the sweeper and concurrent GC threads safe, I had to move down the store >>>> that sets the state to unloaded, and use a release_store there, to make sure >>>> no accesses float below it at compile-time. Once that store is observed, >>>> nmethods may be deleted. >>>> >>>> In the IC miss handler, I also need to lazily clean stale IC caches due to >>>> calling is_unloading nmethods using nmethod entry barriers. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>> >>>> Thanks, >>>> /Erik >> > From aph at redhat.com Tue Nov 20 10:09:22 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 20 Nov 2018 10:09:22 +0000 Subject: RFR: 8213845: ARM32: Interpreter doesn't call result handler after native calls In-Reply-To: References: Message-ID: On 11/20/18 3:05 AM, Nick Gasson wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8213845 > Webrev: http://cr.openjdk.java.net/~njian/8213845/webrev.0/ That looks right, thanks. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.osterlund at oracle.com Tue Nov 20 11:12:31 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 20 Nov 2018 12:12:31 +0100 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> Message-ID: <5BF3EC1F.8060003@oracle.com> Hi Dean, Yeah I think I misunderstood what I observed. So what I know is that in the code cache unloading, I'm not getting all is_alive() AOT methods into my iterator, which messes up the assumptions made by the epoch based scheme for AOT methods. I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT heap, making it no longer observable from the iterators. Then it calls the VM_Deoptimize vm operation after. Throughout all this, the AOTCompiledMethod is alive(), yet when the iterators ask for all is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC caches may still reference these methods and check if it is_unloading, and then we blow up. There may possibly be multiple ways for is_alive() AOTCompiledMethods to not be visible from iterators yet be visible through IC caches using the "invalid" state in the . Anyway, the fix is the same: stop doing the epoch state thingey for is_unloading() on AOTCompiledMethod where it isn't needed, and doesn't seem to play well with the rather different life cycle it has, and just return false instead. Thanks, /Erik On 2018-11-20 00:00, dean.long at oracle.com wrote: > Hi Erik, > > On 11/19/18 12:42 PM, Erik ?sterlund wrote: >> ...except it looks like for AOTCompiledMethods when running with >> tiered compilation, may first be is_alive(), then become !is_alive() >> for a while, and then get resurrected to is_alive() using >> make_entrant(). > > this doesn't sounds quite right. AOTCompiledMethods aren't allowed to > transition to zombie (!alive), only not_used or not_entrant, which > should still have is_alive() returning true. Maybe some code is using > is_not_entrant() instead of !is_alive()? > > dl From erik.osterlund at oracle.com Tue Nov 20 15:10:10 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 20 Nov 2018 16:10:10 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> Message-ID: <5BF423D2.1070800@oracle.com> Ping. /Erik On 2018-11-12 23:02, Erik ?sterlund wrote: > Hi, > > In my change 8209189, I moved a lot of code around relating to code > cache unloading. Unfortunately, I failed to see that when making > nmethods unloaded, and their dependents are flushed... they are not > immediately removed. Instead, they are tagged to have stale entries, > that are later cleaned during weak metadata cleaning of InstanceKlass. > As for method handles, they leak instead, with some hacks to reduce > the leaking by expunging entries while adding and removing. > > Obviously, with InstanceKlass cleanup and code cache unloading now > possibly running in parallel after 8209189, things can go wrong. These > two phases used > to be separated by a "wait barrier" for the worker threads preventing > that parallelism. > > The reason dependency contexts were not cleaned when they were found > during code cache cleaning, is because it was not thread safe when code > cache unloading was made parallel instead of serial. But now that we are > implementing concurrent class unloading, we are going to need concurrent > cleanup of dependency contexts anyway. Therefore I will propose a bug > fix that fixes the problem in a way that works for both serial, > parallel and concurrent class unloading. > > The solution is to during code cache unloading claim cleaning of > encountered stale dependency contexts, and clean them straight away. > Entries are unlinked in a lock-free fashion, and placed on a purge > list that is walked and deleted during ClassLoaderDataGraph::purge(). > This follows the thinking of first unlinking, then syncing all > threads, and then purging. > > New accessors for the head and next links, hide is_unloading entries > and helps unlinking them and putting them into the purge list. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8213565 > > Thanks, > /Erik From gerard.ziemski at oracle.com Tue Nov 20 15:53:10 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 20 Nov 2018 09:53:10 -0600 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <6551274e-f1a0-7040-f425-b64b4610b9c1@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> <6551274e-f1a0-7040-f425-b64b4610b9c1@oracle.com> Message-ID: hi Ioi, Please see my comments in-line. > On Nov 20, 2018, at 12:01 AM, Ioi Lam wrote: > > Hi Gerard, > > Thanks for the review. Please see my comments in-line. > > > On 11/19/18 8:11 AM, Gerard Ziemski wrote: >> Hi Ioi, >> >> Very nice! I only have couple of questions. >> >> #1 Regarding ?maybe_grow()? in src/hotspot/share/utilities/hashtable.hpp: >> >> 235 // Grow the number of buckets if the average entries per bucket is over the load_factor >> 236 bool maybe_grow(int load_factor = 8) { >> 237 if (number_of_entries() / table_size() > load_factor) { >> 238 resize(table_size() * 2); >> 239 return true; >> 240 } else { >> 241 return false; >> 242 } >> 243 } >> >> a) A suggestion: from my personal experience this didn?t make a difference, but technically a hash map benefits from a size that is a prime number. Maybe you could make a comparison run, with the code as is vs with prime number when resizing and see if you get an even better performance? > > In my patch, all tables that call maybe_grow started with a table size of a prime number. E.g., in metaspaceShared.cpp: > > 1074 typedef KVHashtable< > 1075 address, address> RelocationTable; > 1081 _new_loc_table = new RelocationTable(8087); > > So hopefully this will at least be better than starting with a simple power of 2. > > Anyway, in the LotsOfClasses test, the two tables grew to 254224 and 258784 entries. I tried hard-coding them to start with a large prime number (254249 and 258787 entries, respectively): > > BEFORE: 24.004s > AFTER : 22.476s > > The AFTER run also saves all the costs of expanding the tables gradually. So I think doing the simple multiply-by-two is good enough. Thank you for checking. I do like the simplicity of doubling the size, so if the performance benefit of calculating a prime is small (as I was suspecting), then I?m more than fine with the simple solution. I just happened to look at https://wiki.openjdk.java.net/display/HotSpot/StyleGuide yesterday and can?t help, but note here, that according to it, we should ?Assign names to constant literals and use the names instead?, so we should replace the literals like ?8087? with ?static const size_t INITIAL_TABLE_SIZE = 8087? (I know this is a pre-existent code) > >> b) I am a little worried that we don?t have an upper bound here. In our other hash table we always have upper bounds (or don?t resize). > > With 35,000 classes, the tables will have over 1.5 million entries and we expand the table to about 250,000. I think we should support dumping at least 100,000 classes. That means the upper bound for the table size should probably be at least 1,000,000. > > What do you think? If someone hits this bound and it causes an issue, they can tell us and we can always increase it later, but with your fix and the proposed bound, it?s way much better compared to the current behavior. > >> c) Is this table only used from one thread? We are guaranteed here that no other thread will insert/delete anything while we resize? > > Currently maybe_grow() is called only within a safepoint. resize() already has an assert for safepoint. I will add the same assert into maybe_grow: > > > bool maybe_grow(int load_factor = 8) { > + assert(SafepointSynchronize::is_at_safepoint(), "must be at safepoint"); > if (number_of_entries() / table_size() > load_factor) { Thanks. > > Thanks > - Ioi > >> >> Cheers >> >>> On Nov 18, 2018, at 9:50 PM, Ioi Lam wrote: >>> >>> >>> >>> On 11/17/18 10:28 PM, Jiangli Zhou wrote: >>>> Hi Ioi, >>>> >>>> On 11/17/18 6:48 PM, Ioi Lam wrote: >>>> >>>>> >>>>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> This looks good. Glad to see the change that utilizes the existing BasicHashtable. Thanks Coleen for the suggestion! >>>>>> >>>>>> To avoid changing the existing assumptions about BasicHashtable and Hashtable, how about adding a destructor to the new KVHashtable instead of BasicHashtable? Just did a quick test, which seems to work properly. >>>>>> >>>>> Hi Jiangli, >>>>> >>>>> Thanks for the review. >>>>> >>>>> Not freeing the entries when a BasicHashtable is destructed is a bug. None of the existing BasicHashtables (and subclasses thereof) are ever destructed, so I don't think my code would impact them. Also, if any code depends on the entries not freed even if the table is destructed, that's clearly a bug in that code, and it should be fixed. >>>>> >>>>> If I don't add the destructor to BasicHashtable, the next person who wants to allocate/free BasicHashtables will run in to the same issue. >>>> Dictionary, PackageEntryTable, ModuleEntryTable and G1CodeRootSetTable are derived from Hashtable and they call free_buckets() in their destructors. So free_buckets() will be called twice when those tables are freed although it's harmless. If you want to avoid the duplicated call, you could remove the free_buckets() calls from those classes' destructors and keep the current BasicHashtables changes. I don't have a strong opinion on this, I'll leave it to you to decide. >>>> >>>> >>> Hi Jiangli, >>> >>> Good catch. I've remove the call to free_buckets() in the destructor of those 4 classes. Here's the delta from the last webrev: >>> >>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ >>> >>> Also, I found 2 problems while doing this. Hashtables in HotSpot are really a mess. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8214029 >>> https://bugs.openjdk.java.net/browse/JDK-8214030 >>> >>> Thanks >>> - Ioi >>> >>>> Thanks, >>>> >>>> Jiangli >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>>> Thanks, >>>>>> >>>>>> Jiangli >>>>>> >>>>>> >>>>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> I deleted the default value for MEMFLAGS as you suggested. For my instantiated templates, I still use mtInternal, though, since I can't find anything better for using at CDS dump time. >>>>>>> >>>>>>> Also, Jiangli noted that there's a memory leak, because I allocate and free the KVHashtable dynamically. So I added a destructor to BasicHashtable to free the buckets and the block-allocated entries. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>>>> >>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>>>> >>>>>>> This comment around Hashtable::allocate_new_entry() is wrong now -- "The allocator in blocks is preferable but doesn't have free semantics". Maybe I should change it to this? >>>>>>> >>>>>>> "The block allocator in BasicHashtable has less fragmentation, but the memory is not freed until the whole table is freed. Use allocate_new_entry() if you want to immediately free the memory used by each entry". >>>>>>> >>>>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested the solaris/x64 build since it seems to break every time you do something with templates :-( >>>>>>> >>>>>>> Thanks! >>>>>>> - Ioi >>>>>>> >>>>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>>>> Hi Ioi, I really like this new wrapper on the old hashtable to not have to write the boilerplate code! >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>>>> >>>>>>>> + MEMFLAGS F = mtInternal, >>>>>>>> >>>>>>>> >>>>>>>> I think you should require the mt type though and not make it a default parameter. mtInternal is not very useful to finding memory leaks. >>>>>>>> >>>>>>>> Apart from this (which I don't need to see another version), your change looks good and nice to get good performance benefits from this. >>>>>>>> >>>>>>>> thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>>>> Coleen pointed out to me off-line that the good old (and ugly) BasicHashtable already supports resizing. I think that might be a better starting point for this RFE: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>>>> >>>>>>>>> I wrote a new template class called "KVHashtable" (copying the style from ResourceHashtable). That way, you can instantiate different (Key -> Value) mappings without writing tons of boilerplate code. The performance is similar to my previous version, and the code is much cleaner. >>>>>>>>> >>>>>>>>> I also renamed the RFE title, as well as the subject line of this RFR e-mail. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>>>> Hi Ioi, >>>>>>>>>> >>>>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>>>> >>>>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>>>> Hi Ioi, >>>>>>>>>>>> >>>>>>>>>>>> The change looks reasonable to me in general. It would be helpful to see the performance difference with the expendable table. Do you have any data when large number of classes are loaded (>20000)? How much saving does it provide? >>>>>>>>>>>> >>>>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>>>> >>>>>>>>>>> BEFORE: 93.971 sec >>>>>>>>>>> AFTER: 34.761 sec >>>>>>>>>> Thanks for the data! That's about 2.6x improvement with large set of classes. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jiangli >>>>>>>>>>> Thanks >>>>>>>>>>> - Ioi >>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Jiangli >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>>>> >>>>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table size to be >>>>>>>>>>>>> dynamically specified when the table is constructed. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> *** C++ template guru alert *** >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I don't know much about C++ templates, so my attempt on doing this may be >>>>>>>>>>>>> ill-advised. >>>>>>>>>>>>> >>>>>>>>>>>>> I *think* that with my patch, the performance of existing code, which uses >>>>>>>>>>>>> a statically-defined SIZE, should not be affected, as the C++ compiler >>>>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>>>> >>>>>>>>>>>>> ALWAYSINLINE unsigned size() const { >>>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>>> return SIZE; >>>>>>>>>>>>> } else { >>>>>>>>>>>>> return _configured_table_size; >>>>>>>>>>>>> } >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> ALWAYSINLINE Node** get_table() const { >>>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>>> return (Node**)(&_static_table[0]); >>>>>>>>>>>>> } else { >>>>>>>>>>>>> return _configured_table; >>>>>>>>>>>>> } >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>>> unsigned index = hash % size(); <----- >>>>>>>>>>>>> Node** table = get_table(); >>>>>>>>>>>>> Node** ptr = &table[index]; <----- >>>>>>>>>>>>> >>>>>>>>>>>>> back to the old code: >>>>>>>>>>>>> >>>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>>> unsigned index = hash % SIZE; <----- >>>>>>>>>>>>> Node** ptr = &_table[index]; <----- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks! >>>>>>>>>>>>> - Ioi > From erik.osterlund at oracle.com Tue Nov 20 16:04:06 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 20 Nov 2018 17:04:06 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> Message-ID: <5BF43076.4010305@oracle.com> Hi Coleen, Thanks for the review. On 2018-11-19 23:42, coleen.phillimore at oracle.com wrote: > > > On 11/19/18 7:31 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thanks for having a look at this. >> >> New full webrev: >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >> >> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>> >>> How about instead of two bool arguments that are hard to read and >>> error prone and need comments, like: >>> >>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >>> only_not_unloading */); >>> >>> >>> enum FilterAlive { all_blobs, only_alive }; >>> enum FilterUnloading { all_blobs, only_not_unloading }; >>> >>> Then it can be something like: >>> >>> CompiledMethodIterator iter(only_alive, all_blobs); >>> >>> Don't know if you can repeate all_blobs like that. >> >> You can't really have all_blobs in both. I tried out a variant with a >> single enum though, hope you like it. >> The observation was that we only need 3 different variations: >> all_blobs, only_alive or only_alive_and_not_unloading. So I made an >> enum out of that. >> In fact, these three modes are the only ones I support right now, so >> they are also the only valid options, which made the single enum look >> even more reasonable. > > I do like it! >> >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>> >>> >>> I'm not sure if metadata_do() should filter out unloading. It's >>> looking for "old" methods to stop from deallocating memory for them >>> (but it's in a safepoint). If you're unloading, the IC caches are >>> already cleaned out? >> >> IC caches to unloading nmethods are not necessarily cleaned out. The >> concurrent GC threads will clean the IC caches concurrently, but a >> JavaThread can call the IC cache before the GC has gotten around to >> cleaning the IC cache. But any such call into an is_unloading() >> nmethod will be trapped by nmethod entry barriers, which will call >> the IC miss handler, which sorts things out lazily. So you can view >> this as the IC caches to is_unloading() nmethods being logically >> cleared, but not necessarily physically cleared yet. The effect is >> the same - the is_unloading() nmethods may not be entered from >> anywhere, including IC caches. >> >> So if you were to not filter is_unloading(), you would get a whole >> bunch of nmethods that are not really reachable by the application, >> i.e. they are not on any stacks, and may not be entered by the >> application. Therefore, I think it's perfectly fine to filter them >> out here, unless I missed something. And this makes the behaviour >> closer to as-if the nmethods were indeed unloaded in the safepoint. > > Ok. The thing I'm worried about is if we deallocate memory for a > Method* in the is_unloading methods, and some other cleaning or > sweeping thread tries to access that memory. This is used by > MetadataOnStackMark during safepoints to see what's safe to deallocate > from redefinition. I understand. I do think this is safe though, and it only walked is_alive() compiled methods looking for metadata before, e.g. not unloaded or zombie methods. Similarly, nobody should be poking around at the metadata of unloading methods. Should being the key word there I guess... > >> >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>> >>> >>> 1108 // The release is only needed for compile-time ordering, as >>> accesses >>> 1109 // into the nmethod after the store is not safe, due to the >>> sweeper >>> 1110 // being allowed to free it then, in the case of concurrent >>> nmethod >>> 1111 // unloading. Therefore, there is no need for acquire on the >>> loader side. >>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>> >>> I tried to make sense out of this first sentence, but couldn't >>> really. After the store to unloaded, can the sweeper free the >>> nmethod? maybe remove "then, " and it would make more sense? >> >> Right, after the store is observed, the sweeper thread may >> concurrently observe the nmethod to be unloaded, and then it may >> immediately flip it over to zombie. And I don't want any concurrent >> make_unloaded code in that nmethod to still be racing after the >> sweeper turns it into zombie; a very unnecessary race to deal with. >> >> I updated the comment - hopefully it reads better now. >> > > How about removing a comma changing it to accesses are not safe, or > access is not safe. > + // The release is only needed for compile-time ordering, as accesses > + // into the nmethod after the store are not safe, due to the sweeper > + // being allowed to free it when the store is observed during > + // concurrent nmethod unloading. Therefore, there is no need for > + // acquire on the loader side. Comma will be removed before pushing. > http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/src/hotspot/share/code/codeCache.hpp.udiff.html > > Why is this copy constructor defined? Isn't this what the default > copy constructor would do? Yeah it is. I like explicitly typing out what constructors should do. But I can remove it if you prefer that. > It looks almost perfect to me :) Awesome. Thanks for reviewing this. Thanks, /Erik > Coleen > >> Thanks, >> /Erik >> >>> Thanks, >>> Coleen >>> >>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>> ..put in bug number in subject to make mail filters happy. >>>> >>>> /Erik >>>> >>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> All current GCs perform code cache unloading in safepoints. >>>>> Therefore, nmethods that are alive but is_unloading() are never >>>>> observed outside of safepoints. With concurrent class unloading, >>>>> nmethods that are alive but is_unloading() will become observable >>>>> outside of safepoints. This must be handled appropriately. >>>>> >>>>> In this patch I changed the nmethod/compiled method iterators to >>>>> accept parameters describing whether they should filter out not >>>>> is_alive() or is_unloading() methods. Since there is no obvious >>>>> default (all combinations are used depending on call site), you >>>>> have to explicitly set what iteration mode you want. >>>>> >>>>> Other than that, I make sure that the sweeper stays away from >>>>> is_unloading() nmethods that are not yet is_unloaded(). To make >>>>> the interactions between the sweeper and concurrent GC threads >>>>> safe, I had to move down the store that sets the state to >>>>> unloaded, and use a release_store there, to make sure no accesses >>>>> float below it at compile-time. Once that store is observed, >>>>> nmethods may be deleted. >>>>> >>>>> In the IC miss handler, I also need to lazily clean stale IC >>>>> caches due to calling is_unloading nmethods using nmethod entry >>>>> barriers. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>> >>>>> Thanks, >>>>> /Erik >>> >> > From erik.osterlund at oracle.com Tue Nov 20 16:10:22 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 20 Nov 2018 17:10:22 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> Message-ID: <5BF431EE.5000609@oracle.com> Hi Robbin, Thanks for the review. I think I will go without is_dying() now because I couldn't figure out how to put it in, in an intuitive way. Mostly because most of the code is checking for is_alive() && !is_unloading() compiled methods. In that state they are not is_dying(), according to your definition, but checking for !is_dying() doesn't imply that it is alive. So I think I will stick with being more explicit for now. Thanks, /Erik On 2018-11-20 10:57, Robbin Ehn wrote: > Looks good! > > I gave a suggestion IRL about an is_dying method, which covers the > is_alive > and is_unloading query. If choose to take it or not I'll leave it up > to you. > > Thanks, Robbin > > On 11/19/18 1:31 PM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thanks for having a look at this. >> >> New full webrev: >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >> >> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>> >>> How about instead of two bool arguments that are hard to read and >>> error prone and need comments, like: >>> >>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >>> only_not_unloading */); >>> >>> >>> enum FilterAlive { all_blobs, only_alive }; >>> enum FilterUnloading { all_blobs, only_not_unloading }; >>> >>> Then it can be something like: >>> >>> CompiledMethodIterator iter(only_alive, all_blobs); >>> >>> Don't know if you can repeate all_blobs like that. >> >> You can't really have all_blobs in both. I tried out a variant with a >> single enum though, hope you like it. >> The observation was that we only need 3 different variations: >> all_blobs, only_alive or only_alive_and_not_unloading. So I made an >> enum out of that. >> In fact, these three modes are the only ones I support right now, so >> they are also the only valid options, which made the single enum look >> even more reasonable. >> >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>> >>> >>> I'm not sure if metadata_do() should filter out unloading. It's >>> looking for "old" methods to stop from deallocating memory for them >>> (but it's in a safepoint). If you're unloading, the IC caches are >>> already cleaned out? >> >> IC caches to unloading nmethods are not necessarily cleaned out. The >> concurrent GC threads will clean the IC caches concurrently, but a >> JavaThread can call the IC cache before the GC has gotten around to >> cleaning the IC cache. But any such call into an is_unloading() >> nmethod will be trapped by nmethod entry barriers, which will call >> the IC miss handler, which sorts things out lazily. So you can view >> this as the IC caches to is_unloading() nmethods being logically >> cleared, but not necessarily physically cleared yet. The effect is >> the same - the is_unloading() nmethods may not be entered from >> anywhere, including IC caches. >> >> So if you were to not filter is_unloading(), you would get a whole >> bunch of nmethods that are not really reachable by the application, >> i.e. they are not on any stacks, and may not be entered by the >> application. Therefore, I think it's perfectly fine to filter them >> out here, unless I missed something. And this makes the behaviour >> closer to as-if the nmethods were indeed unloaded in the safepoint. >> >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>> >>> >>> 1108 // The release is only needed for compile-time ordering, as >>> accesses >>> 1109 // into the nmethod after the store is not safe, due to the >>> sweeper >>> 1110 // being allowed to free it then, in the case of concurrent >>> nmethod >>> 1111 // unloading. Therefore, there is no need for acquire on the >>> loader side. >>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>> >>> I tried to make sense out of this first sentence, but couldn't >>> really. After the store to unloaded, can the sweeper free the >>> nmethod? maybe remove "then, " and it would make more sense? >> >> Right, after the store is observed, the sweeper thread may >> concurrently observe the nmethod to be unloaded, and then it may >> immediately flip it over to zombie. And I don't want any concurrent >> make_unloaded code in that nmethod to still be racing after the >> sweeper turns it into zombie; a very unnecessary race to deal with. >> >> I updated the comment - hopefully it reads better now. >> >> Thanks, >> /Erik >> >>> Thanks, >>> Coleen >>> >>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>> ..put in bug number in subject to make mail filters happy. >>>> >>>> /Erik >>>> >>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> All current GCs perform code cache unloading in safepoints. >>>>> Therefore, nmethods that are alive but is_unloading() are never >>>>> observed outside of safepoints. With concurrent class unloading, >>>>> nmethods that are alive but is_unloading() will become observable >>>>> outside of safepoints. This must be handled appropriately. >>>>> >>>>> In this patch I changed the nmethod/compiled method iterators to >>>>> accept parameters describing whether they should filter out not >>>>> is_alive() or is_unloading() methods. Since there is no obvious >>>>> default (all combinations are used depending on call site), you >>>>> have to explicitly set what iteration mode you want. >>>>> >>>>> Other than that, I make sure that the sweeper stays away from >>>>> is_unloading() nmethods that are not yet is_unloaded(). To make >>>>> the interactions between the sweeper and concurrent GC threads >>>>> safe, I had to move down the store that sets the state to >>>>> unloaded, and use a release_store there, to make sure no accesses >>>>> float below it at compile-time. Once that store is observed, >>>>> nmethods may be deleted. >>>>> >>>>> In the IC miss handler, I also need to lazily clean stale IC >>>>> caches due to calling is_unloading nmethods using nmethod entry >>>>> barriers. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>> >>>>> Thanks, >>>>> /Erik >>> >> From shade at redhat.com Tue Nov 20 16:16:14 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 20 Nov 2018 17:16:14 +0100 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: <5BF2E012.5010701@oracle.com> References: <5BF2D678.3030208@oracle.com> <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> <5BF2E012.5010701@oracle.com> Message-ID: On 11/19/18 5:08 PM, Erik ?sterlund wrote: > Thanks for the review. I liked your idea of having the casting done by the accessors, like we > did for GC data on Thread. So here is a new webrev with that approach: > http://cr.openjdk.java.net/~eosterlund/8214056/webrev.01/ That was not my idea, but I like the hidden casts. Looks good. -Aleksey From shade at redhat.com Tue Nov 20 16:17:48 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 20 Nov 2018 17:17:48 +0100 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> Message-ID: <05bdab4a-39bf-6a05-2cb9-53b82e95b08b@redhat.com> On 11/19/18 3:43 PM, Daniel D. Daugherty wrote: > On 11/19/18 6:23 AM, Aleksey Shipilev wrote: >> On 11/19/18 6:08 AM, David Holmes wrote: >>>> David suggested making the option "product", but I don't like it very much: "product" option >>>> suggests this is a production-grade feature and it comes with expectations of support, which is >>>> interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode >>>> available in >>>> product builds without the attached notion of support. Users that want fail-fast VM crash can then >>>> use that option on "we know what we are doing" basis. >>> I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported >>> product". If you really want to fail-fast in production then you're looking for a production flag >>> not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. >> Yup, let's hear it. I don't want to bikeshed this too much :) > > If I was doing the work on this bug, I would have made it a diagnostic option > also. I have a bias towards adding options as diagnostic for two reasons: > > 1) Easier to get rid of the option if it's a bad idea. > 2) You have to use the '-XX:UnlockDiagnosticVMOption' option to use it > ?? which makes unintentional use more difficult. Thanks. No further comments? David, are you happy? I am pushing this: diff -r 0811cc0ab16e src/hotspot/share/runtime/globals.hpp --- a/src/hotspot/share/runtime/globals.hpp Tue Nov 20 16:18:12 2018 +0100 +++ b/src/hotspot/share/runtime/globals.hpp Tue Nov 20 17:17:35 2018 +0100 @@ -496,12 +496,12 @@ \ product(bool, SafepointTimeout, false, \ "Time out and warn or fail after SafepointTimeoutDelay " \ "milliseconds if failed to reach safepoint") \ \ - develop(bool, DieOnSafepointTimeout, false, \ - "Die upon failure to reach safepoint (see SafepointTimeout)") \ + diagnostic(bool, AbortVMOnSafepointTimeout, false, \ + "Abort upon failure to reach safepoint (see SafepointTimeout)") \ \ /* 50 retries * (5 * current_retry_count) millis = ~6.375 seconds */ \ /* typically, at most a few retries are needed */ \ product(intx, SuspendRetryCount, 50, \ "Maximum retry count for an external suspend request") \ diff -r 0811cc0ab16e src/hotspot/share/runtime/safepoint.cpp --- a/src/hotspot/share/runtime/safepoint.cpp Tue Nov 20 16:18:12 2018 +0100 +++ b/src/hotspot/share/runtime/safepoint.cpp Tue Nov 20 17:17:35 2018 +0100 @@ -976,13 +976,13 @@ } ls.print_cr("# SafepointSynchronize::begin: (End of list)"); } } - // To debug the long safepoint, specify both DieOnSafepointTimeout & + // To debug the long safepoint, specify both AbortVMOnSafepointTimeout & // ShowMessageBoxOnError. - if (DieOnSafepointTimeout) { + if (AbortVMOnSafepointTimeout) { fatal("Safepoint sync time longer than " INTX_FORMAT "ms detected when executing %s.", SafepointTimeoutDelay, VMThread::vm_safepoint_description()); } } Thanks, -Aleksey From erik.osterlund at oracle.com Tue Nov 20 16:18:32 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 20 Nov 2018 17:18:32 +0100 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: References: <5BF2D678.3030208@oracle.com> <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> <5BF2E012.5010701@oracle.com> Message-ID: <5BF433D8.8010201@oracle.com> Hi Aleksey, Thanks for the review. /Erik On 2018-11-20 17:16, Aleksey Shipilev wrote: > On 11/19/18 5:08 PM, Erik ?sterlund wrote: >> Thanks for the review. I liked your idea of having the casting done by the accessors, like we >> did for GC data on Thread. So here is a new webrev with that approach: >> http://cr.openjdk.java.net/~eosterlund/8214056/webrev.01/ > That was not my idea, but I like the hidden casts. Looks good. > > -Aleksey > From coleen.phillimore at oracle.com Tue Nov 20 16:20:23 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 20 Nov 2018 11:20:23 -0500 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: <05bdab4a-39bf-6a05-2cb9-53b82e95b08b@redhat.com> References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> <05bdab4a-39bf-6a05-2cb9-53b82e95b08b@redhat.com> Message-ID: <1fb3a87e-c14b-1101-844d-d541b9515007@oracle.com> This looks good to me.? I like that it's a diagnostic option so we can use it in product mode. Coleen On 11/20/18 11:17 AM, Aleksey Shipilev wrote: > On 11/19/18 3:43 PM, Daniel D. Daugherty wrote: >> On 11/19/18 6:23 AM, Aleksey Shipilev wrote: >>> On 11/19/18 6:08 AM, David Holmes wrote: >>>>> David suggested making the option "product", but I don't like it very much: "product" option >>>>> suggests this is a production-grade feature and it comes with expectations of support, which is >>>>> interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode >>>>> available in >>>>> product builds without the attached notion of support. Users that want fail-fast VM crash can then >>>>> use that option on "we know what we are doing" basis. >>>> I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported >>>> product". If you really want to fail-fast in production then you're looking for a production flag >>>> not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. >>> Yup, let's hear it. I don't want to bikeshed this too much :) >> If I was doing the work on this bug, I would have made it a diagnostic option >> also. I have a bias towards adding options as diagnostic for two reasons: >> >> 1) Easier to get rid of the option if it's a bad idea. >> 2) You have to use the '-XX:UnlockDiagnosticVMOption' option to use it >> ?? which makes unintentional use more difficult. > Thanks. No further comments? David, are you happy? I am pushing this: > > diff -r 0811cc0ab16e src/hotspot/share/runtime/globals.hpp > --- a/src/hotspot/share/runtime/globals.hpp Tue Nov 20 16:18:12 2018 +0100 > +++ b/src/hotspot/share/runtime/globals.hpp Tue Nov 20 17:17:35 2018 +0100 > @@ -496,12 +496,12 @@ > \ > product(bool, SafepointTimeout, false, \ > "Time out and warn or fail after SafepointTimeoutDelay " \ > "milliseconds if failed to reach safepoint") \ > \ > - develop(bool, DieOnSafepointTimeout, false, \ > - "Die upon failure to reach safepoint (see SafepointTimeout)") \ > + diagnostic(bool, AbortVMOnSafepointTimeout, false, \ > + "Abort upon failure to reach safepoint (see SafepointTimeout)") \ > \ > /* 50 retries * (5 * current_retry_count) millis = ~6.375 seconds */ \ > /* typically, at most a few retries are needed */ \ > product(intx, SuspendRetryCount, 50, \ > "Maximum retry count for an external suspend request") \ > diff -r 0811cc0ab16e src/hotspot/share/runtime/safepoint.cpp > --- a/src/hotspot/share/runtime/safepoint.cpp Tue Nov 20 16:18:12 2018 +0100 > +++ b/src/hotspot/share/runtime/safepoint.cpp Tue Nov 20 17:17:35 2018 +0100 > @@ -976,13 +976,13 @@ > } > ls.print_cr("# SafepointSynchronize::begin: (End of list)"); > } > } > > - // To debug the long safepoint, specify both DieOnSafepointTimeout & > + // To debug the long safepoint, specify both AbortVMOnSafepointTimeout & > // ShowMessageBoxOnError. > - if (DieOnSafepointTimeout) { > + if (AbortVMOnSafepointTimeout) { > fatal("Safepoint sync time longer than " INTX_FORMAT "ms detected when executing %s.", > SafepointTimeoutDelay, VMThread::vm_safepoint_description()); > } > } > > > Thanks, > -Aleksey > > From coleen.phillimore at oracle.com Tue Nov 20 16:22:08 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 20 Nov 2018 11:22:08 -0500 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <5BF43076.4010305@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <5BF43076.4010305@oracle.com> Message-ID: <3a0d691e-5366-f6e2-2086-02a51e43e0c0@oracle.com> On 11/20/18 11:04 AM, Erik ?sterlund wrote: > Hi Coleen, > > Thanks for the review. > > On 2018-11-19 23:42, coleen.phillimore at oracle.com wrote: >> >> >> On 11/19/18 7:31 AM, Erik ?sterlund wrote: >>> Hi Coleen, >>> >>> Thanks for having a look at this. >>> >>> New full webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>> >>> Incremental: >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>> >>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>> >>>> How about instead of two bool arguments that are hard to read and >>>> error prone and need comments, like: >>>> >>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >>>> only_not_unloading */); >>>> >>>> >>>> enum FilterAlive { all_blobs, only_alive }; >>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>> >>>> Then it can be something like: >>>> >>>> ?? CompiledMethodIterator iter(only_alive, all_blobs); >>>> >>>> Don't know if you can repeate all_blobs like that. >>> >>> You can't really have all_blobs in both. I tried out a variant with >>> a single enum though, hope you like it. >>> The observation was that we only need 3 different variations: >>> all_blobs, only_alive or only_alive_and_not_unloading. So I made an >>> enum out of that. >>> In fact, these three modes are the only ones I support right now, so >>> they are also the only valid options, which made the single enum >>> look even more reasonable. >> >> I do like it! >>> >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>> >>>> >>>> I'm not sure if metadata_do() should filter out unloading. It's >>>> looking for "old" methods to stop from deallocating memory for them >>>> (but it's in a safepoint).? If you're unloading, the IC caches are >>>> already cleaned out? >>> >>> IC caches to unloading nmethods are not necessarily cleaned out. The >>> concurrent GC threads will clean the IC caches concurrently, but a >>> JavaThread can call the IC cache before the GC has gotten around to >>> cleaning the IC cache. But any such call into an is_unloading() >>> nmethod will be trapped by nmethod entry barriers, which will call >>> the IC miss handler, which sorts things out lazily. So you can view >>> this as the IC caches to is_unloading() nmethods being logically >>> cleared, but not necessarily physically cleared yet. The effect is >>> the same - the is_unloading() nmethods may not be entered from >>> anywhere, including IC caches. >>> >>> So if you were to not filter is_unloading(), you would get a whole >>> bunch of nmethods that are not really reachable by the application, >>> i.e. they are not on any stacks, and may not be entered by the >>> application. Therefore, I think it's perfectly fine to filter them >>> out here, unless I missed something. And this makes the behaviour >>> closer to as-if the nmethods were indeed unloaded in the safepoint. >> >> Ok.? The thing I'm worried about is if we deallocate memory for a >> Method* in the is_unloading methods, and some other cleaning or >> sweeping thread tries to access that memory.? This is used by >> MetadataOnStackMark during safepoints to see what's safe to >> deallocate from redefinition. > > I understand. I do think this is safe though, and it only walked > is_alive() compiled methods looking for metadata before, e.g. not > unloaded or zombie methods. Similarly, nobody should be poking around > at the metadata of unloading methods. Should being the key word there > I guess... > >> >>> >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>> >>>> >>>> 1108 // The release is only needed for compile-time ordering, as >>>> accesses >>>> 1109 // into the nmethod after the store is not safe, due to the >>>> sweeper >>>> 1110 // being allowed to free it then, in the case of concurrent >>>> nmethod >>>> 1111 // unloading. Therefore, there is no need for acquire on the >>>> loader side. >>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>> >>>> I tried to make sense out of this first sentence, but couldn't >>>> really.? After the store to unloaded, can the sweeper free the >>>> nmethod?? maybe remove "then, " and it would make more sense? >>> >>> Right, after the store is observed, the sweeper thread may >>> concurrently observe the nmethod to be unloaded, and then it may >>> immediately flip it over to zombie. And I don't want any concurrent >>> make_unloaded code in that nmethod to still be racing after the >>> sweeper turns it into zombie; a very unnecessary race to deal with. >>> >>> I updated the comment - hopefully it reads better now. >>> >> >> How about removing a comma changing it to accesses are not safe, or >> access is not safe. >> + // The release is only needed for compile-time ordering, as accesses >> + // into the nmethod after the store are not safe, due to the sweeper >> + // being allowed to free it when the store is observed during >> + // concurrent nmethod unloading. Therefore, there is no need for >> + // acquire on the loader side. > > Comma will be removed before pushing. > >> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/src/hotspot/share/code/codeCache.hpp.udiff.html >> >> Why is this copy constructor defined?? Isn't this what the default >> copy constructor would do? > > Yeah it is. I like explicitly typing out what constructors should do. > But I can remove it if you prefer that. Can you remove it?? It's confusing there since we don't generally have default constructors typed out, one has to study it to see that it's a default copy constructor.? Then ask myself why is it there. I don't need to see another webrev. Thanks, Coleen > >> It looks almost perfect to me :) > > Awesome. Thanks for reviewing this. > > Thanks, > /Erik > >> Coleen >> >>> Thanks, >>> /Erik >>> >>>> Thanks, >>>> Coleen >>>> >>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>> ..put in bug number in subject to make mail filters happy. >>>>> >>>>> /Erik >>>>> >>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> All current GCs perform code cache unloading in safepoints. >>>>>> Therefore, nmethods that are alive but is_unloading() are never >>>>>> observed outside of safepoints. With concurrent class unloading, >>>>>> nmethods that are alive but is_unloading() will become observable >>>>>> outside of safepoints. This must be handled appropriately. >>>>>> >>>>>> In this patch I changed the nmethod/compiled method iterators to >>>>>> accept parameters describing whether they should filter out not >>>>>> is_alive() or is_unloading() methods. Since there is no obvious >>>>>> default (all combinations are used depending on call site), you >>>>>> have to explicitly set what iteration mode you want. >>>>>> >>>>>> Other than that, I make sure that the sweeper stays away from >>>>>> is_unloading() nmethods that are not yet is_unloaded(). To make >>>>>> the interactions between the sweeper and concurrent GC threads >>>>>> safe, I had to move down the store that sets the state to >>>>>> unloaded, and use a release_store there, to make sure no accesses >>>>>> float below it at compile-time. Once that store is observed, >>>>>> nmethods may be deleted. >>>>>> >>>>>> In the IC miss handler, I also need to lazily clean stale IC >>>>>> caches due to calling is_unloading nmethods using nmethod entry >>>>>> barriers. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>> >>> >> > From erik.osterlund at oracle.com Tue Nov 20 16:27:16 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 20 Nov 2018 17:27:16 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <3a0d691e-5366-f6e2-2086-02a51e43e0c0@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <5BF43076.4010305@oracle.com> <3a0d691e-5366-f6e2-2086-02a51e43e0c0@oracle.com> Message-ID: <5BF435E4.7000200@oracle.com> Hi Coleen, Will remove the constructor before pushing. Thanks for the review. /Erik On 2018-11-20 17:22, coleen.phillimore at oracle.com wrote: > > > On 11/20/18 11:04 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thanks for the review. >> >> On 2018-11-19 23:42, coleen.phillimore at oracle.com wrote: >>> >>> >>> On 11/19/18 7:31 AM, Erik ?sterlund wrote: >>>> Hi Coleen, >>>> >>>> Thanks for having a look at this. >>>> >>>> New full webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>>> >>>> Incremental: >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>>> >>>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>>> >>>>> How about instead of two bool arguments that are hard to read and >>>>> error prone and need comments, like: >>>>> >>>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >>>>> only_not_unloading */); >>>>> >>>>> >>>>> enum FilterAlive { all_blobs, only_alive }; >>>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>>> >>>>> Then it can be something like: >>>>> >>>>> CompiledMethodIterator iter(only_alive, all_blobs); >>>>> >>>>> Don't know if you can repeate all_blobs like that. >>>> >>>> You can't really have all_blobs in both. I tried out a variant with >>>> a single enum though, hope you like it. >>>> The observation was that we only need 3 different variations: >>>> all_blobs, only_alive or only_alive_and_not_unloading. So I made an >>>> enum out of that. >>>> In fact, these three modes are the only ones I support right now, >>>> so they are also the only valid options, which made the single enum >>>> look even more reasonable. >>> >>> I do like it! >>>> >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>>> >>>>> >>>>> I'm not sure if metadata_do() should filter out unloading. It's >>>>> looking for "old" methods to stop from deallocating memory for >>>>> them (but it's in a safepoint). If you're unloading, the IC caches >>>>> are already cleaned out? >>>> >>>> IC caches to unloading nmethods are not necessarily cleaned out. >>>> The concurrent GC threads will clean the IC caches concurrently, >>>> but a JavaThread can call the IC cache before the GC has gotten >>>> around to cleaning the IC cache. But any such call into an >>>> is_unloading() nmethod will be trapped by nmethod entry barriers, >>>> which will call the IC miss handler, which sorts things out lazily. >>>> So you can view this as the IC caches to is_unloading() nmethods >>>> being logically cleared, but not necessarily physically cleared >>>> yet. The effect is the same - the is_unloading() nmethods may not >>>> be entered from anywhere, including IC caches. >>>> >>>> So if you were to not filter is_unloading(), you would get a whole >>>> bunch of nmethods that are not really reachable by the application, >>>> i.e. they are not on any stacks, and may not be entered by the >>>> application. Therefore, I think it's perfectly fine to filter them >>>> out here, unless I missed something. And this makes the behaviour >>>> closer to as-if the nmethods were indeed unloaded in the safepoint. >>> >>> Ok. The thing I'm worried about is if we deallocate memory for a >>> Method* in the is_unloading methods, and some other cleaning or >>> sweeping thread tries to access that memory. This is used by >>> MetadataOnStackMark during safepoints to see what's safe to >>> deallocate from redefinition. >> >> I understand. I do think this is safe though, and it only walked >> is_alive() compiled methods looking for metadata before, e.g. not >> unloaded or zombie methods. Similarly, nobody should be poking around >> at the metadata of unloading methods. Should being the key word there >> I guess... >> >>> >>>> >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>>> >>>>> >>>>> 1108 // The release is only needed for compile-time ordering, as >>>>> accesses >>>>> 1109 // into the nmethod after the store is not safe, due to the >>>>> sweeper >>>>> 1110 // being allowed to free it then, in the case of concurrent >>>>> nmethod >>>>> 1111 // unloading. Therefore, there is no need for acquire on the >>>>> loader side. >>>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>>> >>>>> I tried to make sense out of this first sentence, but couldn't >>>>> really. After the store to unloaded, can the sweeper free the >>>>> nmethod? maybe remove "then, " and it would make more sense? >>>> >>>> Right, after the store is observed, the sweeper thread may >>>> concurrently observe the nmethod to be unloaded, and then it may >>>> immediately flip it over to zombie. And I don't want any concurrent >>>> make_unloaded code in that nmethod to still be racing after the >>>> sweeper turns it into zombie; a very unnecessary race to deal with. >>>> >>>> I updated the comment - hopefully it reads better now. >>>> >>> >>> How about removing a comma changing it to accesses are not safe, or >>> access is not safe. >>> + // The release is only needed for compile-time ordering, as accesses >>> + // into the nmethod after the store are not safe, due to the sweeper >>> + // being allowed to free it when the store is observed during >>> + // concurrent nmethod unloading. Therefore, there is no need for >>> + // acquire on the loader side. >> >> Comma will be removed before pushing. >> >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/src/hotspot/share/code/codeCache.hpp.udiff.html >>> >>> Why is this copy constructor defined? Isn't this what the default >>> copy constructor would do? >> >> Yeah it is. I like explicitly typing out what constructors should do. >> But I can remove it if you prefer that. > > Can you remove it? It's confusing there since we don't generally have > default constructors typed out, one has to study it to see that it's a > default copy constructor. Then ask myself why is it there. > > I don't need to see another webrev. > > Thanks, > Coleen >> >>> It looks almost perfect to me :) >> >> Awesome. Thanks for reviewing this. >> >> Thanks, >> /Erik >> >>> Coleen >>> >>>> Thanks, >>>> /Erik >>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>>> ..put in bug number in subject to make mail filters happy. >>>>>> >>>>>> /Erik >>>>>> >>>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>>> Hi, >>>>>>> >>>>>>> All current GCs perform code cache unloading in safepoints. >>>>>>> Therefore, nmethods that are alive but is_unloading() are never >>>>>>> observed outside of safepoints. With concurrent class unloading, >>>>>>> nmethods that are alive but is_unloading() will become >>>>>>> observable outside of safepoints. This must be handled >>>>>>> appropriately. >>>>>>> >>>>>>> In this patch I changed the nmethod/compiled method iterators to >>>>>>> accept parameters describing whether they should filter out not >>>>>>> is_alive() or is_unloading() methods. Since there is no obvious >>>>>>> default (all combinations are used depending on call site), you >>>>>>> have to explicitly set what iteration mode you want. >>>>>>> >>>>>>> Other than that, I make sure that the sweeper stays away from >>>>>>> is_unloading() nmethods that are not yet is_unloaded(). To make >>>>>>> the interactions between the sweeper and concurrent GC threads >>>>>>> safe, I had to move down the store that sets the state to >>>>>>> unloaded, and use a release_store there, to make sure no >>>>>>> accesses float below it at compile-time. Once that store is >>>>>>> observed, nmethods may be deleted. >>>>>>> >>>>>>> In the IC miss handler, I also need to lazily clean stale IC >>>>>>> caches due to calling is_unloading nmethods using nmethod entry >>>>>>> barriers. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>> >>>> >>> >> > From david.griffiths at gmail.com Tue Nov 20 16:33:12 2018 From: david.griffiths at gmail.com (David Griffiths) Date: Tue, 20 Nov 2018 16:33:12 +0000 Subject: Access to jdk.hotspot.agent classes in Java 9? Message-ID: Hi, I'm writing some code that makes use of the SA classes under sun.jvm.hotspot. This compiles and runs fine under Java 8 provided I include sa-jdi.jar but I've just discovered that these are now classified as JDK internal API in Java 9. There is no alternative so what should I do? Thanks, David From daniel.daugherty at oracle.com Tue Nov 20 16:48:55 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 20 Nov 2018 11:48:55 -0500 Subject: Access to jdk.hotspot.agent classes in Java 9? In-Reply-To: References: Message-ID: <1a208b25-8145-a005-0c8d-5bbab3a6f867@oracle.com> Adding serviceability-dev at ... since this is the Serviceability Agent... Dan On 11/20/18 11:33 AM, David Griffiths wrote: > Hi, I'm writing some code that makes use of the SA classes under > sun.jvm.hotspot. This compiles and runs fine under Java 8 provided I > include sa-jdi.jar but I've just discovered that these are now classified > as JDK internal API in Java 9. There is no alternative so what should I do? > > Thanks, > > David From vladimir.kozlov at oracle.com Tue Nov 20 18:08:29 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Nov 2018 10:08:29 -0800 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: <5BF3EC1F.8060003@oracle.com> References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> Message-ID: To clarify. 'invalid' state is set only for AOT methods which never was registers (their state is 'not_set' and corresponding class was not initialized yet). Such methods are never visible to iterators and IC caches. If aot method is registered its state become 'in_use' and in such case it is treated and processed as normal nmethod. And AOTCompiledMethod structure is created only for such case. Vladimir On 11/20/18 3:12 AM, Erik ?sterlund wrote: > Hi Dean, > > Yeah I think I misunderstood what I observed. So what I know is that in the code cache unloading, I'm not getting all > is_alive() AOT methods into my iterator, which messes up the assumptions made by the epoch based scheme for AOT methods. > > I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, int methods_cnt) we make AOTCompiledMethods > "invalid" in the AOT heap, making it no longer observable from the iterators. Then it calls the VM_Deoptimize vm > operation after. Throughout all this, the AOTCompiledMethod is alive(), yet when the iterators ask for all is_alive() > AOTCompiledMethods, it won't be visible. But I suppose IC caches may still reference these methods and check if it > is_unloading, and then we blow up. There may possibly be multiple ways for is_alive() AOTCompiledMethods to not be > visible from iterators yet be visible through IC caches using the "invalid" state in the . > > Anyway, the fix is the same: stop doing the epoch state thingey for is_unloading() on AOTCompiledMethod where it isn't > needed, and doesn't seem to play well with the rather different life cycle it has, and just return false instead. > > Thanks, > /Erik > > On 2018-11-20 00:00, dean.long at oracle.com wrote: >> Hi Erik, >> >> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>> ...except it looks like for AOTCompiledMethods when running with tiered compilation, may first be is_alive(), then >>> become !is_alive() for a while, and then get resurrected to is_alive() using make_entrant(). >> >> this doesn't sounds quite right.? AOTCompiledMethods aren't allowed to transition to zombie (!alive), only not_used or >> not_entrant, which should still have is_alive() returning true. Maybe some code is using is_not_entrant() instead of >> !is_alive()? >> >> dl > From ioi.lam at oracle.com Tue Nov 20 18:10:56 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 20 Nov 2018 10:10:56 -0800 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> <6551274e-f1a0-7040-f425-b64b4610b9c1@oracle.com> Message-ID: <0b349c77-7f64-ba5f-b47a-4a1478d8dfb6@oracle.com> Hi Gerard, I've updated the patch according to your comments. Please take a look. http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v05-delta/ Thanks - Ioi On 11/20/18 7:53 AM, Gerard Ziemski wrote: > hi Ioi, > > Please see my comments in-line. > >> On Nov 20, 2018, at 12:01 AM, Ioi Lam wrote: >> >> Hi Gerard, >> >> Thanks for the review. Please see my comments in-line. >> >> >> On 11/19/18 8:11 AM, Gerard Ziemski wrote: >>> Hi Ioi, >>> >>> Very nice! I only have couple of questions. >>> >>> #1 Regarding ?maybe_grow()? in src/hotspot/share/utilities/hashtable.hpp: >>> >>> 235 // Grow the number of buckets if the average entries per bucket is over the load_factor >>> 236 bool maybe_grow(int load_factor = 8) { >>> 237 if (number_of_entries() / table_size() > load_factor) { >>> 238 resize(table_size() * 2); >>> 239 return true; >>> 240 } else { >>> 241 return false; >>> 242 } >>> 243 } >>> >>> a) A suggestion: from my personal experience this didn?t make a difference, but technically a hash map benefits from a size that is a prime number. Maybe you could make a comparison run, with the code as is vs with prime number when resizing and see if you get an even better performance? >> In my patch, all tables that call maybe_grow started with a table size of a prime number. E.g., in metaspaceShared.cpp: >> >> 1074 typedef KVHashtable< >> 1075 address, address> RelocationTable; >> 1081 _new_loc_table = new RelocationTable(8087); >> >> So hopefully this will at least be better than starting with a simple power of 2. >> >> Anyway, in the LotsOfClasses test, the two tables grew to 254224 and 258784 entries. I tried hard-coding them to start with a large prime number (254249 and 258787 entries, respectively): >> >> BEFORE: 24.004s >> AFTER : 22.476s >> >> The AFTER run also saves all the costs of expanding the tables gradually. So I think doing the simple multiply-by-two is good enough. > Thank you for checking. I do like the simplicity of doubling the size, so if the performance benefit of calculating a prime is small (as I was suspecting), then I?m more than fine with the simple solution. > > I just happened to look at https://wiki.openjdk.java.net/display/HotSpot/StyleGuide yesterday and can?t help, but note here, that according to it, we should ?Assign names to constant literals and use the names instead?, so we should replace the literals like ?8087? with ?static const size_t INITIAL_TABLE_SIZE = 8087? (I know this is a pre-existent code) > >>> b) I am a little worried that we don?t have an upper bound here. In our other hash table we always have upper bounds (or don?t resize). >> With 35,000 classes, the tables will have over 1.5 million entries and we expand the table to about 250,000. I think we should support dumping at least 100,000 classes. That means the upper bound for the table size should probably be at least 1,000,000. >> >> What do you think? > If someone hits this bound and it causes an issue, they can tell us and we can always increase it later, but with your fix and the proposed bound, it?s way much better compared to the current behavior. > > >>> c) Is this table only used from one thread? We are guaranteed here that no other thread will insert/delete anything while we resize? >> Currently maybe_grow() is called only within a safepoint. resize() already has an assert for safepoint. I will add the same assert into maybe_grow: >> >> >> bool maybe_grow(int load_factor = 8) { >> + assert(SafepointSynchronize::is_at_safepoint(), "must be at safepoint"); >> if (number_of_entries() / table_size() > load_factor) { > Thanks. > > >> Thanks >> - Ioi >> >>> Cheers >>> >>>> On Nov 18, 2018, at 9:50 PM, Ioi Lam wrote: >>>> >>>> >>>> >>>> On 11/17/18 10:28 PM, Jiangli Zhou wrote: >>>>> Hi Ioi, >>>>> >>>>> On 11/17/18 6:48 PM, Ioi Lam wrote: >>>>> >>>>>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>>>>> Hi Ioi, >>>>>>> >>>>>>> This looks good. Glad to see the change that utilizes the existing BasicHashtable. Thanks Coleen for the suggestion! >>>>>>> >>>>>>> To avoid changing the existing assumptions about BasicHashtable and Hashtable, how about adding a destructor to the new KVHashtable instead of BasicHashtable? Just did a quick test, which seems to work properly. >>>>>>> >>>>>> Hi Jiangli, >>>>>> >>>>>> Thanks for the review. >>>>>> >>>>>> Not freeing the entries when a BasicHashtable is destructed is a bug. None of the existing BasicHashtables (and subclasses thereof) are ever destructed, so I don't think my code would impact them. Also, if any code depends on the entries not freed even if the table is destructed, that's clearly a bug in that code, and it should be fixed. >>>>>> >>>>>> If I don't add the destructor to BasicHashtable, the next person who wants to allocate/free BasicHashtables will run in to the same issue. >>>>> Dictionary, PackageEntryTable, ModuleEntryTable and G1CodeRootSetTable are derived from Hashtable and they call free_buckets() in their destructors. So free_buckets() will be called twice when those tables are freed although it's harmless. If you want to avoid the duplicated call, you could remove the free_buckets() calls from those classes' destructors and keep the current BasicHashtables changes. I don't have a strong opinion on this, I'll leave it to you to decide. >>>>> >>>>> >>>> Hi Jiangli, >>>> >>>> Good catch. I've remove the call to free_buckets() in the destructor of those 4 classes. Here's the delta from the last webrev: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ >>>> >>>> Also, I found 2 problems while doing this. Hashtables in HotSpot are really a mess. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8214029 >>>> https://bugs.openjdk.java.net/browse/JDK-8214030 >>>> >>>> Thanks >>>> - Ioi >>>> >>>>> Thanks, >>>>> >>>>> Jiangli >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jiangli >>>>>>> >>>>>>> >>>>>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> I deleted the default value for MEMFLAGS as you suggested. For my instantiated templates, I still use mtInternal, though, since I can't find anything better for using at CDS dump time. >>>>>>>> >>>>>>>> Also, Jiangli noted that there's a memory leak, because I allocate and free the KVHashtable dynamically. So I added a destructor to BasicHashtable to free the buckets and the block-allocated entries. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>>>>> >>>>>>>> This comment around Hashtable::allocate_new_entry() is wrong now -- "The allocator in blocks is preferable but doesn't have free semantics". Maybe I should change it to this? >>>>>>>> >>>>>>>> "The block allocator in BasicHashtable has less fragmentation, but the memory is not freed until the whole table is freed. Use allocate_new_entry() if you want to immediately free the memory used by each entry". >>>>>>>> >>>>>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested the solaris/x64 build since it seems to break every time you do something with templates :-( >>>>>>>> >>>>>>>> Thanks! >>>>>>>> - Ioi >>>>>>>> >>>>>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>>>>> Hi Ioi, I really like this new wrapper on the old hashtable to not have to write the boilerplate code! >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>>>>> >>>>>>>>> + MEMFLAGS F = mtInternal, >>>>>>>>> >>>>>>>>> >>>>>>>>> I think you should require the mt type though and not make it a default parameter. mtInternal is not very useful to finding memory leaks. >>>>>>>>> >>>>>>>>> Apart from this (which I don't need to see another version), your change looks good and nice to get good performance benefits from this. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>>>>> Coleen pointed out to me off-line that the good old (and ugly) BasicHashtable already supports resizing. I think that might be a better starting point for this RFE: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>>>>> >>>>>>>>>> I wrote a new template class called "KVHashtable" (copying the style from ResourceHashtable). That way, you can instantiate different (Key -> Value) mappings without writing tons of boilerplate code. The performance is similar to my previous version, and the code is much cleaner. >>>>>>>>>> >>>>>>>>>> I also renamed the RFE title, as well as the subject line of this RFR e-mail. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> - Ioi >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>>>>> Hi Ioi, >>>>>>>>>>> >>>>>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>>>>> Hi Ioi, >>>>>>>>>>>>> >>>>>>>>>>>>> The change looks reasonable to me in general. It would be helpful to see the performance difference with the expendable table. Do you have any data when large number of classes are loaded (>20000)? How much saving does it provide? >>>>>>>>>>>>> >>>>>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>>>>> >>>>>>>>>>>> BEFORE: 93.971 sec >>>>>>>>>>>> AFTER: 34.761 sec >>>>>>>>>>> Thanks for the data! That's about 2.6x improvement with large set of classes. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jiangli >>>>>>>>>>>> Thanks >>>>>>>>>>>> - Ioi >>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Jiangli >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table size to be >>>>>>>>>>>>>> dynamically specified when the table is constructed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> *** C++ template guru alert *** >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't know much about C++ templates, so my attempt on doing this may be >>>>>>>>>>>>>> ill-advised. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I *think* that with my patch, the performance of existing code, which uses >>>>>>>>>>>>>> a statically-defined SIZE, should not be affected, as the C++ compiler >>>>>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ALWAYSINLINE unsigned size() const { >>>>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>>>> return SIZE; >>>>>>>>>>>>>> } else { >>>>>>>>>>>>>> return _configured_table_size; >>>>>>>>>>>>>> } >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> ALWAYSINLINE Node** get_table() const { >>>>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>>>> return (Node**)(&_static_table[0]); >>>>>>>>>>>>>> } else { >>>>>>>>>>>>>> return _configured_table; >>>>>>>>>>>>>> } >>>>>>>>>>>>>> } >>>>>>>>>>>>>> >>>>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>>>> unsigned index = hash % size(); <----- >>>>>>>>>>>>>> Node** table = get_table(); >>>>>>>>>>>>>> Node** ptr = &table[index]; <----- >>>>>>>>>>>>>> >>>>>>>>>>>>>> back to the old code: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>>>> unsigned index = hash % SIZE; <----- >>>>>>>>>>>>>> Node** ptr = &_table[index]; <----- >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>> - Ioi From dean.long at oracle.com Tue Nov 20 18:21:33 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 20 Nov 2018 10:21:33 -0800 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: <5BF3EC1F.8060003@oracle.com> References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> Message-ID: OK.? I agree it makes sense to move the unloading from CompiledMethod to nmethod.? I was just curious what problem you were running into with AOT, because the iterators look OK to me.? You probably ran into trouble here: // Clean inline caches pointing to both zombie and not_entrant methods *if* (clean_all || !nm ->is_in_use () || nm ->is_unloading () || (nm ->method ()->code () != nm )) { because !is_in_use() would be false for the not_used state. dl On 11/20/18 3:12 AM, Erik ?sterlund wrote: > Hi Dean, > > Yeah I think I misunderstood what I observed. So what I know is that > in the code cache unloading, I'm not getting all is_alive() AOT > methods into my iterator, which messes up the assumptions made by the > epoch based scheme for AOT methods. > > I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, > int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT heap, > making it no longer observable from the iterators. Then it calls the > VM_Deoptimize vm operation after. Throughout all this, the > AOTCompiledMethod is alive(), yet when the iterators ask for all > is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC > caches may still reference these methods and check if it is_unloading, > and then we blow up. There may possibly be multiple ways for > is_alive() AOTCompiledMethods to not be visible from iterators yet be > visible through IC caches using the "invalid" state in the . > > Anyway, the fix is the same: stop doing the epoch state thingey for > is_unloading() on AOTCompiledMethod where it isn't needed, and doesn't > seem to play well with the rather different life cycle it has, and > just return false instead. > > Thanks, > /Erik > > On 2018-11-20 00:00, dean.long at oracle.com wrote: >> Hi Erik, >> >> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>> ...except it looks like for AOTCompiledMethods when running with >>> tiered compilation, may first be is_alive(), then become !is_alive() >>> for a while, and then get resurrected to is_alive() using >>> make_entrant(). >> >> this doesn't sounds quite right.? AOTCompiledMethods aren't allowed >> to transition to zombie (!alive), only not_used or not_entrant, which >> should still have is_alive() returning true. Maybe some code is using >> is_not_entrant() instead of !is_alive()? >> >> dl > From gerard.ziemski at oracle.com Tue Nov 20 18:37:21 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Tue, 20 Nov 2018 12:37:21 -0600 Subject: RFR (S) 8213587 - Speed up CDS dump time by using resizable hashtables In-Reply-To: <0b349c77-7f64-ba5f-b47a-4a1478d8dfb6@oracle.com> References: <94667bf9-2e44-b12c-3016-c5b934992b1e@oracle.com> <01202b77-261f-e2c1-95e2-a23599e30113@oracle.com> <3ca57f11-143e-c8d7-7393-c23ee5295d69@oracle.com> <53b2cc79-d219-5002-1fea-db026f4ea90b@oracle.com> <59617d3c-b81d-04e3-5237-d1352cbb81eb@oracle.com> <7a8c982c-8dcb-fe2e-ad09-c5ea25f4ab3c@oracle.com> <64283a85-935d-bc06-9929-c522e83f8b07@oracle.com> <6551274e-f1a0-7040-f425-b64b4610b9c1@oracle.com> <0b349c77-7f64-ba5f-b47a-4a1478d8dfb6@oracle.com> Message-ID: hi Ioi, Looks good. cheers > On Nov 20, 2018, at 12:10 PM, Ioi Lam wrote: > > Hi Gerard, > > I've updated the patch according to your comments. Please take a look. > > http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v05-delta/ > > Thanks > > - Ioi > > > On 11/20/18 7:53 AM, Gerard Ziemski wrote: >> hi Ioi, >> >> Please see my comments in-line. >> >>> On Nov 20, 2018, at 12:01 AM, Ioi Lam wrote: >>> >>> Hi Gerard, >>> >>> Thanks for the review. Please see my comments in-line. >>> >>> >>> On 11/19/18 8:11 AM, Gerard Ziemski wrote: >>>> Hi Ioi, >>>> >>>> Very nice! I only have couple of questions. >>>> >>>> #1 Regarding ?maybe_grow()? in src/hotspot/share/utilities/hashtable.hpp: >>>> >>>> 235 // Grow the number of buckets if the average entries per bucket is over the load_factor >>>> 236 bool maybe_grow(int load_factor = 8) { >>>> 237 if (number_of_entries() / table_size() > load_factor) { >>>> 238 resize(table_size() * 2); >>>> 239 return true; >>>> 240 } else { >>>> 241 return false; >>>> 242 } >>>> 243 } >>>> >>>> a) A suggestion: from my personal experience this didn?t make a difference, but technically a hash map benefits from a size that is a prime number. Maybe you could make a comparison run, with the code as is vs with prime number when resizing and see if you get an even better performance? >>> In my patch, all tables that call maybe_grow started with a table size of a prime number. E.g., in metaspaceShared.cpp: >>> >>> 1074 typedef KVHashtable< >>> 1075 address, address> RelocationTable; >>> 1081 _new_loc_table = new RelocationTable(8087); >>> >>> So hopefully this will at least be better than starting with a simple power of 2. >>> >>> Anyway, in the LotsOfClasses test, the two tables grew to 254224 and 258784 entries. I tried hard-coding them to start with a large prime number (254249 and 258787 entries, respectively): >>> >>> BEFORE: 24.004s >>> AFTER : 22.476s >>> >>> The AFTER run also saves all the costs of expanding the tables gradually. So I think doing the simple multiply-by-two is good enough. >> Thank you for checking. I do like the simplicity of doubling the size, so if the performance benefit of calculating a prime is small (as I was suspecting), then I?m more than fine with the simple solution. >> >> I just happened to look at https://wiki.openjdk.java.net/display/HotSpot/StyleGuide yesterday and can?t help, but note here, that according to it, we should ?Assign names to constant literals and use the names instead?, so we should replace the literals like ?8087? with ?static const size_t INITIAL_TABLE_SIZE = 8087? (I know this is a pre-existent code) >> >>>> b) I am a little worried that we don?t have an upper bound here. In our other hash table we always have upper bounds (or don?t resize). >>> With 35,000 classes, the tables will have over 1.5 million entries and we expand the table to about 250,000. I think we should support dumping at least 100,000 classes. That means the upper bound for the table size should probably be at least 1,000,000. >>> >>> What do you think? >> If someone hits this bound and it causes an issue, they can tell us and we can always increase it later, but with your fix and the proposed bound, it?s way much better compared to the current behavior. >> >> >>>> c) Is this table only used from one thread? We are guaranteed here that no other thread will insert/delete anything while we resize? >>> Currently maybe_grow() is called only within a safepoint. resize() already has an assert for safepoint. I will add the same assert into maybe_grow: >>> >>> >>> bool maybe_grow(int load_factor = 8) { >>> + assert(SafepointSynchronize::is_at_safepoint(), "must be at safepoint"); >>> if (number_of_entries() / table_size() > load_factor) { >> Thanks. >> >> >>> Thanks >>> - Ioi >>> >>>> Cheers >>>> >>>>> On Nov 18, 2018, at 9:50 PM, Ioi Lam wrote: >>>>> >>>>> >>>>> >>>>> On 11/17/18 10:28 PM, Jiangli Zhou wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> On 11/17/18 6:48 PM, Ioi Lam wrote: >>>>>> >>>>>>> On 11/17/18 1:54 PM, Jiangli Zhou wrote: >>>>>>>> Hi Ioi, >>>>>>>> >>>>>>>> This looks good. Glad to see the change that utilizes the existing BasicHashtable. Thanks Coleen for the suggestion! >>>>>>>> >>>>>>>> To avoid changing the existing assumptions about BasicHashtable and Hashtable, how about adding a destructor to the new KVHashtable instead of BasicHashtable? Just did a quick test, which seems to work properly. >>>>>>>> >>>>>>> Hi Jiangli, >>>>>>> >>>>>>> Thanks for the review. >>>>>>> >>>>>>> Not freeing the entries when a BasicHashtable is destructed is a bug. None of the existing BasicHashtables (and subclasses thereof) are ever destructed, so I don't think my code would impact them. Also, if any code depends on the entries not freed even if the table is destructed, that's clearly a bug in that code, and it should be fixed. >>>>>>> >>>>>>> If I don't add the destructor to BasicHashtable, the next person who wants to allocate/free BasicHashtables will run in to the same issue. >>>>>> Dictionary, PackageEntryTable, ModuleEntryTable and G1CodeRootSetTable are derived from Hashtable and they call free_buckets() in their destructors. So free_buckets() will be called twice when those tables are freed although it's harmless. If you want to avoid the duplicated call, you could remove the free_buckets() calls from those classes' destructors and keep the current BasicHashtables changes. I don't have a strong opinion on this, I'll leave it to you to decide. >>>>>> >>>>>> >>>>> Hi Jiangli, >>>>> >>>>> Good catch. I've remove the call to free_buckets() in the destructor of those 4 classes. Here's the delta from the last webrev: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v04-delta/ >>>>> >>>>> Also, I found 2 problems while doing this. Hashtables in HotSpot are really a mess. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8214029 >>>>> https://bugs.openjdk.java.net/browse/JDK-8214030 >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>>> Thanks, >>>>>> >>>>>> Jiangli >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Jiangli >>>>>>>> >>>>>>>> >>>>>>>> On 11/16/18 7:55 PM, Ioi Lam wrote: >>>>>>>>> Hi Coleen, >>>>>>>>> >>>>>>>>> I deleted the default value for MEMFLAGS as you suggested. For my instantiated templates, I still use mtInternal, though, since I can't find anything better for using at CDS dump time. >>>>>>>>> >>>>>>>>> Also, Jiangli noted that there's a memory leak, because I allocate and free the KVHashtable dynamically. So I added a destructor to BasicHashtable to free the buckets and the block-allocated entries. >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03/ >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v03-delta/ >>>>>>>>> >>>>>>>>> This comment around Hashtable::allocate_new_entry() is wrong now -- "The allocator in blocks is preferable but doesn't have free semantics". Maybe I should change it to this? >>>>>>>>> >>>>>>>>> "The block allocator in BasicHashtable has less fragmentation, but the memory is not freed until the whole table is freed. Use allocate_new_entry() if you want to immediately free the memory used by each entry". >>>>>>>>> >>>>>>>>> I am rerunning hs-tiers{1,2,3,4} to catch any issues. I also tested the solaris/x64 build since it seems to break every time you do something with templates :-( >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> - Ioi >>>>>>>>> >>>>>>>>> On 11/16/18 1:36 PM, coleen.phillimore at oracle.com wrote: >>>>>>>>>> Hi Ioi, I really like this new wrapper on the old hashtable to not have to write the boilerplate code! >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/src/hotspot/share/utilities/hashtable.hpp.udiff.html >>>>>>>>>> >>>>>>>>>> + MEMFLAGS F = mtInternal, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I think you should require the mt type though and not make it a default parameter. mtInternal is not very useful to finding memory leaks. >>>>>>>>>> >>>>>>>>>> Apart from this (which I don't need to see another version), your change looks good and nice to get good performance benefits from this. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Coleen >>>>>>>>>> >>>>>>>>>> On 11/15/18 12:31 AM, Ioi Lam wrote: >>>>>>>>>>> Coleen pointed out to me off-line that the good old (and ugly) BasicHashtable already supports resizing. I think that might be a better starting point for this RFE: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-resize-cds-hashtables.v02/ >>>>>>>>>>> >>>>>>>>>>> I wrote a new template class called "KVHashtable" (copying the style from ResourceHashtable). That way, you can instantiate different (Key -> Value) mappings without writing tons of boilerplate code. The performance is similar to my previous version, and the code is much cleaner. >>>>>>>>>>> >>>>>>>>>>> I also renamed the RFE title, as well as the subject line of this RFR e-mail. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> - Ioi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 11/14/18 5:14 PM, Jiangli Zhou wrote: >>>>>>>>>>>> Hi Ioi, >>>>>>>>>>>> >>>>>>>>>>>> On 11/14/18 9:09 AM, Ioi Lam wrote: >>>>>>>>>>>>> On 11/13/18 4:05 PM, Jiangli Zhou wrote: >>>>>>>>>>>>>> Hi Ioi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> The change looks reasonable to me in general. It would be helpful to see the performance difference with the expendable table. Do you have any data when large number of classes are loaded (>20000)? How much saving does it provide? >>>>>>>>>>>>>> >>>>>>>>>>>>> Hi Jiangli, thanks for the review. For dumping 30292 classes: >>>>>>>>>>>>> >>>>>>>>>>>>> BEFORE: 93.971 sec >>>>>>>>>>>>> AFTER: 34.761 sec >>>>>>>>>>>> Thanks for the data! That's about 2.6x improvement with large set of classes. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Jiangli >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> - Ioi >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Jiangli >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 11/8/18 10:35 PM, Ioi Lam wrote: >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213587 >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~iklam/jdk12/8213587-configurable-resource-hash.v01/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> TL;DR: -- add a subclass to ResourceHashtable to allow the table size to be >>>>>>>>>>>>>>> dynamically specified when the table is constructed. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *** C++ template guru alert *** >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I don't know much about C++ templates, so my attempt on doing this may be >>>>>>>>>>>>>>> ill-advised. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I *think* that with my patch, the performance of existing code, which uses >>>>>>>>>>>>>>> a statically-defined SIZE, should not be affected, as the C++ compiler >>>>>>>>>>>>>>> should be able to constant-propagate and reduce the new code: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ALWAYSINLINE unsigned size() const { >>>>>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>>>>> return SIZE; >>>>>>>>>>>>>>> } else { >>>>>>>>>>>>>>> return _configured_table_size; >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ALWAYSINLINE Node** get_table() const { >>>>>>>>>>>>>>> if (SIZE != CONFIGURABLE_SIZE) { >>>>>>>>>>>>>>> return (Node**)(&_static_table[0]); >>>>>>>>>>>>>>> } else { >>>>>>>>>>>>>>> return _configured_table; >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>>>>> unsigned index = hash % size(); <----- >>>>>>>>>>>>>>> Node** table = get_table(); >>>>>>>>>>>>>>> Node** ptr = &table[index]; <----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> back to the old code: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Node** lookup_node(unsigned hash, K const& key) { >>>>>>>>>>>>>>> unsigned index = hash % SIZE; <----- >>>>>>>>>>>>>>> Node** ptr = &_table[index]; <----- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If anyone has a better way of doing this, I'd love to hear it! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>> - Ioi > From dean.long at oracle.com Tue Nov 20 19:11:50 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 20 Nov 2018 11:11:50 -0800 Subject: RFR(S) 8213259: [AOT] AOTing java.base fails with "java.lang.AssertionError: no fingerprint for Ljdk/internal/event/Event" Message-ID: <4bdf8604-2980-f35d-df54-32a8cbc79d25@oracle.com> http://cr.openjdk.java.net/~dlong/8213259/webrev/ https://bugs.openjdk.java.net/browse/JDK-8213259 Previously, some fingerprint logic was in ClassFileParser::create_instance_klass, but some was left up to the caller.? This caused problems if the caller didn't compute a needed fingerprinter, which is what happened in JFR's create_new_instance_klass.? The simplifying fix is to move all fingerprint logic into create_instance_klass. dl From ioi.lam at oracle.com Tue Nov 20 19:47:59 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 20 Nov 2018 11:47:59 -0800 Subject: RFR(S) 8213259: [AOT] AOTing java.base fails with "java.lang.AssertionError: no fingerprint for Ljdk/internal/event/Event" In-Reply-To: <4bdf8604-2980-f35d-df54-32a8cbc79d25@oracle.com> References: <4bdf8604-2980-f35d-df54-32a8cbc79d25@oracle.com> Message-ID: <69f8f3d4-061f-2f2e-6eb7-172b8686561e@oracle.com> Hi Dean, the fixes looks good. Thanks for fixing this bug. - Ioi On 11/20/18 11:11 AM, dean.long at oracle.com wrote: > http://cr.openjdk.java.net/~dlong/8213259/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8213259 > > Previously, some fingerprint logic was in > ClassFileParser::create_instance_klass, but some was left up to the > caller.? This caused problems if the caller didn't compute a needed > fingerprinter, which is what happened in JFR's > create_new_instance_klass.? The simplifying fix is to move all > fingerprint logic into create_instance_klass. > > dl From dean.long at oracle.com Tue Nov 20 19:56:40 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 20 Nov 2018 11:56:40 -0800 Subject: RFR(S) 8213259: [AOT] AOTing java.base fails with "java.lang.AssertionError: no fingerprint for Ljdk/internal/event/Event" In-Reply-To: <69f8f3d4-061f-2f2e-6eb7-172b8686561e@oracle.com> References: <4bdf8604-2980-f35d-df54-32a8cbc79d25@oracle.com> <69f8f3d4-061f-2f2e-6eb7-172b8686561e@oracle.com> Message-ID: <2742ce32-a95a-b255-5ec7-a734f549df4e@oracle.com> Thanks Ioi! dl On 11/20/18 11:47 AM, Ioi Lam wrote: > Hi Dean, the fixes looks good. Thanks for fixing this bug. > > - Ioi > > > On 11/20/18 11:11 AM, dean.long at oracle.com wrote: >> http://cr.openjdk.java.net/~dlong/8213259/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8213259 >> >> Previously, some fingerprint logic was in >> ClassFileParser::create_instance_klass, but some was left up to the >> caller.? This caused problems if the caller didn't compute a needed >> fingerprinter, which is what happened in JFR's >> create_new_instance_klass.? The simplifying fix is to move all >> fingerprint logic into create_instance_klass. >> >> dl > From vladimir.kozlov at oracle.com Tue Nov 20 19:58:42 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Nov 2018 11:58:42 -0800 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: <1fb3a87e-c14b-1101-844d-d541b9515007@oracle.com> References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> <05bdab4a-39bf-6a05-2cb9-53b82e95b08b@redhat.com> <1fb3a87e-c14b-1101-844d-d541b9515007@oracle.com> Message-ID: <0fb9700d-8b94-f105-7177-bb5a2453ec90@oracle.com> +1 for diagnostic Vladimir On 11/20/18 8:20 AM, coleen.phillimore at oracle.com wrote: > This looks good to me.? I like that it's a diagnostic option so we can use it in product mode. > Coleen > > On 11/20/18 11:17 AM, Aleksey Shipilev wrote: >> On 11/19/18 3:43 PM, Daniel D. Daugherty wrote: >>> On 11/19/18 6:23 AM, Aleksey Shipilev wrote: >>>> On 11/19/18 6:08 AM, David Holmes wrote: >>>>>> David suggested making the option "product", but I don't like it very much: "product" option >>>>>> suggests this is a production-grade feature and it comes with expectations of support, which is >>>>>> interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode >>>>>> available in >>>>>> product builds without the attached notion of support. Users that want fail-fast VM crash can then >>>>>> use that option on "we know what we are doing" basis. >>>>> I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported >>>>> product". If you really want to fail-fast in production then you're looking for a production flag >>>>> not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. >>>> Yup, let's hear it. I don't want to bikeshed this too much :) >>> If I was doing the work on this bug, I would have made it a diagnostic option >>> also. I have a bias towards adding options as diagnostic for two reasons: >>> >>> 1) Easier to get rid of the option if it's a bad idea. >>> 2) You have to use the '-XX:UnlockDiagnosticVMOption' option to use it >>> ??? which makes unintentional use more difficult. >> Thanks. No further comments? David, are you happy? I am pushing this: >> >> diff -r 0811cc0ab16e src/hotspot/share/runtime/globals.hpp >> --- a/src/hotspot/share/runtime/globals.hpp???? Tue Nov 20 16:18:12 2018 +0100 >> +++ b/src/hotspot/share/runtime/globals.hpp???? Tue Nov 20 17:17:35 2018 +0100 >> @@ -496,12 +496,12 @@ >> ????????????????????????????????????????????????????????????????????????????? \ >> ??? product(bool, SafepointTimeout, false,??????????????????????????????????? \ >> ??????????? "Time out and warn or fail after SafepointTimeoutDelay "????????? \ >> ??????????? "milliseconds if failed to reach safepoint")????????????????????? \ >> ????????????????????????????????????????????????????????????????????????????? \ >> -? develop(bool, DieOnSafepointTimeout, false,?????????????????????????????? \ >> -????????? "Die upon failure to reach safepoint (see SafepointTimeout)")???? \ >> +? diagnostic(bool, AbortVMOnSafepointTimeout, false,??????????????????????? \ >> +????????? "Abort upon failure to reach safepoint (see SafepointTimeout)")?? \ >> ????????????????????????????????????????????????????????????????????????????? \ >> ??? /* 50 retries * (5 * current_retry_count) millis = ~6.375 seconds */????? \ >> ??? /* typically, at most a few retries are needed??????????????????? */????? \ >> ??? product(intx, SuspendRetryCount, 50,????????????????????????????????????? \ >> ??????????? "Maximum retry count for an external suspend request")??????????? \ >> diff -r 0811cc0ab16e src/hotspot/share/runtime/safepoint.cpp >> --- a/src/hotspot/share/runtime/safepoint.cpp?? Tue Nov 20 16:18:12 2018 +0100 >> +++ b/src/hotspot/share/runtime/safepoint.cpp?? Tue Nov 20 17:17:35 2018 +0100 >> @@ -976,13 +976,13 @@ >> ??????? } >> ??????? ls.print_cr("# SafepointSynchronize::begin: (End of list)"); >> ????? } >> ??? } >> >> -? // To debug the long safepoint, specify both DieOnSafepointTimeout & >> +? // To debug the long safepoint, specify both AbortVMOnSafepointTimeout & >> ??? // ShowMessageBoxOnError. >> -? if (DieOnSafepointTimeout) { >> +? if (AbortVMOnSafepointTimeout) { >> ????? fatal("Safepoint sync time longer than " INTX_FORMAT "ms detected when executing %s.", >> ??????????? SafepointTimeoutDelay, VMThread::vm_safepoint_description()); >> ??? } >> ? } >> >> >> Thanks, >> -Aleksey >> >> > From vladimir.kozlov at oracle.com Tue Nov 20 20:00:41 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Nov 2018 12:00:41 -0800 Subject: RFR(S) 8213259: [AOT] AOTing java.base fails with "java.lang.AssertionError: no fingerprint for Ljdk/internal/event/Event" In-Reply-To: <69f8f3d4-061f-2f2e-6eb7-172b8686561e@oracle.com> References: <4bdf8604-2980-f35d-df54-32a8cbc79d25@oracle.com> <69f8f3d4-061f-2f2e-6eb7-172b8686561e@oracle.com> Message-ID: <322c487b-478c-7b74-5fa7-e13724ab7619@oracle.com> +1 Make sure AOT tests passed. Thanks, Vladimir On 11/20/18 11:47 AM, Ioi Lam wrote: > Hi Dean, the fixes looks good. Thanks for fixing this bug. > > - Ioi > > > On 11/20/18 11:11 AM, dean.long at oracle.com wrote: >> http://cr.openjdk.java.net/~dlong/8213259/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8213259 >> >> Previously, some fingerprint logic was in ClassFileParser::create_instance_klass, but some was left up to the caller. >> This caused problems if the caller didn't compute a needed fingerprinter, which is what happened in JFR's >> create_new_instance_klass.? The simplifying fix is to move all fingerprint logic into create_instance_klass. >> >> dl > From dean.long at oracle.com Tue Nov 20 20:17:05 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 20 Nov 2018 12:17:05 -0800 Subject: RFR(S) 8213259: [AOT] AOTing java.base fails with "java.lang.AssertionError: no fingerprint for Ljdk/internal/event/Event" In-Reply-To: <322c487b-478c-7b74-5fa7-e13724ab7619@oracle.com> References: <4bdf8604-2980-f35d-df54-32a8cbc79d25@oracle.com> <69f8f3d4-061f-2f2e-6eb7-172b8686561e@oracle.com> <322c487b-478c-7b74-5fa7-e13724ab7619@oracle.com> Message-ID: <88ddd5a3-4b09-a6bc-dc52-750290e39d6e@oracle.com> Thanks Vladimir. dl On 11/20/18 12:00 PM, Vladimir Kozlov wrote: > +1 > > Make sure AOT tests passed. > > Thanks, > Vladimir > > On 11/20/18 11:47 AM, Ioi Lam wrote: >> Hi Dean, the fixes looks good. Thanks for fixing this bug. >> >> - Ioi >> >> >> On 11/20/18 11:11 AM, dean.long at oracle.com wrote: >>> http://cr.openjdk.java.net/~dlong/8213259/webrev/ >>> https://bugs.openjdk.java.net/browse/JDK-8213259 >>> >>> Previously, some fingerprint logic was in >>> ClassFileParser::create_instance_klass, but some was left up to the >>> caller.? This caused problems if the caller didn't compute a needed >>> fingerprinter, which is what happened in JFR's >>> create_new_instance_klass.? The simplifying fix is to move all >>> fingerprint logic into create_instance_klass. >>> >>> dl >> From vladimir.kozlov at oracle.com Tue Nov 20 20:35:27 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Nov 2018 12:35:27 -0800 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: Message-ID: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. Vladimir On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: > Hi, > > When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? > > Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 > > This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. > Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. > compiler/c2/cr6340864/TestByteVect.java > compiler/c2/cr6340864/TestIntVect.java > compiler/c2/cr6340864/TestShortVect.java > compiler/codegen/TestCharVect2.java > > Regards > Yang > > From david.holmes at oracle.com Tue Nov 20 20:47:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 Nov 2018 06:47:00 +1000 Subject: Access to jdk.hotspot.agent classes in Java 9? In-Reply-To: <1a208b25-8145-a005-0c8d-5bbab3a6f867@oracle.com> References: <1a208b25-8145-a005-0c8d-5bbab3a6f867@oracle.com> Message-ID: <546be9aa-118d-ea5e-f709-f4a5a88c3945@oracle.com> Hi David, The sun.jvm.hotspot classes have always been a JDK internal API. What has changed in Java 9 is that the Java Platform Module System now enforces that restriction. To use these classes you must explicitly export them when running (and compiling?) your application e.g. --add-exports=jdk.hotspot.agent/sun.jvm.hotspot=ALL-UNNAMED You may need to do that for each package you need to access. HTH. David Holmes On 21/11/2018 2:48 am, Daniel D. Daugherty wrote: > Adding serviceability-dev at ... since this is the Serviceability Agent... > > Dan > > > On 11/20/18 11:33 AM, David Griffiths wrote: >> Hi, I'm writing some code that makes use of the SA classes under >> sun.jvm.hotspot. This compiles and runs fine under Java 8 provided I >> include sa-jdi.jar but I've just discovered that these are now classified >> as JDK internal API in Java 9. There is no alternative so what should >> I do? >> >> Thanks, >> >> David > From david.holmes at oracle.com Tue Nov 20 21:10:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 21 Nov 2018 07:10:31 +1000 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: <05bdab4a-39bf-6a05-2cb9-53b82e95b08b@redhat.com> References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> <05bdab4a-39bf-6a05-2cb9-53b82e95b08b@redhat.com> Message-ID: <92e3650e-e030-14b5-14dc-a4c2ef9c6303@oracle.com> Okay I guess we're redefining what "diagnostic" means :) Cheers, David On 21/11/2018 2:17 am, Aleksey Shipilev wrote: > On 11/19/18 3:43 PM, Daniel D. Daugherty wrote: >> On 11/19/18 6:23 AM, Aleksey Shipilev wrote: >>> On 11/19/18 6:08 AM, David Holmes wrote: >>>>> David suggested making the option "product", but I don't like it very much: "product" option >>>>> suggests this is a production-grade feature and it comes with expectations of support, which is >>>>> interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode >>>>> available in >>>>> product builds without the attached notion of support. Users that want fail-fast VM crash can then >>>>> use that option on "we know what we are doing" basis. >>>> I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported >>>> product". If you really want to fail-fast in production then you're looking for a production flag >>>> not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. >>> Yup, let's hear it. I don't want to bikeshed this too much :) >> >> If I was doing the work on this bug, I would have made it a diagnostic option >> also. I have a bias towards adding options as diagnostic for two reasons: >> >> 1) Easier to get rid of the option if it's a bad idea. >> 2) You have to use the '-XX:UnlockDiagnosticVMOption' option to use it >> ?? which makes unintentional use more difficult. > > Thanks. No further comments? David, are you happy? I am pushing this: > > diff -r 0811cc0ab16e src/hotspot/share/runtime/globals.hpp > --- a/src/hotspot/share/runtime/globals.hpp Tue Nov 20 16:18:12 2018 +0100 > +++ b/src/hotspot/share/runtime/globals.hpp Tue Nov 20 17:17:35 2018 +0100 > @@ -496,12 +496,12 @@ > \ > product(bool, SafepointTimeout, false, \ > "Time out and warn or fail after SafepointTimeoutDelay " \ > "milliseconds if failed to reach safepoint") \ > \ > - develop(bool, DieOnSafepointTimeout, false, \ > - "Die upon failure to reach safepoint (see SafepointTimeout)") \ > + diagnostic(bool, AbortVMOnSafepointTimeout, false, \ > + "Abort upon failure to reach safepoint (see SafepointTimeout)") \ > \ > /* 50 retries * (5 * current_retry_count) millis = ~6.375 seconds */ \ > /* typically, at most a few retries are needed */ \ > product(intx, SuspendRetryCount, 50, \ > "Maximum retry count for an external suspend request") \ > diff -r 0811cc0ab16e src/hotspot/share/runtime/safepoint.cpp > --- a/src/hotspot/share/runtime/safepoint.cpp Tue Nov 20 16:18:12 2018 +0100 > +++ b/src/hotspot/share/runtime/safepoint.cpp Tue Nov 20 17:17:35 2018 +0100 > @@ -976,13 +976,13 @@ > } > ls.print_cr("# SafepointSynchronize::begin: (End of list)"); > } > } > > - // To debug the long safepoint, specify both DieOnSafepointTimeout & > + // To debug the long safepoint, specify both AbortVMOnSafepointTimeout & > // ShowMessageBoxOnError. > - if (DieOnSafepointTimeout) { > + if (AbortVMOnSafepointTimeout) { > fatal("Safepoint sync time longer than " INTX_FORMAT "ms detected when executing %s.", > SafepointTimeoutDelay, VMThread::vm_safepoint_description()); > } > } > > > Thanks, > -Aleksey > > From shade at redhat.com Tue Nov 20 22:02:05 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 20 Nov 2018 23:02:05 +0100 Subject: RFR (XS) 8213992: Rename and make DieOnSafepointTimeout the diagnostic option In-Reply-To: <92e3650e-e030-14b5-14dc-a4c2ef9c6303@oracle.com> References: <7fe0caaa-53a6-e102-7e72-9df6267a51c1@redhat.com> <9155de3d-af73-4e6e-13ee-68accbd31612@redhat.com> <05bdab4a-39bf-6a05-2cb9-53b82e95b08b@redhat.com> <92e3650e-e030-14b5-14dc-a4c2ef9c6303@oracle.com> Message-ID: <5bd367b7-5dd0-ff3a-d669-8fd61bd8a74a@redhat.com> Pushed. Thanks everyone! -Aleksey On 11/20/18 10:10 PM, David Holmes wrote: > Okay I guess we're redefining what "diagnostic" means :) > > Cheers, > David > > On 21/11/2018 2:17 am, Aleksey Shipilev wrote: >> On 11/19/18 3:43 PM, Daniel D. Daugherty wrote: >>> On 11/19/18 6:23 AM, Aleksey Shipilev wrote: >>>> On 11/19/18 6:08 AM, David Holmes wrote: >>>>>> David suggested making the option "product", but I don't like it very much: "product" option >>>>>> suggests this is a production-grade feature and it comes with expectations of support, which is >>>>>> interesting in itself when product feature crashes the VM. "diagnostic" keeps this mode >>>>>> available in >>>>>> product builds without the attached notion of support. Users that want fail-fast VM crash can >>>>>> then >>>>>> use that option on "we know what we are doing" basis. >>>>> I feel a little uncomfortable that "diagnostic" is being used as a synonym for "unsupported >>>>> product". If you really want to fail-fast in production then you're looking for a production flag >>>>> not a "diagnostic" one IMHO. I won't fight it but I'd like to know what others think. >>>> Yup, let's hear it. I don't want to bikeshed this too much :) >>> >>> If I was doing the work on this bug, I would have made it a diagnostic option >>> also. I have a bias towards adding options as diagnostic for two reasons: >>> >>> 1) Easier to get rid of the option if it's a bad idea. >>> 2) You have to use the '-XX:UnlockDiagnosticVMOption' option to use it >>> ??? which makes unintentional use more difficult. >> >> Thanks. No further comments? David, are you happy? I am pushing this: >> >> diff -r 0811cc0ab16e src/hotspot/share/runtime/globals.hpp >> --- a/src/hotspot/share/runtime/globals.hpp???? Tue Nov 20 16:18:12 2018 +0100 >> +++ b/src/hotspot/share/runtime/globals.hpp???? Tue Nov 20 17:17:35 2018 +0100 >> @@ -496,12 +496,12 @@ >> ????????????????????????????????????????????????????????????????????????????? \ >> ??? product(bool, SafepointTimeout, false,??????????????????????????????????? \ >> ??????????? "Time out and warn or fail after SafepointTimeoutDelay "????????? \ >> ??????????? "milliseconds if failed to reach safepoint")????????????????????? \ >> ????????????????????????????????????????????????????????????????????????????? \ >> -? develop(bool, DieOnSafepointTimeout, false,?????????????????????????????? \ >> -????????? "Die upon failure to reach safepoint (see SafepointTimeout)")???? \ >> +? diagnostic(bool, AbortVMOnSafepointTimeout, false,??????????????????????? \ >> +????????? "Abort upon failure to reach safepoint (see SafepointTimeout)")?? \ >> ????????????????????????????????????????????????????????????????????????????? \ >> ??? /* 50 retries * (5 * current_retry_count) millis = ~6.375 seconds */????? \ >> ??? /* typically, at most a few retries are needed??????????????????? */????? \ >> ??? product(intx, SuspendRetryCount, 50,????????????????????????????????????? \ >> ??????????? "Maximum retry count for an external suspend request")??????????? \ >> diff -r 0811cc0ab16e src/hotspot/share/runtime/safepoint.cpp >> --- a/src/hotspot/share/runtime/safepoint.cpp?? Tue Nov 20 16:18:12 2018 +0100 >> +++ b/src/hotspot/share/runtime/safepoint.cpp?? Tue Nov 20 17:17:35 2018 +0100 >> @@ -976,13 +976,13 @@ >> ??????? } >> ??????? ls.print_cr("# SafepointSynchronize::begin: (End of list)"); >> ????? } >> ??? } >> >> -? // To debug the long safepoint, specify both DieOnSafepointTimeout & >> +? // To debug the long safepoint, specify both AbortVMOnSafepointTimeout & >> ??? // ShowMessageBoxOnError. >> -? if (DieOnSafepointTimeout) { >> +? if (AbortVMOnSafepointTimeout) { >> ????? fatal("Safepoint sync time longer than " INTX_FORMAT "ms detected when executing %s.", >> ??????????? SafepointTimeoutDelay, VMThread::vm_safepoint_description()); >> ??? } >> ? } >> >> >> Thanks, >> -Aleksey >> >> From jcbeyler at google.com Tue Nov 20 22:14:45 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 20 Nov 2018 14:14:45 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> Message-ID: Hi all, Chris thought it made sense to have more eyes on this change than just serviceability as it will modify to tests that are not only serviceability tests so I've moved this to conversation here :) For convenience, I've copy-pasted the initial RFR: Could I have a review for the extension and usage of the ExceptionJniWrapper. This adds lines and filenames to the end of the wrapper JNI methods, adds tracing, and throws an error if need be. I've ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've ported a few of the tests that were already changed for the assignment webrev for JDK-8212884. Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 For illustration, if I force an error to the AP04/ap04t03 test and set the verbosity on, I get something like: >> Calling JNI method FindClass from ap04t003.cpp:343 >> Calling with these parameter(s): java/lang/Threadd Wait for thread to finish << Called JNI method FindClass from ap04t003.cpp:343 Exception in thread "Thread-0" java.lang.NoClassDefFoundError: java/lang/Threadd at nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native Method) at nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) at nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) Caused by: java.lang.ClassNotFoundException: java.lang.Threadd at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ... 3 more FATAL ERROR in native method: JNI method FindClass : internal error from ap04t003.cpp:343 at nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native Method) at nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) at nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) Questions/comments I have about this are: - Do we want to force fatal errors when a JNI call fails in general? Most of these tests do the right thing and test the return of the JNI calls, for example: thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); if (thrClass == NULL) { but now the wrapper actually would do a fatal if the FindClass call would return a nullptr, so we could remove that test altogether. What do you think? - I prefer to leave them as the tests then become closer to what real users would have in their code and is the "recommended" way of doing it - The alternative is to use the NonFatalError I added which then just prints out that something went wrong, letting the test continue. Question will be what should be the default? The fatal or the non-fatal error handling? On a different subject: - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI wrapper handles the tracing and the verify in almost the same way; only difference I can really tell is that the complain method from NSK has a max complain before stopping to "complain"; I have not added that part of the code in this webrev Once we decide on these, I can continue on the files from JDK-8212884 and then do both the assignment in an if extraction followed-by this type of webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can be deprecated as well as we go forward. Thanks! Jc On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer wrote: > On 11/19/18 10:07 AM, JC Beyler wrote: > > Hi all, > > @David/Chris: should I then push this RFR to the hotspot mailing or the > runtime one? For what it's worth, a lot of the tests under the vmTestbase > are jvmti so the review also affects serviceability; it just turns out I > started with the GC originally and then hit some other tests I had touched > via the assignment extraction. > > I think hotspot would be best. > > Chris > > > @Serguei: Done for the method renaming, for the indent, are you talking > about going from the 8-indent to 4-indent? If so, would it not just be > better to do a new JBS bug and do the whole files in one go? I ask because > otherwise, it will look a bit weird to have parts of the file as 8-indent > and others 4-indent? > > Thanks for looking at it! > Jc > > On Mon, Nov 19, 2018 at 1:25 AM serguei.spitsyn at oracle.com < > serguei.spitsyn at oracle.com> wrote: > >> Hi Jc, >> >> We have to start this review anyway. :) >> It looks good to me in general. >> Thank you for your consistency in this refactoring! >> >> Some minor comments. >> >> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >> >> +static const char* remove_folders(const char* fullname) { >> >> I'd suggest to rename the function name to something traditional like get_basename. >> Otherwise, it sounds like this function has to really remove folders. :) >> >> >> Also, all *Locker.cpp have wrong indent in the bodies of if and while statements. >> Could this be fixed with the refactoring? >> >> I did not look on how this impacts the tests other than serviceability. >> >> Thanks, >> Serguei >> >> >> On 11/16/18 19:43, JC Beyler wrote: >> >> Hi all, >> >> Anybody motivated to review this? :) >> Jc >> >> On Wed, Nov 7, 2018 at 9:53 PM JC Beyler wrote: >> >>> Hi all, >>> >>> Could I have a review for the extension and usage of the >>> ExceptionJniWrapper. This adds lines and filenames to the end of the >>> wrapper JNI methods, adds tracing, and throws an error if need be. I've >>> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >>> ported a few of the tests that were already changed for the assignment >>> webrev for JDK-8212884. >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> >>> For illustration, if I force an error to the AP04/ap04t03 test and set >>> the verbosity on, I get something like: >>> >>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>> >> Calling with these parameter(s): >>> java/lang/Threadd >>> Wait for thread to finish >>> << Called JNI method FindClass from ap04t003.cpp:343 >>> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >>> java/lang/Threadd >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> Method) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >>> at >>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>> at >>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>> at >>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>> ... 3 more >>> FATAL ERROR in native method: JNI method FindClass : internal error from >>> ap04t003.cpp:343 >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> Method) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> >>> Questions/comments I have about this are: >>> - Do we want to force fatal errors when a JNI call fails in general? >>> Most of these tests do the right thing and test the return of the JNI >>> calls, for example: >>> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >>> if (thrClass == NULL) { >>> >>> but now the wrapper actually would do a fatal if the FindClass call >>> would return a nullptr, so we could remove that test altogether. What do >>> you think? >>> - I prefer to leave them as the tests then become closer to what >>> real users would have in their code and is the "recommended" way of doing it >>> >>> - The alternative is to use the NonFatalError I added which then just >>> prints out that something went wrong, letting the test continue. Question >>> will be what should be the default? The fatal or the non-fatal error >>> handling? >>> >>> On a different subject: >>> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >>> wrapper handles the tracing and the verify in almost the same way; only >>> difference I can really tell is that the complain method from NSK has a max >>> complain before stopping to "complain"; I have not added that part of the >>> code in this webrev >>> >>> Once we decide on these, I can continue on the files from JDK-8212884 >>> and then do both the assignment in an if extraction followed-by this type >>> of webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can >>> be deprecated as well as we go forward. >>> >>> Thank you for the reviews/comments :) >>> Jc >>> >> >> >> -- >> >> Thanks, >> Jc >> >> >> > > -- > > Thanks, > Jc > > > -- Thanks, Jc From vladimir.kozlov at oracle.com Wed Nov 21 01:04:56 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Nov 2018 17:04:56 -0800 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <5BF431EE.5000609@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> <5BF431EE.5000609@oracle.com> Message-ID: <8025dfcc-aa78-26c4-8cfd-395c388efd8c@oracle.com> I also thought about combining repeative checks is_alive() && !is_unloading() - to match new enum: bool is_alive_and_not_unloading() { return is_alive() && !is_unloading(); } But then I see only 5 cases and separate checks seems better for me. One question is where you clean up 'unloading' state when nmethod state change to 'unloaded'? Is it possible that nmethod to have both state at the same time? Thanks, Vladimir On 11/20/18 8:10 AM, Erik ?sterlund wrote: > Hi Robbin, > > Thanks for the review. I think I will go without is_dying() now because I couldn't figure out how to put it in, in an > intuitive way. Mostly because most of the code is checking for is_alive() && !is_unloading() compiled methods. In that > state they are not is_dying(), according to your definition, but checking for !is_dying() doesn't imply that it is > alive. So I think I will stick with being more explicit for now. > > Thanks, > /Erik > > On 2018-11-20 10:57, Robbin Ehn wrote: >> Looks good! >> >> I gave a suggestion IRL about an is_dying method, which covers the is_alive >> and is_unloading query. If choose to take it or not I'll leave it up to you. >> >> Thanks, Robbin >> >> On 11/19/18 1:31 PM, Erik ?sterlund wrote: >>> Hi Coleen, >>> >>> Thanks for having a look at this. >>> >>> New full webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>> >>> Incremental: >>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>> >>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>> >>>> How about instead of two bool arguments that are hard to read and error prone and need comments, like: >>>> >>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* only_not_unloading */); >>>> >>>> >>>> enum FilterAlive { all_blobs, only_alive }; >>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>> >>>> Then it can be something like: >>>> >>>> ?? CompiledMethodIterator iter(only_alive, all_blobs); >>>> >>>> Don't know if you can repeate all_blobs like that. >>> >>> You can't really have all_blobs in both. I tried out a variant with a single enum though, hope you like it. >>> The observation was that we only need 3 different variations: all_blobs, only_alive or only_alive_and_not_unloading. >>> So I made an enum out of that. >>> In fact, these three modes are the only ones I support right now, so they are also the only valid options, which made >>> the single enum look even more reasonable. >>> >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>> >>>> I'm not sure if metadata_do() should filter out unloading. It's looking for "old" methods to stop from deallocating >>>> memory for them (but it's in a safepoint).? If you're unloading, the IC caches are already cleaned out? >>> >>> IC caches to unloading nmethods are not necessarily cleaned out. The concurrent GC threads will clean the IC caches >>> concurrently, but a JavaThread can call the IC cache before the GC has gotten around to cleaning the IC cache. But >>> any such call into an is_unloading() nmethod will be trapped by nmethod entry barriers, which will call the IC miss >>> handler, which sorts things out lazily. So you can view this as the IC caches to is_unloading() nmethods being >>> logically cleared, but not necessarily physically cleared yet. The effect is the same - the is_unloading() nmethods >>> may not be entered from anywhere, including IC caches. >>> >>> So if you were to not filter is_unloading(), you would get a whole bunch of nmethods that are not really reachable by >>> the application, i.e. they are not on any stacks, and may not be entered by the application. Therefore, I think it's >>> perfectly fine to filter them out here, unless I missed something. And this makes the behaviour closer to as-if the >>> nmethods were indeed unloaded in the safepoint. >>> >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>> >>>> 1108 // The release is only needed for compile-time ordering, as accesses >>>> 1109 // into the nmethod after the store is not safe, due to the sweeper >>>> 1110 // being allowed to free it then, in the case of concurrent nmethod >>>> 1111 // unloading. Therefore, there is no need for acquire on the loader side. >>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>> >>>> I tried to make sense out of this first sentence, but couldn't really.? After the store to unloaded, can the sweeper >>>> free the nmethod?? maybe remove "then, " and it would make more sense? >>> >>> Right, after the store is observed, the sweeper thread may concurrently observe the nmethod to be unloaded, and then >>> it may immediately flip it over to zombie. And I don't want any concurrent make_unloaded code in that nmethod to >>> still be racing after the sweeper turns it into zombie; a very unnecessary race to deal with. >>> >>> I updated the comment - hopefully it reads better now. >>> >>> Thanks, >>> /Erik >>> >>>> Thanks, >>>> Coleen >>>> >>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>> ..put in bug number in subject to make mail filters happy. >>>>> >>>>> /Erik >>>>> >>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> All current GCs perform code cache unloading in safepoints. Therefore, nmethods that are alive but is_unloading() >>>>>> are never observed outside of safepoints. With concurrent class unloading, nmethods that are alive but >>>>>> is_unloading() will become observable outside of safepoints. This must be handled appropriately. >>>>>> >>>>>> In this patch I changed the nmethod/compiled method iterators to accept parameters describing whether they should >>>>>> filter out not is_alive() or is_unloading() methods. Since there is no obvious default (all combinations are used >>>>>> depending on call site), you have to explicitly set what iteration mode you want. >>>>>> >>>>>> Other than that, I make sure that the sweeper stays away from is_unloading() nmethods that are not yet >>>>>> is_unloaded(). To make the interactions between the sweeper and concurrent GC threads safe, I had to move down the >>>>>> store that sets the state to unloaded, and use a release_store there, to make sure no accesses float below it at >>>>>> compile-time. Once that store is observed, nmethods may be deleted. >>>>>> >>>>>> In the IC miss handler, I also need to lazily clean stale IC caches due to calling is_unloading nmethods using >>>>>> nmethod entry barriers. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>> >>> > From vladimir.kozlov at oracle.com Wed Nov 21 01:08:17 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 20 Nov 2018 17:08:17 -0800 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: <9db0d59e-e8ef-1a60-f62a-4569177de23c@redhat.com> References: <5BF2D678.3030208@oracle.com> <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> <5BF2E012.5010701@oracle.com> <9db0d59e-e8ef-1a60-f62a-4569177de23c@redhat.com> Message-ID: +1 thanks, Vladimir On 11/19/18 8:34 AM, Andrew Dinn wrote: > On 19/11/2018 16:08, Erik ?sterlund wrote: >> Thanks for the review. I liked your idea of having the casting done by >> the accessors, like we did for GC data on Thread. >> So here is a new webrev with that approach: >> http://cr.openjdk.java.net/~eosterlund/8214056/webrev.01/ > Yes, that's a lot nicer. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From Yang.Zhang at arm.com Wed Nov 21 06:28:52 2018 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Wed, 21 Nov 2018 06:28:52 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> Message-ID: >I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. >In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). >aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. Matcher::vector_width_in_bytes() limits vector size to min(16, MaxVectorSize). Matcher::vector_shift_count_ideal_reg() assert(MaxVectorSize >= size, "Length isn't supported"); When MaxVectorSize is 8, size is 8. When MaxVectorSize is greater than or equal 16, size is 16. So (MaxVectorSize >= size) is always true. Test cases of cr6340864 are passed with or without this new assert. Do I need to remove this new assert? Regards Yang On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: > Hi, > > When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? > > Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 > > This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. > Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. > compiler/c2/cr6340864/TestByteVect.java > compiler/c2/cr6340864/TestIntVect.java > compiler/c2/cr6340864/TestShortVect.java > compiler/codegen/TestCharVect2.java > > Regards > Yang > > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. From gnu.andrew at redhat.com Wed Nov 21 06:45:21 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 21 Nov 2018 06:45:21 +0000 Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8140482 Original changeset: https://hg.openjdk.java.net/jdk-updates/jdk9u/hotspot/rev/cd86b5699825 Webrev: https://cr.openjdk.java.net/~andrew/openjdk8/8140482/webrev.01/ The patch largely applies as is, with some adjustment for context and the dropping of the changes to src/cpu/x86/vm/stubRoutines_x86.cpp, src/share/vm/runtime/task.cpp and src/os/windows/vm/attachListener_windows.cpp which don't exist in 8u. A clean backport of 7127191 is included, which allows the changes to agent/src/os/linux/libproc_impl.c to apply as-is. Applying the change to 8u improves the code quality there and aids in backporting other changes, such as 8210836 [0]. Ok for 8u? [0] https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-November/025991.html Thanks, -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From per.liden at oracle.com Wed Nov 21 09:11:23 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Nov 2018 10:11:23 +0100 Subject: RFR: 8214068: ZGC crashes with vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java Message-ID: This patch adds a missing load barrier to the JVMTI heap walker. In VM_HeapWalkOperation::collect_simple_roots() the SimpleRootsClosure closure should not do raw loads of oops. Instead it should be loading them using NativeAccess::oop_load(). This problem only affects ZGC, since it's (so far) the only collector that scans JNIHandles concurrently. The problem was easily reproducible when running vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java. However, for some reason this isn't reproducing in our CI pipeline, likely because of luck with timing. For example, this is harder to reproduce in a scenario where your using a smaller heap and hence have GC cycles happen more frequently. Bug: https://bugs.openjdk.java.net/browse/JDK-8214068 Webrev: http://cr.openjdk.java.net/~pliden/8214068/webrev.0 /Per From erik.osterlund at oracle.com Wed Nov 21 10:08:04 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 11:08:04 +0100 Subject: RFR: 8214068: ZGC crashes with vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java In-Reply-To: References: Message-ID: <5BF52E84.1040206@oracle.com> Hi Per, Looks good. Thanks, /Erik On 2018-11-21 10:11, Per Liden wrote: > This patch adds a missing load barrier to the JVMTI heap walker. In > VM_HeapWalkOperation::collect_simple_roots() the SimpleRootsClosure > closure should not do raw loads of oops. Instead it should be loading > them using NativeAccess::oop_load(). > > This problem only affects ZGC, since it's (so far) the only collector > that scans JNIHandles concurrently. > > The problem was easily reproducible when running > vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java. > However, for some reason this isn't reproducing in our CI pipeline, > likely because of luck with timing. For example, this is harder to > reproduce in a scenario where your using a smaller heap and hence have > GC cycles happen more frequently. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214068 > Webrev: http://cr.openjdk.java.net/~pliden/8214068/webrev.0 > > /Per From per.liden at oracle.com Wed Nov 21 10:10:39 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Nov 2018 11:10:39 +0100 Subject: RFR: 8214068: ZGC crashes with vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java In-Reply-To: <5BF52E84.1040206@oracle.com> References: <5BF52E84.1040206@oracle.com> Message-ID: Thanks for reviewing Erik! /Per On 11/21/18 11:08 AM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 2018-11-21 10:11, Per Liden wrote: >> This patch adds a missing load barrier to the JVMTI heap walker. In >> VM_HeapWalkOperation::collect_simple_roots() the SimpleRootsClosure >> closure should not do raw loads of oops. Instead it should be loading >> them using NativeAccess::oop_load(). >> >> This problem only affects ZGC, since it's (so far) the only collector >> that scans JNIHandles concurrently. >> >> The problem was easily reproducible when running >> vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java. >> However, for some reason this isn't reproducing in our CI pipeline, >> likely because of luck with timing. For example, this is harder to >> reproduce in a scenario where your using a smaller heap and hence have >> GC cycles happen more frequently. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214068 >> Webrev: http://cr.openjdk.java.net/~pliden/8214068/webrev.0 >> >> /Per > From david.griffiths at gmail.com Wed Nov 21 10:22:40 2018 From: david.griffiths at gmail.com (David Griffiths) Date: Wed, 21 Nov 2018 10:22:40 +0000 Subject: Access to jdk.hotspot.agent classes in Java 9? In-Reply-To: <546be9aa-118d-ea5e-f709-f4a5a88c3945@oracle.com> References: <1a208b25-8145-a005-0c8d-5bbab3a6f867@oracle.com> <546be9aa-118d-ea5e-f709-f4a5a88c3945@oracle.com> Message-ID: Ok, thanks, that worked and I can now use them again. Cheers, David On Tue, 20 Nov 2018 at 20:47, David Holmes wrote: > Hi David, > > The sun.jvm.hotspot classes have always been a JDK internal API. What > has changed in Java 9 is that the Java Platform Module System now > enforces that restriction. To use these classes you must explicitly > export them when running (and compiling?) your application e.g. > > --add-exports=jdk.hotspot.agent/sun.jvm.hotspot=ALL-UNNAMED > > You may need to do that for each package you need to access. > > HTH. > > David Holmes > > On 21/11/2018 2:48 am, Daniel D. Daugherty wrote: > > Adding serviceability-dev at ... since this is the Serviceability Agent... > > > > Dan > > > > > > On 11/20/18 11:33 AM, David Griffiths wrote: > >> Hi, I'm writing some code that makes use of the SA classes under > >> sun.jvm.hotspot. This compiles and runs fine under Java 8 provided I > >> include sa-jdi.jar but I've just discovered that these are now > classified > >> as JDK internal API in Java 9. There is no alternative so what should > >> I do? > >> > >> Thanks, > >> > >> David > > > From Nick.Gasson at arm.com Wed Nov 21 10:29:26 2018 From: Nick.Gasson at arm.com (Nick Gasson) Date: Wed, 21 Nov 2018 10:29:26 +0000 Subject: RFR: 8213845: ARM32: Interpreter doesn't call result handler after native calls In-Reply-To: References: Message-ID: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213845 > > Webrev: http://cr.openjdk.java.net/~njian/8213845/webrev.0/ > > That looks right, thanks. > Thanks Andrew! Could you help to commit this patch for me if it doesn't need other reviews? Nick From robbin.ehn at oracle.com Wed Nov 21 11:06:23 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 21 Nov 2018 12:06:23 +0100 Subject: RFR: 8214068: ZGC crashes with vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java In-Reply-To: <5BF52E84.1040206@oracle.com> References: <5BF52E84.1040206@oracle.com> Message-ID: <6e496ccb-89e2-92ae-d342-6dfa5897fd35@oracle.com> On 2018-11-21 11:08, Erik ?sterlund wrote: > Hi Per, > > Looks good. Ship it! /Robbin > > Thanks, > /Erik > > On 2018-11-21 10:11, Per Liden wrote: >> This patch adds a missing load barrier to the JVMTI heap walker. In >> VM_HeapWalkOperation::collect_simple_roots() the SimpleRootsClosure closure >> should not do raw loads of oops. Instead it should be loading them using >> NativeAccess::oop_load(). >> >> This problem only affects ZGC, since it's (so far) the only collector that >> scans JNIHandles concurrently. >> >> The problem was easily reproducible when running >> vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java. >> However, for some reason this isn't reproducing in our CI pipeline, likely >> because of luck with timing. For example, this is harder to reproduce in a >> scenario where your using a smaller heap and hence have GC cycles happen more >> frequently. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214068 >> Webrev: http://cr.openjdk.java.net/~pliden/8214068/webrev.0 >> >> /Per > From per.liden at oracle.com Wed Nov 21 13:21:46 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Nov 2018 14:21:46 +0100 Subject: RFR: 8214068: ZGC crashes with vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java In-Reply-To: <6e496ccb-89e2-92ae-d342-6dfa5897fd35@oracle.com> References: <5BF52E84.1040206@oracle.com> <6e496ccb-89e2-92ae-d342-6dfa5897fd35@oracle.com> Message-ID: <26a38c48-d60d-be4a-20fc-aa2e1bd46b57@oracle.com> Thanks for reviewing, Robbin! /Per On 11/21/18 12:06 PM, Robbin Ehn wrote: > On 2018-11-21 11:08, Erik ?sterlund wrote: >> Hi Per, >> >> Looks good. > > Ship it! > > /Robbin > >> >> Thanks, >> /Erik >> >> On 2018-11-21 10:11, Per Liden wrote: >>> This patch adds a missing load barrier to the JVMTI heap walker. In >>> VM_HeapWalkOperation::collect_simple_roots() the SimpleRootsClosure >>> closure should not do raw loads of oops. Instead it should be loading >>> them using NativeAccess::oop_load(). >>> >>> This problem only affects ZGC, since it's (so far) the only collector >>> that scans JNIHandles concurrently. >>> >>> The problem was easily reproducible when running >>> vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java. >>> However, for some reason this isn't reproducing in our CI pipeline, >>> likely because of luck with timing. For example, this is harder to >>> reproduce in a scenario where your using a smaller heap and hence >>> have GC cycles happen more frequently. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8214068 >>> Webrev: http://cr.openjdk.java.net/~pliden/8214068/webrev.0 >>> >>> /Per >> From erik.osterlund at oracle.com Wed Nov 21 13:40:47 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 14:40:47 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <8025dfcc-aa78-26c4-8cfd-395c388efd8c@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> <5BF431EE.5000609@oracle.com> <8025dfcc-aa78-26c4-8cfd-395c388efd8c@oracle.com> Message-ID: <5BF5605F.70708@oracle.com> Hi Vladimir, Thanks for reviewing. On 2018-11-21 02:04, Vladimir Kozlov wrote: > I also thought about combining repeative checks is_alive() && > !is_unloading() - to match new enum: > > bool is_alive_and_not_unloading() { return is_alive() && > !is_unloading(); } > > But then I see only 5 cases and separate checks seems better for me. > > One question is where you clean up 'unloading' state when nmethod > state change to 'unloaded'? Is it possible that nmethod to have both > state at the same time? Yes. The answer to is_alive() monotonically goes from true to false and never back. Similarly, the answer to is_unloading() monotonically goes from false to true and never back. Once is_unloading - always is_unloading(). This is intentional to avoid races where reading is_unloading() again would race and give a different answer. Thanks, /Erik > Thanks, > Vladimir > > On 11/20/18 8:10 AM, Erik ?sterlund wrote: >> Hi Robbin, >> >> Thanks for the review. I think I will go without is_dying() now >> because I couldn't figure out how to put it in, in an intuitive way. >> Mostly because most of the code is checking for is_alive() && >> !is_unloading() compiled methods. In that state they are not >> is_dying(), according to your definition, but checking for >> !is_dying() doesn't imply that it is alive. So I think I will stick >> with being more explicit for now. >> >> Thanks, >> /Erik >> >> On 2018-11-20 10:57, Robbin Ehn wrote: >>> Looks good! >>> >>> I gave a suggestion IRL about an is_dying method, which covers the >>> is_alive >>> and is_unloading query. If choose to take it or not I'll leave it up >>> to you. >>> >>> Thanks, Robbin >>> >>> On 11/19/18 1:31 PM, Erik ?sterlund wrote: >>>> Hi Coleen, >>>> >>>> Thanks for having a look at this. >>>> >>>> New full webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>>> >>>> Incremental: >>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>>> >>>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>>> >>>>> How about instead of two bool arguments that are hard to read and >>>>> error prone and need comments, like: >>>>> >>>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >>>>> only_not_unloading */); >>>>> >>>>> >>>>> enum FilterAlive { all_blobs, only_alive }; >>>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>>> >>>>> Then it can be something like: >>>>> >>>>> CompiledMethodIterator iter(only_alive, all_blobs); >>>>> >>>>> Don't know if you can repeate all_blobs like that. >>>> >>>> You can't really have all_blobs in both. I tried out a variant with >>>> a single enum though, hope you like it. >>>> The observation was that we only need 3 different variations: >>>> all_blobs, only_alive or only_alive_and_not_unloading. So I made an >>>> enum out of that. >>>> In fact, these three modes are the only ones I support right now, >>>> so they are also the only valid options, which made the single enum >>>> look even more reasonable. >>>> >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>>> >>>>> >>>>> I'm not sure if metadata_do() should filter out unloading. It's >>>>> looking for "old" methods to stop from deallocating memory for >>>>> them (but it's in a safepoint). If you're unloading, the IC >>>>> caches are already cleaned out? >>>> >>>> IC caches to unloading nmethods are not necessarily cleaned out. >>>> The concurrent GC threads will clean the IC caches concurrently, >>>> but a JavaThread can call the IC cache before the GC has gotten >>>> around to cleaning the IC cache. But any such call into an >>>> is_unloading() nmethod will be trapped by nmethod entry barriers, >>>> which will call the IC miss handler, which sorts things out lazily. >>>> So you can view this as the IC caches to is_unloading() nmethods >>>> being logically cleared, but not necessarily physically cleared >>>> yet. The effect is the same - the is_unloading() nmethods may not >>>> be entered from anywhere, including IC caches. >>>> >>>> So if you were to not filter is_unloading(), you would get a whole >>>> bunch of nmethods that are not really reachable by the application, >>>> i.e. they are not on any stacks, and may not be entered by the >>>> application. Therefore, I think it's perfectly fine to filter them >>>> out here, unless I missed something. And this makes the behaviour >>>> closer to as-if the nmethods were indeed unloaded in the safepoint. >>>> >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>>> >>>>> >>>>> 1108 // The release is only needed for compile-time ordering, as >>>>> accesses >>>>> 1109 // into the nmethod after the store is not safe, due to the >>>>> sweeper >>>>> 1110 // being allowed to free it then, in the case of concurrent >>>>> nmethod >>>>> 1111 // unloading. Therefore, there is no need for acquire on the >>>>> loader side. >>>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>>> >>>>> I tried to make sense out of this first sentence, but couldn't >>>>> really. After the store to unloaded, can the sweeper free the >>>>> nmethod? maybe remove "then, " and it would make more sense? >>>> >>>> Right, after the store is observed, the sweeper thread may >>>> concurrently observe the nmethod to be unloaded, and then it may >>>> immediately flip it over to zombie. And I don't want any concurrent >>>> make_unloaded code in that nmethod to still be racing after the >>>> sweeper turns it into zombie; a very unnecessary race to deal with. >>>> >>>> I updated the comment - hopefully it reads better now. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>>> ..put in bug number in subject to make mail filters happy. >>>>>> >>>>>> /Erik >>>>>> >>>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>>> Hi, >>>>>>> >>>>>>> All current GCs perform code cache unloading in safepoints. >>>>>>> Therefore, nmethods that are alive but is_unloading() are never >>>>>>> observed outside of safepoints. With concurrent class unloading, >>>>>>> nmethods that are alive but is_unloading() will become >>>>>>> observable outside of safepoints. This must be handled >>>>>>> appropriately. >>>>>>> >>>>>>> In this patch I changed the nmethod/compiled method iterators to >>>>>>> accept parameters describing whether they should filter out not >>>>>>> is_alive() or is_unloading() methods. Since there is no obvious >>>>>>> default (all combinations are used depending on call site), you >>>>>>> have to explicitly set what iteration mode you want. >>>>>>> >>>>>>> Other than that, I make sure that the sweeper stays away from >>>>>>> is_unloading() nmethods that are not yet is_unloaded(). To make >>>>>>> the interactions between the sweeper and concurrent GC threads >>>>>>> safe, I had to move down the store that sets the state to >>>>>>> unloaded, and use a release_store there, to make sure no >>>>>>> accesses float below it at compile-time. Once that store is >>>>>>> observed, nmethods may be deleted. >>>>>>> >>>>>>> In the IC miss handler, I also need to lazily clean stale IC >>>>>>> caches due to calling is_unloading nmethods using nmethod entry >>>>>>> barriers. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>> >>>> >> From thomas.stuefe at gmail.com Wed Nov 21 14:28:38 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 21 Nov 2018 15:28:38 +0100 Subject: Extend NMT to JDK native libraries? Message-ID: Hi all, (yet again not sure if this is serviceablity-dev or not - I start at hs-dev, feel free to move this mail around.) Do we have any plans to extend NMT to cover native JDK libaries too? That would be a really cool feature. -- We at SAP have done a similar thing in the past: We have a monitoring facility in our port which tracks C-heap allocations, non-imaginatively called "malloc statistic". This feature predates NMT somewhat - had we had NMT at that time, we would not have bothered. Our Malloc statistic is less powerful than NMT and implementation-wise completely at odds with it, so I never felt the urge to bring it upstream. However, one thing we did do is we extended its coverage to the JDK native code. This has been quite helpful in the past to find leaks in JDK, see e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 We did this by exposing os::malloc, os::free etc from libjvm.so ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we then either manually replaced calls to raw ::malloc(), ::free() etc with JVM_malloc(), JVM_free(). Or, in places where this was possible, we did this replacement stuff wholesale by employing a header which re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, we also had to add a number of linkage dependencies to the libjvm.so. All this is pretty standard stuff. One detail stood out: malloc headers are evil. In our experience, JDK native code was more difficult to control and "unbalanced malloc/frees" kept creeping in - especially with the wholesale-redefinition technique. Unbalanced mallocs/frees means cases where malloc() is instrumented but ::free() stays raw, or the other way around. Both combinations are catastrophic since os::malloc uses malloc headers. We typically corrupted the C-Heap and crashed, often much later in completely unrelated places. These types of bugs were very hard to spot and hence very expensive. And they can creep in in many ways. One example, there exist a surprising number of system APIs which return results in C-heap and require the user to free that, which of course must happen with raw ::free(), not os::free(). We fixed this by not using malloc headers. That means a pointer returned by os::malloc() is compatible with raw ::free() and vice versa. The only bad thing happening would be our statistic numbers being slightly off. Instead of malloc headers we use a hand-groomed hash table to track the malloced memory. It is actually quite fast, fast enough that this malloc statistic feature is on-by-default in our port. -- Of course, if we extend NMT to JDK native code we also would want to extend it to mmap() etc - we never did this with our statistic, since it only tracked malloc. What do you think? Did anyone else play with similar ideas? Would it be worth the effort? Cheers, Thomas From erik.osterlund at oracle.com Wed Nov 21 14:26:04 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 15:26:04 +0100 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> Message-ID: <5BF56AFC.70607@oracle.com> Hi Vladimir, Thank you for the explanation. I inserted some debugging code to be able to draw better conclusions. I put in a bool that tags CompiledMethods that are visited during code cache unloading. When I crash, I find the AOTCompiledMethod in the following state: * The CodeCache::_unloading_cycle is 2, meaning there has been at least one unloading that occurred since the VM started. * My new boolean saying if it was found during code cache walking said false; it has never been observed by GC code cache unloading. * The _is_unloading_state of the AOT method is 2, meaning it was from CodeCache::_unloading_cycle number 1. So there has been at least one code cache unloading cycle since the AOT method was created. * It is in the CompiledMethod state in_use. * The corresponding CodeToAMethod entry has _state equal to in_use, and the _aot pointer points right back to the AOT method as expected. So the conclusions I can draw from this is that we are looking at an AOTCompiledMethod that is in_use, alive, and published, but was created at least one unloading cycle earlier, yet was *not* found by the evident code cache unloading that has occurred between creating the AOT method and the crash. In fact it has not been found in any code cache unloading event at all. So I am certain about the problem being that the AOT method is not found during our code cache walk, and that the solution to that problem is to move the epoch business to nmethod instead. However, it would be desirable to know exactly why the AOT method isn't showing up in the code cache walk, but I'm afraid I can't see right now exactly why it has been missed out, only that it definitely has been missed out. :/ /Erik On 2018-11-20 19:08, Vladimir Kozlov wrote: > To clarify. > > 'invalid' state is set only for AOT methods which never was registers > (their state is 'not_set' and corresponding class was not initialized > yet). Such methods are never visible to iterators and IC caches. > > If aot method is registered its state become 'in_use' and in such case > it is treated and processed as normal nmethod. And AOTCompiledMethod > structure is created only for such case. > > Vladimir > > On 11/20/18 3:12 AM, Erik ?sterlund wrote: >> Hi Dean, >> >> Yeah I think I misunderstood what I observed. So what I know is that >> in the code cache unloading, I'm not getting all is_alive() AOT >> methods into my iterator, which messes up the assumptions made by the >> epoch based scheme for AOT methods. >> >> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, >> int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT >> heap, making it no longer observable from the iterators. Then it >> calls the VM_Deoptimize vm operation after. Throughout all this, the >> AOTCompiledMethod is alive(), yet when the iterators ask for all >> is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC >> caches may still reference these methods and check if it >> is_unloading, and then we blow up. There may possibly be multiple >> ways for is_alive() AOTCompiledMethods to not be visible from >> iterators yet be visible through IC caches using the "invalid" state >> in the . >> >> Anyway, the fix is the same: stop doing the epoch state thingey for >> is_unloading() on AOTCompiledMethod where it isn't needed, and >> doesn't seem to play well with the rather different life cycle it >> has, and just return false instead. >> >> Thanks, >> /Erik >> >> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>> Hi Erik, >>> >>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>> ...except it looks like for AOTCompiledMethods when running with >>>> tiered compilation, may first be is_alive(), then become >>>> !is_alive() for a while, and then get resurrected to is_alive() >>>> using make_entrant(). >>> >>> this doesn't sounds quite right. AOTCompiledMethods aren't allowed >>> to transition to zombie (!alive), only not_used or not_entrant, >>> which should still have is_alive() returning true. Maybe some code >>> is using is_not_entrant() instead of !is_alive()? >>> >>> dl >> From erik.osterlund at oracle.com Wed Nov 21 14:28:26 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 15:28:26 +0100 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> Message-ID: <5BF56B8A.8060100@oracle.com> Hi Dean, On 2018-11-20 19:21, dean.long at oracle.com wrote: > OK. I agree it makes sense to move the unloading from CompiledMethod > to nmethod. I was just curious what problem you were running into > with AOT, > because the iterators look OK to me. I am also curious! > You probably ran into trouble here: > > // Clean inline caches pointing to both zombie and not_entrant methods > *if* (clean_all > > || !nm > ->is_in_use > () > || nm > ->is_unloading > () > || (nm > ->method > ()->code > () != > nm )) { > because !is_in_use() would be false for the not_used state. No, if it is not_used, then !is_in_use() will return true, and therefore nm->is_unloading() will not be called. In fact, upon crashing here, I know that th AOT method is in fact in_use. So now I am also trying to figure out how the iterators manage to miss the AOT method. I just know with pretty high certainty that they do. Thanks, /Erik > dl > > On 11/20/18 3:12 AM, Erik ?sterlund wrote: >> Hi Dean, >> >> Yeah I think I misunderstood what I observed. So what I know is that >> in the code cache unloading, I'm not getting all is_alive() AOT >> methods into my iterator, which messes up the assumptions made by the >> epoch based scheme for AOT methods. >> >> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, >> int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT >> heap, making it no longer observable from the iterators. Then it >> calls the VM_Deoptimize vm operation after. Throughout all this, the >> AOTCompiledMethod is alive(), yet when the iterators ask for all >> is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC >> caches may still reference these methods and check if it >> is_unloading, and then we blow up. There may possibly be multiple >> ways for is_alive() AOTCompiledMethods to not be visible from >> iterators yet be visible through IC caches using the "invalid" state >> in the . >> >> Anyway, the fix is the same: stop doing the epoch state thingey for >> is_unloading() on AOTCompiledMethod where it isn't needed, and >> doesn't seem to play well with the rather different life cycle it >> has, and just return false instead. >> >> Thanks, >> /Erik >> >> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>> Hi Erik, >>> >>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>> ...except it looks like for AOTCompiledMethods when running with >>>> tiered compilation, may first be is_alive(), then become >>>> !is_alive() for a while, and then get resurrected to is_alive() >>>> using make_entrant(). >>> >>> this doesn't sounds quite right. AOTCompiledMethods aren't allowed >>> to transition to zombie (!alive), only not_used or not_entrant, >>> which should still have is_alive() returning true. Maybe some code >>> is using is_not_entrant() instead of !is_alive()? >>> >>> dl >> > From stefan.johansson at oracle.com Wed Nov 21 14:43:59 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 21 Nov 2018 15:43:59 +0100 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails Message-ID: Hi, Please review this fix to avoid leaving heap metadata in an inconsistent state when CDS archive mapping fails. Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ Summary ------- Since JDK 12b15 a default CDS archive is shipped with the JDK. If this archive for some reason fails to be mapped into the heap, the cleanup leaves the heap metadata in a inconsistent state. Instead of clearing the range in the map storing which regions are archive, it once again marks the range as archive. This information is used by the G1 Full collections and because a regions is marked as archive even though it is not, we skip marking through this region. This causes the heap to be corrupt and we can crash in many different ways. Testing ------- Verified that the new code clears the map correctly and re-ran the benchmark previously failing many times. Tier 1-3 of normal functional testing also run through m5 without problems. Thanks, Stefan From zgu at redhat.com Wed Nov 21 14:54:37 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 21 Nov 2018 09:54:37 -0500 Subject: Extend NMT to JDK native libraries? In-Reply-To: References: Message-ID: FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) https://bugs.openjdk.java.net/browse/JDK-6995789 -Zhengyu On 11/21/18 9:28 AM, Thomas St?fe wrote: > Hi all, > > (yet again not sure if this is serviceablity-dev or not - I start at > hs-dev, feel free to move this mail around.) > > Do we have any plans to extend NMT to cover native JDK libaries too? > That would be a really cool feature. > > -- > > We at SAP have done a similar thing in the past: > > We have a monitoring facility in our port which tracks C-heap > allocations, non-imaginatively called "malloc statistic". This feature > predates NMT somewhat - had we had NMT at that time, we would not have > bothered. Our Malloc statistic is less powerful than NMT and > implementation-wise completely at odds with it, so I never felt the > urge to bring it upstream. However, one thing we did do is we extended > its coverage to the JDK native code. > > This has been quite helpful in the past to find leaks in JDK, see > e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 > > We did this by exposing os::malloc, os::free etc from libjvm.so > ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we > then either manually replaced calls to raw ::malloc(), ::free() etc > with JVM_malloc(), JVM_free(). Or, in places where this was possible, > we did this replacement stuff wholesale by employing a header which > re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, > we also had to add a number of linkage dependencies to the libjvm.so. > > All this is pretty standard stuff. > > One detail stood out: malloc headers are evil. In our experience, JDK > native code was more difficult to control and "unbalanced > malloc/frees" kept creeping in - especially with the > wholesale-redefinition technique. Unbalanced mallocs/frees means cases > where malloc() is instrumented but ::free() stays raw, or the other > way around. Both combinations are catastrophic since os::malloc uses > malloc headers. We typically corrupted the C-Heap and crashed, often > much later in completely unrelated places. > > These types of bugs were very hard to spot and hence very expensive. > And they can creep in in many ways. One example, there exist a > surprising number of system APIs which return results in C-heap and > require the user to free that, which of course must happen with raw > ::free(), not os::free(). > > We fixed this by not using malloc headers. That means a pointer > returned by os::malloc() is compatible with raw ::free() and vice > versa. The only bad thing happening would be our statistic numbers > being slightly off. > > Instead of malloc headers we use a hand-groomed hash table to track > the malloced memory. It is actually quite fast, fast enough that this > malloc statistic feature is on-by-default in our port. > > -- > > Of course, if we extend NMT to JDK native code we also would want to > extend it to mmap() etc - we never did this with our statistic, since > it only tracked malloc. > > What do you think? Did anyone else play with similar ideas? Would it > be worth the effort? > > Cheers, Thomas > From daniel.daugherty at oracle.com Wed Nov 21 14:58:08 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 21 Nov 2018 09:58:08 -0500 Subject: RFR: 8214068: ZGC crashes with vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java In-Reply-To: References: Message-ID: On 11/21/18 4:11 AM, Per Liden wrote: > This patch adds a missing load barrier to the JVMTI heap walker. In > VM_HeapWalkOperation::collect_simple_roots() the SimpleRootsClosure > closure should not do raw loads of oops. Instead it should be loading > them using NativeAccess::oop_load(). > > This problem only affects ZGC, since it's (so far) the only collector > that scans JNIHandles concurrently. > > The problem was easily reproducible when running > vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java. > However, for some reason this isn't reproducing in our CI pipeline, > likely because of luck with timing. For example, this is harder to > reproduce in a scenario where your using a smaller heap and hence have > GC cycles happen more frequently. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8214068 > Webrev: http://cr.openjdk.java.net/~pliden/8214068/webrev.0 src/hotspot/share/prims/jvmtiTagMap.cpp ??? No comments. Nice catch! Thumbs up. Since this is in the JVM/TI heap walker, I wonder if there are other sightings in JBS that might be tracked back to this. Looks like Zhengyu put a nice stack trace in the bug report so dup matching should be easy (if there are any). Dan > > /Per From thomas.stuefe at gmail.com Wed Nov 21 14:59:26 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 21 Nov 2018 15:59:26 +0100 Subject: Extend NMT to JDK native libraries? In-Reply-To: References: Message-ID: Thanks Zhengyu. Hm, closed as wont fix I see. But the solution outlined there differs from our proposal. (I think even if we were to instrument parts of the JDK - e.g. just NIO - this would already be very helpful. In parts we do this already for mtOther.). On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu wrote: > > FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) > https://bugs.openjdk.java.net/browse/JDK-6995789 > > -Zhengyu > > On 11/21/18 9:28 AM, Thomas St?fe wrote: > > Hi all, > > > > (yet again not sure if this is serviceablity-dev or not - I start at > > hs-dev, feel free to move this mail around.) > > > > Do we have any plans to extend NMT to cover native JDK libaries too? > > That would be a really cool feature. > > > > -- > > > > We at SAP have done a similar thing in the past: > > > > We have a monitoring facility in our port which tracks C-heap > > allocations, non-imaginatively called "malloc statistic". This feature > > predates NMT somewhat - had we had NMT at that time, we would not have > > bothered. Our Malloc statistic is less powerful than NMT and > > implementation-wise completely at odds with it, so I never felt the > > urge to bring it upstream. However, one thing we did do is we extended > > its coverage to the JDK native code. > > > > This has been quite helpful in the past to find leaks in JDK, see > > e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 > > > > We did this by exposing os::malloc, os::free etc from libjvm.so > > ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we > > then either manually replaced calls to raw ::malloc(), ::free() etc > > with JVM_malloc(), JVM_free(). Or, in places where this was possible, > > we did this replacement stuff wholesale by employing a header which > > re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, > > we also had to add a number of linkage dependencies to the libjvm.so. > > > > All this is pretty standard stuff. > > > > One detail stood out: malloc headers are evil. In our experience, JDK > > native code was more difficult to control and "unbalanced > > malloc/frees" kept creeping in - especially with the > > wholesale-redefinition technique. Unbalanced mallocs/frees means cases > > where malloc() is instrumented but ::free() stays raw, or the other > > way around. Both combinations are catastrophic since os::malloc uses > > malloc headers. We typically corrupted the C-Heap and crashed, often > > much later in completely unrelated places. > > > > These types of bugs were very hard to spot and hence very expensive. > > And they can creep in in many ways. One example, there exist a > > surprising number of system APIs which return results in C-heap and > > require the user to free that, which of course must happen with raw > > ::free(), not os::free(). > > > > We fixed this by not using malloc headers. That means a pointer > > returned by os::malloc() is compatible with raw ::free() and vice > > versa. The only bad thing happening would be our statistic numbers > > being slightly off. > > > > Instead of malloc headers we use a hand-groomed hash table to track > > the malloced memory. It is actually quite fast, fast enough that this > > malloc statistic feature is on-by-default in our port. > > > > -- > > > > Of course, if we extend NMT to JDK native code we also would want to > > extend it to mmap() etc - we never did this with our statistic, since > > it only tracked malloc. > > > > What do you think? Did anyone else play with similar ideas? Would it > > be worth the effort? > > > > Cheers, Thomas > > From zgu at redhat.com Wed Nov 21 15:31:22 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 21 Nov 2018 10:31:22 -0500 Subject: Extend NMT to JDK native libraries? In-Reply-To: References: Message-ID: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> Hi Thomas, On 11/21/18 9:59 AM, Thomas St?fe wrote: > Thanks Zhengyu. > > Hm, closed as wont fix I see. But the solution outlined there differs > from our proposal. It was just a placeholder. In fact, we did discussed the approach that you purposed. IIRC, we concluded that initial effort was just too great. Global replacement of ::malloc()/::free()/::realloc() can be a good starting point, at the end, we do want library code to follow hotspot's convention - provides correct memory tags. A single mtOther for all library code does not seem very helpful. I do believe NMT can be very helpful for library development too. It helped us to catch several memory leaks during Shenandoah development. The gradual approach, you mentioned, make sense to me. Matter of fact, we already started with Unsafe.allocateMemory() :-) Thanks, -Zhengyu > > (I think even if we were to instrument parts of the JDK - e.g. just > NIO - this would already be very helpful. In parts we do this already > for mtOther.). > > On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu wrote: >> >> FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) >> https://bugs.openjdk.java.net/browse/JDK-6995789 >> >> -Zhengyu >> >> On 11/21/18 9:28 AM, Thomas St?fe wrote: >>> Hi all, >>> >>> (yet again not sure if this is serviceablity-dev or not - I start at >>> hs-dev, feel free to move this mail around.) >>> >>> Do we have any plans to extend NMT to cover native JDK libaries too? >>> That would be a really cool feature. >>> >>> -- >>> >>> We at SAP have done a similar thing in the past: >>> >>> We have a monitoring facility in our port which tracks C-heap >>> allocations, non-imaginatively called "malloc statistic". This feature >>> predates NMT somewhat - had we had NMT at that time, we would not have >>> bothered. Our Malloc statistic is less powerful than NMT and >>> implementation-wise completely at odds with it, so I never felt the >>> urge to bring it upstream. However, one thing we did do is we extended >>> its coverage to the JDK native code. >>> >>> This has been quite helpful in the past to find leaks in JDK, see >>> e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 >>> >>> We did this by exposing os::malloc, os::free etc from libjvm.so >>> ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we >>> then either manually replaced calls to raw ::malloc(), ::free() etc >>> with JVM_malloc(), JVM_free(). Or, in places where this was possible, >>> we did this replacement stuff wholesale by employing a header which >>> re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, >>> we also had to add a number of linkage dependencies to the libjvm.so. >>> >>> All this is pretty standard stuff. >>> >>> One detail stood out: malloc headers are evil. In our experience, JDK >>> native code was more difficult to control and "unbalanced >>> malloc/frees" kept creeping in - especially with the >>> wholesale-redefinition technique. Unbalanced mallocs/frees means cases >>> where malloc() is instrumented but ::free() stays raw, or the other >>> way around. Both combinations are catastrophic since os::malloc uses >>> malloc headers. We typically corrupted the C-Heap and crashed, often >>> much later in completely unrelated places. >>> >>> These types of bugs were very hard to spot and hence very expensive. >>> And they can creep in in many ways. One example, there exist a >>> surprising number of system APIs which return results in C-heap and >>> require the user to free that, which of course must happen with raw >>> ::free(), not os::free(). >>> >>> We fixed this by not using malloc headers. That means a pointer >>> returned by os::malloc() is compatible with raw ::free() and vice >>> versa. The only bad thing happening would be our statistic numbers >>> being slightly off. >>> >>> Instead of malloc headers we use a hand-groomed hash table to track >>> the malloced memory. It is actually quite fast, fast enough that this >>> malloc statistic feature is on-by-default in our port. >>> >>> -- >>> >>> Of course, if we extend NMT to JDK native code we also would want to >>> extend it to mmap() etc - we never did this with our statistic, since >>> it only tracked malloc. >>> >>> What do you think? Did anyone else play with similar ideas? Would it >>> be worth the effort? >>> >>> Cheers, Thomas >>> From thomas.stuefe at gmail.com Wed Nov 21 15:43:02 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 21 Nov 2018 16:43:02 +0100 Subject: Extend NMT to JDK native libraries? In-Reply-To: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> References: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> Message-ID: On Wed, Nov 21, 2018 at 4:31 PM Zhengyu Gu wrote: > > Hi Thomas, > > On 11/21/18 9:59 AM, Thomas St?fe wrote: > > Thanks Zhengyu. > > > > Hm, closed as wont fix I see. But the solution outlined there differs > > from our proposal. > > It was just a placeholder. In fact, we did discussed the approach that > you purposed. IIRC, we concluded that initial effort was just too great. > > Global replacement of ::malloc()/::free()/::realloc() can be a good > starting point, at the end, we do want library code to follow hotspot's > convention - provides correct memory tags. A single mtOther for all > library code does not seem very helpful. > > I do believe NMT can be very helpful for library development too. It > helped us to catch several memory leaks during Shenandoah development. > > The gradual approach, you mentioned, make sense to me. Matter of fact, > we already started with Unsafe.allocateMemory() :-) yes okay, this could be done piecemeal. But note the unbalanced-malloc issue. The more we expose NMT to the JDK wild, the more probable are such bugs. Within the hotspot all is nice and tidy. If we wanted to avoid these bugs, we would have to remove malloc headers from both os::malloc() and NMT MallocTracker and add a malloc pointer hashtable of some sorts to the latter. This is not very difficult, but would still need an initial investment. Thanks, Thomas > > Thanks, > > -Zhengyu > > > > > (I think even if we were to instrument parts of the JDK - e.g. just > > NIO - this would already be very helpful. In parts we do this already > > for mtOther.). > > > > On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu wrote: > >> > >> FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) > >> https://bugs.openjdk.java.net/browse/JDK-6995789 > >> > >> -Zhengyu > >> > >> On 11/21/18 9:28 AM, Thomas St?fe wrote: > >>> Hi all, > >>> > >>> (yet again not sure if this is serviceablity-dev or not - I start at > >>> hs-dev, feel free to move this mail around.) > >>> > >>> Do we have any plans to extend NMT to cover native JDK libaries too? > >>> That would be a really cool feature. > >>> > >>> -- > >>> > >>> We at SAP have done a similar thing in the past: > >>> > >>> We have a monitoring facility in our port which tracks C-heap > >>> allocations, non-imaginatively called "malloc statistic". This feature > >>> predates NMT somewhat - had we had NMT at that time, we would not have > >>> bothered. Our Malloc statistic is less powerful than NMT and > >>> implementation-wise completely at odds with it, so I never felt the > >>> urge to bring it upstream. However, one thing we did do is we extended > >>> its coverage to the JDK native code. > >>> > >>> This has been quite helpful in the past to find leaks in JDK, see > >>> e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 > >>> > >>> We did this by exposing os::malloc, os::free etc from libjvm.so > >>> ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we > >>> then either manually replaced calls to raw ::malloc(), ::free() etc > >>> with JVM_malloc(), JVM_free(). Or, in places where this was possible, > >>> we did this replacement stuff wholesale by employing a header which > >>> re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, > >>> we also had to add a number of linkage dependencies to the libjvm.so. > >>> > >>> All this is pretty standard stuff. > >>> > >>> One detail stood out: malloc headers are evil. In our experience, JDK > >>> native code was more difficult to control and "unbalanced > >>> malloc/frees" kept creeping in - especially with the > >>> wholesale-redefinition technique. Unbalanced mallocs/frees means cases > >>> where malloc() is instrumented but ::free() stays raw, or the other > >>> way around. Both combinations are catastrophic since os::malloc uses > >>> malloc headers. We typically corrupted the C-Heap and crashed, often > >>> much later in completely unrelated places. > >>> > >>> These types of bugs were very hard to spot and hence very expensive. > >>> And they can creep in in many ways. One example, there exist a > >>> surprising number of system APIs which return results in C-heap and > >>> require the user to free that, which of course must happen with raw > >>> ::free(), not os::free(). > >>> > >>> We fixed this by not using malloc headers. That means a pointer > >>> returned by os::malloc() is compatible with raw ::free() and vice > >>> versa. The only bad thing happening would be our statistic numbers > >>> being slightly off. > >>> > >>> Instead of malloc headers we use a hand-groomed hash table to track > >>> the malloced memory. It is actually quite fast, fast enough that this > >>> malloc statistic feature is on-by-default in our port. > >>> > >>> -- > >>> > >>> Of course, if we extend NMT to JDK native code we also would want to > >>> extend it to mmap() etc - we never did this with our statistic, since > >>> it only tracked malloc. > >>> > >>> What do you think? Did anyone else play with similar ideas? Would it > >>> be worth the effort? > >>> > >>> Cheers, Thomas > >>> From erik.osterlund at oracle.com Wed Nov 21 15:44:31 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 16:44:31 +0100 Subject: RFR: 8214056: Allow the GC to attach context information to CompiledMethod In-Reply-To: References: <5BF2D678.3030208@oracle.com> <5f3e983a-e53a-a7bd-b589-c3e497ef5364@redhat.com> <5BF2E012.5010701@oracle.com> <9db0d59e-e8ef-1a60-f62a-4569177de23c@redhat.com> Message-ID: <5BF57D5F.6020005@oracle.com> Hi Vladimir, Thanks for the review. /Erik On 2018-11-21 02:08, Vladimir Kozlov wrote: > +1 > > thanks, > Vladimir > > On 11/19/18 8:34 AM, Andrew Dinn wrote: >> On 19/11/2018 16:08, Erik ?sterlund wrote: >>> Thanks for the review. I liked your idea of having the casting done by >>> the accessors, like we did for GC data on Thread. >>> So here is a new webrev with that approach: >>> http://cr.openjdk.java.net/~eosterlund/8214056/webrev.01/ >> Yes, that's a lot nicer. >> >> regards, >> >> >> Andrew Dinn >> ----------- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in England and Wales under Company Registration No. 03798903 >> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander >> From jcbeyler at google.com Wed Nov 21 16:19:34 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 21 Nov 2018 08:19:34 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> Message-ID: Hi Chris, Thanks for taking the time to look at it and yes you have raised exactly why the webrev is between two worlds: in cases where a fatal error on failure is wanted, should we simplify the code to remove the return tests since we do them internally? Now that I've looked around for non-fatal cases, I think the answer is yes, it simplifies the code while maintaining the checks. I looked a bit and it seems that I can't find easily a case where the test accepts a JNI failure to then move on. Therefore, perhaps, for now, the fail with a Fatal is enough and we can work on the tests to clean them up? That means that this is the new webrev with only Fatal and cleans up the tests so that it is no longer in between two worlds: Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 (This passes testing on my dev machine for all the modified tests) with the example you provided, it now looks like: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html Where it does, to me at least, seem cleaner and less "noisy". Let me know what you think, Jc On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer wrote: > Hi JC, > > Sorry about the delay. I had to go back an look at the initial 8210842 > webrev and RFR thread to see what this was initially all about. > > In general the changes look good. > > I don't have a good answer to your FatalError/NonFatalError question. It > makes the code a lot cleaner to use FatalError, but then it is a behavior > change, and you also need to deal with tests that intentionally induce > errors (do you have an example of that). > > In any case, right now your webrev seems to be between two worlds. You are > producing FatalError, but still checking results. Here's a good example: > > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > I'm not sure if this is just a temporary state until it was decided which > approach to take. > > thanks, > > Chris > > > On 11/20/18 2:14 PM, JC Beyler wrote: > > Hi all, > > Chris thought it made sense to have more eyes on this change than just > serviceability as it will modify to tests that are not only serviceability > tests so I've moved this to conversation here :) > > For convenience, I've copy-pasted the initial RFR: > > Could I have a review for the extension and usage of the > ExceptionJniWrapper. This adds lines and filenames to the end of the > wrapper JNI methods, adds tracing, and throws an error if need be. I've > ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've > ported a few of the tests that were already changed for the assignment > webrev for JDK-8212884. > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > For illustration, if I force an error to the AP04/ap04t03 test and set the > verbosity on, I get something like: > > >> Calling JNI method FindClass from ap04t003.cpp:343 > >> Calling with these parameter(s): > java/lang/Threadd > Wait for thread to finish > << Called JNI method FindClass from ap04t003.cpp:343 > Exception in thread "Thread-0" java.lang.NoClassDefFoundError: > java/lang/Threadd > at > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > Method) > at > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > at > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > Caused by: java.lang.ClassNotFoundException: java.lang.Threadd > at > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) > at > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) > ... 3 more > FATAL ERROR in native method: JNI method FindClass : internal error from > ap04t003.cpp:343 > at > nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native > Method) > at > nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) > at > nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) > > Questions/comments I have about this are: > - Do we want to force fatal errors when a JNI call fails in general? > Most of these tests do the right thing and test the return of the JNI > calls, for example: > thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); > if (thrClass == NULL) { > > but now the wrapper actually would do a fatal if the FindClass call would > return a nullptr, so we could remove that test altogether. What do you > think? > - I prefer to leave them as the tests then become closer to what real > users would have in their code and is the "recommended" way of doing it > > > - The alternative is to use the NonFatalError I added which then just > prints out that something went wrong, letting the test continue. Question > will be what should be the default? The fatal or the non-fatal error > handling? > > On a different subject: > - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI > wrapper handles the tracing and the verify in almost the same way; only > difference I can really tell is that the complain method from NSK has a max > complain before stopping to "complain"; I have not added that part of the > code in this webrev > > Once we decide on these, I can continue on the files from JDK-8212884 and > then do both the assignment in an if extraction followed-by this type of > webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can be > deprecated as well as we go forward. > > Thanks! > Jc > > On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer > wrote: > >> On 11/19/18 10:07 AM, JC Beyler wrote: >> >> Hi all, >> >> @David/Chris: should I then push this RFR to the hotspot mailing or the >> runtime one? For what it's worth, a lot of the tests under the vmTestbase >> are jvmti so the review also affects serviceability; it just turns out I >> started with the GC originally and then hit some other tests I had touched >> via the assignment extraction. >> >> I think hotspot would be best. >> >> Chris >> >> >> @Serguei: Done for the method renaming, for the indent, are you talking >> about going from the 8-indent to 4-indent? If so, would it not just be >> better to do a new JBS bug and do the whole files in one go? I ask because >> otherwise, it will look a bit weird to have parts of the file as 8-indent >> and others 4-indent? >> >> Thanks for looking at it! >> Jc >> >> On Mon, Nov 19, 2018 at 1:25 AM serguei.spitsyn at oracle.com < >> serguei.spitsyn at oracle.com> wrote: >> >>> Hi Jc, >>> >>> We have to start this review anyway. :) >>> It looks good to me in general. >>> Thank you for your consistency in this refactoring! >>> >>> Some minor comments. >>> >>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>> >>> +static const char* remove_folders(const char* fullname) { >>> >>> I'd suggest to rename the function name to something traditional like get_basename. >>> Otherwise, it sounds like this function has to really remove folders. :) >>> >>> >>> Also, all *Locker.cpp have wrong indent in the bodies of if and while statements. >>> Could this be fixed with the refactoring? >>> >>> I did not look on how this impacts the tests other than serviceability. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/16/18 19:43, JC Beyler wrote: >>> >>> Hi all, >>> >>> Anybody motivated to review this? :) >>> Jc >>> >>> On Wed, Nov 7, 2018 at 9:53 PM JC Beyler wrote: >>> >>>> Hi all, >>>> >>>> Could I have a review for the extension and usage of the >>>> ExceptionJniWrapper. This adds lines and filenames to the end of the >>>> wrapper JNI methods, adds tracing, and throws an error if need be. I've >>>> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >>>> ported a few of the tests that were already changed for the assignment >>>> webrev for JDK-8212884. >>>> >>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>> >>>> For illustration, if I force an error to the AP04/ap04t03 test and set >>>> the verbosity on, I get something like: >>>> >>>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>>> >> Calling with these parameter(s): >>>> java/lang/Threadd >>>> Wait for thread to finish >>>> << Called JNI method FindClass from ap04t003.cpp:343 >>>> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >>>> java/lang/Threadd >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> Method) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >>>> at >>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>> at >>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>> at >>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>> ... 3 more >>>> FATAL ERROR in native method: JNI method FindClass : internal error >>>> from ap04t003.cpp:343 >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>> Method) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>> at >>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>> >>>> Questions/comments I have about this are: >>>> - Do we want to force fatal errors when a JNI call fails in general? >>>> Most of these tests do the right thing and test the return of the JNI >>>> calls, for example: >>>> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >>>> if (thrClass == NULL) { >>>> >>>> but now the wrapper actually would do a fatal if the FindClass call >>>> would return a nullptr, so we could remove that test altogether. What do >>>> you think? >>>> - I prefer to leave them as the tests then become closer to what >>>> real users would have in their code and is the "recommended" way of doing it >>>> >>>> - The alternative is to use the NonFatalError I added which then >>>> just prints out that something went wrong, letting the test continue. >>>> Question will be what should be the default? The fatal or the non-fatal >>>> error handling? >>>> >>>> On a different subject: >>>> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >>>> wrapper handles the tracing and the verify in almost the same way; only >>>> difference I can really tell is that the complain method from NSK has a max >>>> complain before stopping to "complain"; I have not added that part of the >>>> code in this webrev >>>> >>>> Once we decide on these, I can continue on the files from JDK-8212884 >>>> and then do both the assignment in an if extraction followed-by this type >>>> of webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can >>>> be deprecated as well as we go forward. >>>> >>>> Thank you for the reviews/comments :) >>>> Jc >>>> >>> >>> >>> -- >>> >>> Thanks, >>> Jc >>> >>> >>> >> >> -- >> >> Thanks, >> Jc >> >> >> > > -- > > Thanks, > Jc > > > -- Thanks, Jc From zgu at redhat.com Wed Nov 21 16:26:11 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 21 Nov 2018 11:26:11 -0500 Subject: Extend NMT to JDK native libraries? In-Reply-To: References: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> Message-ID: <22785794-bee7-98e9-2151-c8487f65744a@redhat.com> > > But note the unbalanced-malloc issue. The more we expose NMT to the > JDK wild, the more probable are such bugs. Within the hotspot all is > nice and tidy. Hotspot also has a few unbalanced-malloc instances. And yes, we will find more in library, I would consider it as initial investment to fix them, especially, if we can do them module-by-module, no? Mismatched statistics is quite annoying ... Did see people actually counting bytes and expecting to match :-) JDK-8191369 actually was driven by customers, who tried to match smap. Thanks, -Zhengyu > > If we wanted to avoid these bugs, we would have to remove malloc > headers from both os::malloc() and NMT MallocTracker and add a malloc > pointer hashtable of some sorts to the latter. This is not very > difficult, but would still need an initial investment. > > Thanks, Thomas > >> >> Thanks, >> >> -Zhengyu >> >>> >>> (I think even if we were to instrument parts of the JDK - e.g. just >>> NIO - this would already be very helpful. In parts we do this already >>> for mtOther.). >>> >>> On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu wrote: >>>> >>>> FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) >>>> https://bugs.openjdk.java.net/browse/JDK-6995789 >>>> >>>> -Zhengyu >>>> >>>> On 11/21/18 9:28 AM, Thomas St?fe wrote: >>>>> Hi all, >>>>> >>>>> (yet again not sure if this is serviceablity-dev or not - I start at >>>>> hs-dev, feel free to move this mail around.) >>>>> >>>>> Do we have any plans to extend NMT to cover native JDK libaries too? >>>>> That would be a really cool feature. >>>>> >>>>> -- >>>>> >>>>> We at SAP have done a similar thing in the past: >>>>> >>>>> We have a monitoring facility in our port which tracks C-heap >>>>> allocations, non-imaginatively called "malloc statistic". This feature >>>>> predates NMT somewhat - had we had NMT at that time, we would not have >>>>> bothered. Our Malloc statistic is less powerful than NMT and >>>>> implementation-wise completely at odds with it, so I never felt the >>>>> urge to bring it upstream. However, one thing we did do is we extended >>>>> its coverage to the JDK native code. >>>>> >>>>> This has been quite helpful in the past to find leaks in JDK, see >>>>> e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 >>>>> >>>>> We did this by exposing os::malloc, os::free etc from libjvm.so >>>>> ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we >>>>> then either manually replaced calls to raw ::malloc(), ::free() etc >>>>> with JVM_malloc(), JVM_free(). Or, in places where this was possible, >>>>> we did this replacement stuff wholesale by employing a header which >>>>> re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, >>>>> we also had to add a number of linkage dependencies to the libjvm.so. >>>>> >>>>> All this is pretty standard stuff. >>>>> >>>>> One detail stood out: malloc headers are evil. In our experience, JDK >>>>> native code was more difficult to control and "unbalanced >>>>> malloc/frees" kept creeping in - especially with the >>>>> wholesale-redefinition technique. Unbalanced mallocs/frees means cases >>>>> where malloc() is instrumented but ::free() stays raw, or the other >>>>> way around. Both combinations are catastrophic since os::malloc uses >>>>> malloc headers. We typically corrupted the C-Heap and crashed, often >>>>> much later in completely unrelated places. >>>>> >>>>> These types of bugs were very hard to spot and hence very expensive. >>>>> And they can creep in in many ways. One example, there exist a >>>>> surprising number of system APIs which return results in C-heap and >>>>> require the user to free that, which of course must happen with raw >>>>> ::free(), not os::free(). >>>>> >>>>> We fixed this by not using malloc headers. That means a pointer >>>>> returned by os::malloc() is compatible with raw ::free() and vice >>>>> versa. The only bad thing happening would be our statistic numbers >>>>> being slightly off. >>>>> >>>>> Instead of malloc headers we use a hand-groomed hash table to track >>>>> the malloced memory. It is actually quite fast, fast enough that this >>>>> malloc statistic feature is on-by-default in our port. >>>>> >>>>> -- >>>>> >>>>> Of course, if we extend NMT to JDK native code we also would want to >>>>> extend it to mmap() etc - we never did this with our statistic, since >>>>> it only tracked malloc. >>>>> >>>>> What do you think? Did anyone else play with similar ideas? Would it >>>>> be worth the effort? >>>>> >>>>> Cheers, Thomas >>>>> From dean.long at oracle.com Wed Nov 21 16:32:49 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 21 Nov 2018 08:32:49 -0800 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: <5BF56AFC.70607@oracle.com> References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> <5BF56AFC.70607@oracle.com> Message-ID: I think the problem is CompiledMethodIterator vs NMethodIterator. CompiledMethodIterator will visit AOT methods, but NMethodIterator will not.? As long as AOT methods can't be unloaded, it makes sense to continue to use NMethodIterator and move unloading logic from CompiledMethod to nmethod. dl On 11/21/18 6:26 AM, Erik ?sterlund wrote: > Hi Vladimir, > > Thank you for the explanation. > > I inserted some debugging code to be able to draw better conclusions. > I put in a bool that tags CompiledMethods that are visited during code > cache unloading. > > When I crash, I find the AOTCompiledMethod in the following state: > > * The CodeCache::_unloading_cycle is 2, meaning there has been at > least one unloading that occurred since the VM started. > * My new boolean saying if it was found during code cache walking said > false; it has never been observed by GC code cache unloading. > * The _is_unloading_state of the AOT method is 2, meaning it was from > CodeCache::_unloading_cycle number 1. So there has been at least one > code cache unloading cycle since the AOT method was created. > * It is in the CompiledMethod state in_use. > * The corresponding CodeToAMethod entry has _state equal to in_use, > and the _aot pointer points right back to the AOT method as expected. > > So the conclusions I can draw from this is that we are looking at an > AOTCompiledMethod that is in_use, alive, and published, but was > created at least one unloading cycle earlier, yet was *not* found by > the evident code cache unloading that has occurred between creating > the AOT method and the crash. In fact it has not been found in any > code cache unloading event at all. > > So I am certain about the problem being that the AOT method is not > found during our code cache walk, and that the solution to that > problem is to move the epoch business to nmethod instead. However, it > would be desirable to know exactly why the AOT method isn't showing up > in the code cache walk, but I'm afraid I can't see right now exactly > why it has been missed out, only that it definitely has been missed > out. :/ > > /Erik > > On 2018-11-20 19:08, Vladimir Kozlov wrote: >> To clarify. >> >> 'invalid' state is set only for AOT methods which never was registers >> (their state is 'not_set' and corresponding class was not initialized >> yet). Such methods are never visible to iterators and IC caches. >> >> If aot method is registered its state become 'in_use' and in such >> case it is treated and processed as normal nmethod. And >> AOTCompiledMethod structure is created only for such case. >> >> Vladimir >> >> On 11/20/18 3:12 AM, Erik ?sterlund wrote: >>> Hi Dean, >>> >>> Yeah I think I misunderstood what I observed. So what I know is that >>> in the code cache unloading, I'm not getting all is_alive() AOT >>> methods into my iterator, which messes up the assumptions made by >>> the epoch based scheme for AOT methods. >>> >>> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, >>> int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT >>> heap, making it no longer observable from the iterators. Then it >>> calls the VM_Deoptimize vm operation after. Throughout all this, the >>> AOTCompiledMethod is alive(), yet when the iterators ask for all >>> is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC >>> caches may still reference these methods and check if it >>> is_unloading, and then we blow up. There may possibly be multiple >>> ways for is_alive() AOTCompiledMethods to not be visible from >>> iterators yet be visible through IC caches using the "invalid" state >>> in the . >>> >>> Anyway, the fix is the same: stop doing the epoch state thingey for >>> is_unloading() on AOTCompiledMethod where it isn't needed, and >>> doesn't seem to play well with the rather different life cycle it >>> has, and just return false instead. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>>> Hi Erik, >>>> >>>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>>> ...except it looks like for AOTCompiledMethods when running with >>>>> tiered compilation, may first be is_alive(), then become >>>>> !is_alive() for a while, and then get resurrected to is_alive() >>>>> using make_entrant(). >>>> >>>> this doesn't sounds quite right.? AOTCompiledMethods aren't allowed >>>> to transition to zombie (!alive), only not_used or not_entrant, >>>> which should still have is_alive() returning true. Maybe some code >>>> is using is_not_entrant() instead of !is_alive()? >>>> >>>> dl >>> > From erik.osterlund at oracle.com Wed Nov 21 17:18:28 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 18:18:28 +0100 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> <5BF56AFC.70607@oracle.com> Message-ID: <2d0d17c6-5591-3200-83ea-aa0097a00715@oracle.com> Hi Dean, Code cache unloading does indeed use CompiledMethodIterator though. And it needs to continue doing so as it also cleans IC caches of AOT methods. I did make a discovery though. I inserted an assert after creating AOT methods in AOTCodeHeap::register_stubs(), to check that the published AOT methods are reachable from the iterators. And they are not. The same assert passes when publishing normal AOT methods. I'm guessing the stubs do not have IC caches, and hence nobody has needed them to show up from the iterators until now. So now I'm gonna go with that as my explanation what the problem is, and stick with the same proposed solution as before. Since we don't unload AOT methods, I want is_unloading() to just return false on them, and leave the state fiddling to nmethods. /Erik On 2018-11-21 17:32, dean.long at oracle.com wrote: > I think the problem is CompiledMethodIterator vs NMethodIterator. > CompiledMethodIterator will visit AOT methods, but NMethodIterator will > not.? As long as AOT methods can't be unloaded, it makes sense to > continue to use NMethodIterator and move unloading logic from > CompiledMethod to nmethod. > > dl > > On 11/21/18 6:26 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> >> Thank you for the explanation. >> >> I inserted some debugging code to be able to draw better conclusions. >> I put in a bool that tags CompiledMethods that are visited during code >> cache unloading. >> >> When I crash, I find the AOTCompiledMethod in the following state: >> >> * The CodeCache::_unloading_cycle is 2, meaning there has been at >> least one unloading that occurred since the VM started. >> * My new boolean saying if it was found during code cache walking said >> false; it has never been observed by GC code cache unloading. >> * The _is_unloading_state of the AOT method is 2, meaning it was from >> CodeCache::_unloading_cycle number 1. So there has been at least one >> code cache unloading cycle since the AOT method was created. >> * It is in the CompiledMethod state in_use. >> * The corresponding CodeToAMethod entry has _state equal to in_use, >> and the _aot pointer points right back to the AOT method as expected. >> >> So the conclusions I can draw from this is that we are looking at an >> AOTCompiledMethod that is in_use, alive, and published, but was >> created at least one unloading cycle earlier, yet was *not* found by >> the evident code cache unloading that has occurred between creating >> the AOT method and the crash. In fact it has not been found in any >> code cache unloading event at all. >> >> So I am certain about the problem being that the AOT method is not >> found during our code cache walk, and that the solution to that >> problem is to move the epoch business to nmethod instead. However, it >> would be desirable to know exactly why the AOT method isn't showing up >> in the code cache walk, but I'm afraid I can't see right now exactly >> why it has been missed out, only that it definitely has been missed >> out. :/ >> >> /Erik >> >> On 2018-11-20 19:08, Vladimir Kozlov wrote: >>> To clarify. >>> >>> 'invalid' state is set only for AOT methods which never was registers >>> (their state is 'not_set' and corresponding class was not initialized >>> yet). Such methods are never visible to iterators and IC caches. >>> >>> If aot method is registered its state become 'in_use' and in such >>> case it is treated and processed as normal nmethod. And >>> AOTCompiledMethod structure is created only for such case. >>> >>> Vladimir >>> >>> On 11/20/18 3:12 AM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>> Yeah I think I misunderstood what I observed. So what I know is that >>>> in the code cache unloading, I'm not getting all is_alive() AOT >>>> methods into my iterator, which messes up the assumptions made by >>>> the epoch based scheme for AOT methods. >>>> >>>> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, >>>> int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT >>>> heap, making it no longer observable from the iterators. Then it >>>> calls the VM_Deoptimize vm operation after. Throughout all this, the >>>> AOTCompiledMethod is alive(), yet when the iterators ask for all >>>> is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC >>>> caches may still reference these methods and check if it >>>> is_unloading, and then we blow up. There may possibly be multiple >>>> ways for is_alive() AOTCompiledMethods to not be visible from >>>> iterators yet be visible through IC caches using the "invalid" state >>>> in the . >>>> >>>> Anyway, the fix is the same: stop doing the epoch state thingey for >>>> is_unloading() on AOTCompiledMethod where it isn't needed, and >>>> doesn't seem to play well with the rather different life cycle it >>>> has, and just return false instead. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>>>> Hi Erik, >>>>> >>>>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>>>> ...except it looks like for AOTCompiledMethods when running with >>>>>> tiered compilation, may first be is_alive(), then become >>>>>> !is_alive() for a while, and then get resurrected to is_alive() >>>>>> using make_entrant(). >>>>> >>>>> this doesn't sounds quite right.? AOTCompiledMethods aren't allowed >>>>> to transition to zombie (!alive), only not_used or not_entrant, >>>>> which should still have is_alive() returning true. Maybe some code >>>>> is using is_not_entrant() instead of !is_alive()? >>>>> >>>>> dl >>>> >> > From aph at redhat.com Wed Nov 21 17:29:20 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 21 Nov 2018 17:29:20 +0000 Subject: RFR: 8213845: ARM32: Interpreter doesn't call result handler after native calls In-Reply-To: References: Message-ID: <799d69ad-5eae-26bb-04b0-70e25862f326@redhat.com> On 11/21/18 10:29 AM, Nick Gasson wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213845 >>> Webrev: http://cr.openjdk.java.net/~njian/8213845/webrev.0/ >> >> That looks right, thanks. > > Thanks Andrew! Could you help to commit this patch for me if it > doesn't need other reviews? All my 32-bit Arm hardware is dead right now, so it's rather hard for me to test it. Perhaps someone else could try building it, or you could figure out some way to get me access to appropriate hardware. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Wed Nov 21 17:37:05 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 21 Nov 2018 17:37:05 +0000 Subject: Problems with AES-GCM native acceleration In-Reply-To: References: <92a16009b9e047eacf102b006bc0612bac4a2cfb.camel@redhat.com> <1b1fe9e4-ad71-7d48-13eb-48dbe267962e@oracle.com> <864a763d-0d67-d894-234c-fbbef78e0320@oracle.com> Message-ID: On 11/15/18 10:42 AM, Gidon Gershinsky wrote: > Having the decryption optimized in the HotSpot engine would be ideal. I agree with you. I did a few experiments, and it can take a very long time for C2 compilation to kick in, especially because GCM is glacially slow until the intrinsics are used. I think this would be a generally useful enhancement to HotSpot, and I'm kicking around an experimental patch which adds the intrinsics to c1 and the interpreter. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From boris.ulasevich at bell-sw.com Wed Nov 21 17:45:16 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Wed, 21 Nov 2018 20:45:16 +0300 Subject: RFR: 8213845: ARM32: Interpreter doesn't call result handler after native calls In-Reply-To: <799d69ad-5eae-26bb-04b0-70e25862f326@redhat.com> References: <799d69ad-5eae-26bb-04b0-70e25862f326@redhat.com> Message-ID: Hi Nick, ? We will check and push the fix. Thanks! Boris 21.11.2018 20:29, Andrew Haley ?????: > On 11/21/18 10:29 AM, Nick Gasson wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213845 >>>> Webrev: http://cr.openjdk.java.net/~njian/8213845/webrev.0/ >>> That looks right, thanks. >> Thanks Andrew! Could you help to commit this patch for me if it >> doesn't need other reviews? > All my 32-bit Arm hardware is dead right now, so it's rather hard > for me to test it. Perhaps someone else could try building it, or > you could figure out some way to get me access to appropriate > hardware. > From vladimir.kozlov at oracle.com Wed Nov 21 17:49:42 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 21 Nov 2018 09:49:42 -0800 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> <5BF56AFC.70607@oracle.com> Message-ID: From the start AOT methods were never scanned for unloading because they don't have embedded oops. We don't need to explicitly "unload" aot methods to avoid GC to scan them - they don't have oops. Oops and metadata referenced by aot methods are kept in separate arrays. Scanning aot oops and metadata arrays are done by separate code. At least this is what I remember. If you look on original aot unloading code - it did nothing: http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/hotspot/share/aot/aotCompiledMethod.cpp#l78 Based on that I think it is correct to move unloading logic to nmethod as you suggested. I still can't remember how we proved that should not be any "activations"/call to such aot methods if they are not marked as 'unloaded'. Vladimir On 11/21/18 8:32 AM, dean.long at oracle.com wrote: > I think the problem is CompiledMethodIterator vs NMethodIterator. CompiledMethodIterator will visit AOT methods, but > NMethodIterator will not.? As long as AOT methods can't be unloaded, it makes sense to continue to use NMethodIterator > and move unloading logic from CompiledMethod to nmethod. > > dl > > On 11/21/18 6:26 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> >> Thank you for the explanation. >> >> I inserted some debugging code to be able to draw better conclusions. >> I put in a bool that tags CompiledMethods that are visited during code cache unloading. >> >> When I crash, I find the AOTCompiledMethod in the following state: >> >> * The CodeCache::_unloading_cycle is 2, meaning there has been at least one unloading that occurred since the VM started. >> * My new boolean saying if it was found during code cache walking said false; it has never been observed by GC code >> cache unloading. >> * The _is_unloading_state of the AOT method is 2, meaning it was from CodeCache::_unloading_cycle number 1. So there >> has been at least one code cache unloading cycle since the AOT method was created. >> * It is in the CompiledMethod state in_use. >> * The corresponding CodeToAMethod entry has _state equal to in_use, and the _aot pointer points right back to the AOT >> method as expected. >> >> So the conclusions I can draw from this is that we are looking at an AOTCompiledMethod that is in_use, alive, and >> published, but was created at least one unloading cycle earlier, yet was *not* found by the evident code cache >> unloading that has occurred between creating the AOT method and the crash. In fact it has not been found in any code >> cache unloading event at all. >> >> So I am certain about the problem being that the AOT method is not found during our code cache walk, and that the >> solution to that problem is to move the epoch business to nmethod instead. However, it would be desirable to know >> exactly why the AOT method isn't showing up in the code cache walk, but I'm afraid I can't see right now exactly why >> it has been missed out, only that it definitely has been missed out. :/ >> >> /Erik >> >> On 2018-11-20 19:08, Vladimir Kozlov wrote: >>> To clarify. >>> >>> 'invalid' state is set only for AOT methods which never was registers (their state is 'not_set' and corresponding >>> class was not initialized yet). Such methods are never visible to iterators and IC caches. >>> >>> If aot method is registered its state become 'in_use' and in such case it is treated and processed as normal nmethod. >>> And AOTCompiledMethod structure is created only for such case. >>> >>> Vladimir >>> >>> On 11/20/18 3:12 AM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>> Yeah I think I misunderstood what I observed. So what I know is that in the code cache unloading, I'm not getting >>>> all is_alive() AOT methods into my iterator, which messes up the assumptions made by the epoch based scheme for AOT >>>> methods. >>>> >>>> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, int methods_cnt) we make AOTCompiledMethods >>>> "invalid" in the AOT heap, making it no longer observable from the iterators. Then it calls the VM_Deoptimize vm >>>> operation after. Throughout all this, the AOTCompiledMethod is alive(), yet when the iterators ask for all >>>> is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC caches may still reference these methods and >>>> check if it is_unloading, and then we blow up. There may possibly be multiple ways for is_alive() AOTCompiledMethods >>>> to not be visible from iterators yet be visible through IC caches using the "invalid" state in the . >>>> >>>> Anyway, the fix is the same: stop doing the epoch state thingey for is_unloading() on AOTCompiledMethod where it >>>> isn't needed, and doesn't seem to play well with the rather different life cycle it has, and just return false instead. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>>>> Hi Erik, >>>>> >>>>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>>>> ...except it looks like for AOTCompiledMethods when running with tiered compilation, may first be is_alive(), then >>>>>> become !is_alive() for a while, and then get resurrected to is_alive() using make_entrant(). >>>>> >>>>> this doesn't sounds quite right.? AOTCompiledMethods aren't allowed to transition to zombie (!alive), only not_used >>>>> or not_entrant, which should still have is_alive() returning true. Maybe some code is using is_not_entrant() >>>>> instead of !is_alive()? >>>>> >>>>> dl >>>> >> > From vladimir.kozlov at oracle.com Wed Nov 21 17:55:47 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 21 Nov 2018 09:55:47 -0800 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> Message-ID: <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> You are right. I think I was confused that you are not limiting MaxVectorSize but simple use min(16, MaxVectorSize). The assert is fine. Thanks, Vladimir On 11/20/18 10:28 PM, Yang Zhang (Arm Technology China) wrote: > >> I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. >> In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). >> aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. > > Matcher::vector_width_in_bytes() limits vector size to min(16, MaxVectorSize). > Matcher::vector_shift_count_ideal_reg() assert(MaxVectorSize >= size, "Length isn't supported"); > When MaxVectorSize is 8, size is 8. > When MaxVectorSize is greater than or equal 16, size is 16. > So (MaxVectorSize >= size) is always true. > > Test cases of cr6340864 are passed with or without this new assert. > > Do I need to remove this new assert? > > Regards > Yang > > > On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: >> Hi, >> >> When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? >> >> Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ >> JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 >> >> This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. >> Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. >> compiler/c2/cr6340864/TestByteVect.java >> compiler/c2/cr6340864/TestIntVect.java >> compiler/c2/cr6340864/TestShortVect.java >> compiler/codegen/TestCharVect2.java >> >> Regards >> Yang >> >> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > From erik.osterlund at oracle.com Wed Nov 21 18:21:50 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 21 Nov 2018 19:21:50 +0100 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> <5BF56AFC.70607@oracle.com> Message-ID: Hi Vladimir, Thank you for the background. It sounds like we agree about the solution. Note though as I replied to Dean that we do need to call do_unloading on all AOT methods seen in iterators to clean their IC caches referring to unloading nmethods. Seems like the runtime stub AOT methods are hidden from iterators (according to an assert I added), and hence go under the radar due to not having IC caches. But still the solution is to just return false for is_unloading() on AOT methods and move the state fiddling to nmethod. /Erik > On 21 Nov 2018, at 18:49, Vladimir Kozlov wrote: > > From the start AOT methods were never scanned for unloading because they don't have embedded oops. > We don't need to explicitly "unload" aot methods to avoid GC to scan them - they don't have oops. Oops and metadata referenced by aot methods are kept in separate arrays. Scanning aot oops and metadata arrays are done by separate code. > At least this is what I remember. If you look on original aot unloading code - it did nothing: > http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/hotspot/share/aot/aotCompiledMethod.cpp#l78 > > Based on that I think it is correct to move unloading logic to nmethod as you suggested. > > I still can't remember how we proved that should not be any "activations"/call to such aot methods if they are not marked as 'unloaded'. > > Vladimir > >> On 11/21/18 8:32 AM, dean.long at oracle.com wrote: >> I think the problem is CompiledMethodIterator vs NMethodIterator. CompiledMethodIterator will visit AOT methods, but NMethodIterator will not. As long as AOT methods can't be unloaded, it makes sense to continue to use NMethodIterator and move unloading logic from CompiledMethod to nmethod. >> dl >>> On 11/21/18 6:26 AM, Erik ?sterlund wrote: >>> Hi Vladimir, >>> >>> Thank you for the explanation. >>> >>> I inserted some debugging code to be able to draw better conclusions. >>> I put in a bool that tags CompiledMethods that are visited during code cache unloading. >>> >>> When I crash, I find the AOTCompiledMethod in the following state: >>> >>> * The CodeCache::_unloading_cycle is 2, meaning there has been at least one unloading that occurred since the VM started. >>> * My new boolean saying if it was found during code cache walking said false; it has never been observed by GC code cache unloading. >>> * The _is_unloading_state of the AOT method is 2, meaning it was from CodeCache::_unloading_cycle number 1. So there has been at least one code cache unloading cycle since the AOT method was created. >>> * It is in the CompiledMethod state in_use. >>> * The corresponding CodeToAMethod entry has _state equal to in_use, and the _aot pointer points right back to the AOT method as expected. >>> >>> So the conclusions I can draw from this is that we are looking at an AOTCompiledMethod that is in_use, alive, and published, but was created at least one unloading cycle earlier, yet was *not* found by the evident code cache unloading that has occurred between creating the AOT method and the crash. In fact it has not been found in any code cache unloading event at all. >>> >>> So I am certain about the problem being that the AOT method is not found during our code cache walk, and that the solution to that problem is to move the epoch business to nmethod instead. However, it would be desirable to know exactly why the AOT method isn't showing up in the code cache walk, but I'm afraid I can't see right now exactly why it has been missed out, only that it definitely has been missed out. :/ >>> >>> /Erik >>> >>>> On 2018-11-20 19:08, Vladimir Kozlov wrote: >>>> To clarify. >>>> >>>> 'invalid' state is set only for AOT methods which never was registers (their state is 'not_set' and corresponding class was not initialized yet). Such methods are never visible to iterators and IC caches. >>>> >>>> If aot method is registered its state become 'in_use' and in such case it is treated and processed as normal nmethod. And AOTCompiledMethod structure is created only for such case. >>>> >>>> Vladimir >>>> >>>>> On 11/20/18 3:12 AM, Erik ?sterlund wrote: >>>>> Hi Dean, >>>>> >>>>> Yeah I think I misunderstood what I observed. So what I know is that in the code cache unloading, I'm not getting all is_alive() AOT methods into my iterator, which messes up the assumptions made by the epoch based scheme for AOT methods. >>>>> >>>>> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT heap, making it no longer observable from the iterators. Then it calls the VM_Deoptimize vm operation after. Throughout all this, the AOTCompiledMethod is alive(), yet when the iterators ask for all is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC caches may still reference these methods and check if it is_unloading, and then we blow up. There may possibly be multiple ways for is_alive() AOTCompiledMethods to not be visible from iterators yet be visible through IC caches using the "invalid" state in the . >>>>> >>>>> Anyway, the fix is the same: stop doing the epoch state thingey for is_unloading() on AOTCompiledMethod where it isn't needed, and doesn't seem to play well with the rather different life cycle it has, and just return false instead. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>>>>> Hi Erik, >>>>>> >>>>>>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>>>>> ...except it looks like for AOTCompiledMethods when running with tiered compilation, may first be is_alive(), then become !is_alive() for a while, and then get resurrected to is_alive() using make_entrant(). >>>>>> >>>>>> this doesn't sounds quite right. AOTCompiledMethods aren't allowed to transition to zombie (!alive), only not_used or not_entrant, which should still have is_alive() returning true. Maybe some code is using is_not_entrant() instead of !is_alive()? >>>>>> >>>>>> dl >>>>> >>> From vladimir.kozlov at oracle.com Wed Nov 21 18:30:54 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 21 Nov 2018 10:30:54 -0800 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <5BF5605F.70708@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> <5BF431EE.5000609@oracle.com> <8025dfcc-aa78-26c4-8cfd-395c388efd8c@oracle.com> <5BF5605F.70708@oracle.com> Message-ID: On 11/21/18 5:40 AM, Erik ?sterlund wrote: > Hi Vladimir, > > Thanks for reviewing. > > On 2018-11-21 02:04, Vladimir Kozlov wrote: >> I also thought about combining repeative checks is_alive() && !is_unloading() - to match new enum: >> >> bool is_alive_and_not_unloading() { return is_alive() && !is_unloading(); } >> >> But then I see only 5 cases and separate checks seems better for me. >> >> One question is where you clean up 'unloading' state when nmethod state change to 'unloaded'? Is it possible that >> nmethod to have both state at the same time? > > Yes. The answer to is_alive() monotonically goes from true to false and never back. Similarly, the answer to > is_unloading() monotonically goes from false to true and never back. Once is_unloading - always is_unloading(). This is > intentional to avoid races where reading is_unloading() again would race and give a different answer. Okay, so nmethod will be is_unloading() regardless _state value until it is removed from CodeCache. I am confused by code in bool nmethod::can_convert_to_zombie() which from one side check only is_unloading() in assert but both (!is_unloading() || is_unloaded()) in final condition. On other hand can_convert_to_zombie() is only called when cm->is_not_entrant() is true in NMethodSweeper::process_compiled_method(). Thanks, Vladimir > > Thanks, > /Erik > >> Thanks, >> Vladimir >> >> On 11/20/18 8:10 AM, Erik ?sterlund wrote: >>> Hi Robbin, >>> >>> Thanks for the review. I think I will go without is_dying() now because I couldn't figure out how to put it in, in an >>> intuitive way. Mostly because most of the code is checking for is_alive() && !is_unloading() compiled methods. In >>> that state they are not is_dying(), according to your definition, but checking for !is_dying() doesn't imply that it >>> is alive. So I think I will stick with being more explicit for now. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-11-20 10:57, Robbin Ehn wrote: >>>> Looks good! >>>> >>>> I gave a suggestion IRL about an is_dying method, which covers the is_alive >>>> and is_unloading query. If choose to take it or not I'll leave it up to you. >>>> >>>> Thanks, Robbin >>>> >>>> On 11/19/18 1:31 PM, Erik ?sterlund wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for having a look at this. >>>>> >>>>> New full webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>>>> >>>>> Incremental: >>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>>>> >>>>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>>>> >>>>>> How about instead of two bool arguments that are hard to read and error prone and need comments, like: >>>>>> >>>>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* only_not_unloading */); >>>>>> >>>>>> >>>>>> enum FilterAlive { all_blobs, only_alive }; >>>>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>>>> >>>>>> Then it can be something like: >>>>>> >>>>>> ?? CompiledMethodIterator iter(only_alive, all_blobs); >>>>>> >>>>>> Don't know if you can repeate all_blobs like that. >>>>> >>>>> You can't really have all_blobs in both. I tried out a variant with a single enum though, hope you like it. >>>>> The observation was that we only need 3 different variations: all_blobs, only_alive or >>>>> only_alive_and_not_unloading. So I made an enum out of that. >>>>> In fact, these three modes are the only ones I support right now, so they are also the only valid options, which >>>>> made the single enum look even more reasonable. >>>>> >>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>>>> >>>>>> I'm not sure if metadata_do() should filter out unloading. It's looking for "old" methods to stop from >>>>>> deallocating memory for them (but it's in a safepoint).? If you're unloading, the IC caches are already cleaned out? >>>>> >>>>> IC caches to unloading nmethods are not necessarily cleaned out. The concurrent GC threads will clean the IC caches >>>>> concurrently, but a JavaThread can call the IC cache before the GC has gotten around to cleaning the IC cache. But >>>>> any such call into an is_unloading() nmethod will be trapped by nmethod entry barriers, which will call the IC miss >>>>> handler, which sorts things out lazily. So you can view this as the IC caches to is_unloading() nmethods being >>>>> logically cleared, but not necessarily physically cleared yet. The effect is the same - the is_unloading() nmethods >>>>> may not be entered from anywhere, including IC caches. >>>>> >>>>> So if you were to not filter is_unloading(), you would get a whole bunch of nmethods that are not really reachable >>>>> by the application, i.e. they are not on any stacks, and may not be entered by the application. Therefore, I think >>>>> it's perfectly fine to filter them out here, unless I missed something. And this makes the behaviour closer to >>>>> as-if the nmethods were indeed unloaded in the safepoint. >>>>> >>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>>>> >>>>>> 1108 // The release is only needed for compile-time ordering, as accesses >>>>>> 1109 // into the nmethod after the store is not safe, due to the sweeper >>>>>> 1110 // being allowed to free it then, in the case of concurrent nmethod >>>>>> 1111 // unloading. Therefore, there is no need for acquire on the loader side. >>>>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>>>> >>>>>> I tried to make sense out of this first sentence, but couldn't really.? After the store to unloaded, can the >>>>>> sweeper free the nmethod?? maybe remove "then, " and it would make more sense? >>>>> >>>>> Right, after the store is observed, the sweeper thread may concurrently observe the nmethod to be unloaded, and >>>>> then it may immediately flip it over to zombie. And I don't want any concurrent make_unloaded code in that nmethod >>>>> to still be racing after the sweeper turns it into zombie; a very unnecessary race to deal with. >>>>> >>>>> I updated the comment - hopefully it reads better now. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> Thanks, >>>>>> Coleen >>>>>> >>>>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>>>> ..put in bug number in subject to make mail filters happy. >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> All current GCs perform code cache unloading in safepoints. Therefore, nmethods that are alive but >>>>>>>> is_unloading() are never observed outside of safepoints. With concurrent class unloading, nmethods that are >>>>>>>> alive but is_unloading() will become observable outside of safepoints. This must be handled appropriately. >>>>>>>> >>>>>>>> In this patch I changed the nmethod/compiled method iterators to accept parameters describing whether they >>>>>>>> should filter out not is_alive() or is_unloading() methods. Since there is no obvious default (all combinations >>>>>>>> are used depending on call site), you have to explicitly set what iteration mode you want. >>>>>>>> >>>>>>>> Other than that, I make sure that the sweeper stays away from is_unloading() nmethods that are not yet >>>>>>>> is_unloaded(). To make the interactions between the sweeper and concurrent GC threads safe, I had to move down >>>>>>>> the store that sets the state to unloaded, and use a release_store there, to make sure no accesses float below >>>>>>>> it at compile-time. Once that store is observed, nmethods may be deleted. >>>>>>>> >>>>>>>> In the IC miss handler, I also need to lazily clean stale IC caches due to calling is_unloading nmethods using >>>>>>>> nmethod entry barriers. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>>>> >>>>>>>> Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>> >>>>> >>> > From vladimir.kozlov at oracle.com Wed Nov 21 18:33:03 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 21 Nov 2018 10:33:03 -0800 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> <5BF56AFC.70607@oracle.com> Message-ID: Sounds good. Vladimir On 11/21/18 10:21 AM, Erik Osterlund wrote: > Hi Vladimir, > > Thank you for the background. It sounds like we agree about the solution. > > Note though as I replied to Dean that we do need to call do_unloading on all AOT methods seen in iterators to clean their IC caches referring to unloading nmethods. Seems like the runtime stub AOT methods are hidden from iterators (according to an assert I added), and hence go under the radar due to not having IC caches. > > But still the solution is to just return false for is_unloading() on AOT methods and move the state fiddling to nmethod. > > /Erik > >> On 21 Nov 2018, at 18:49, Vladimir Kozlov wrote: >> >> From the start AOT methods were never scanned for unloading because they don't have embedded oops. >> We don't need to explicitly "unload" aot methods to avoid GC to scan them - they don't have oops. Oops and metadata referenced by aot methods are kept in separate arrays. Scanning aot oops and metadata arrays are done by separate code. >> At least this is what I remember. If you look on original aot unloading code - it did nothing: >> http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/hotspot/share/aot/aotCompiledMethod.cpp#l78 >> >> Based on that I think it is correct to move unloading logic to nmethod as you suggested. >> >> I still can't remember how we proved that should not be any "activations"/call to such aot methods if they are not marked as 'unloaded'. >> >> Vladimir >> >>> On 11/21/18 8:32 AM, dean.long at oracle.com wrote: >>> I think the problem is CompiledMethodIterator vs NMethodIterator. CompiledMethodIterator will visit AOT methods, but NMethodIterator will not. As long as AOT methods can't be unloaded, it makes sense to continue to use NMethodIterator and move unloading logic from CompiledMethod to nmethod. >>> dl >>>> On 11/21/18 6:26 AM, Erik ?sterlund wrote: >>>> Hi Vladimir, >>>> >>>> Thank you for the explanation. >>>> >>>> I inserted some debugging code to be able to draw better conclusions. >>>> I put in a bool that tags CompiledMethods that are visited during code cache unloading. >>>> >>>> When I crash, I find the AOTCompiledMethod in the following state: >>>> >>>> * The CodeCache::_unloading_cycle is 2, meaning there has been at least one unloading that occurred since the VM started. >>>> * My new boolean saying if it was found during code cache walking said false; it has never been observed by GC code cache unloading. >>>> * The _is_unloading_state of the AOT method is 2, meaning it was from CodeCache::_unloading_cycle number 1. So there has been at least one code cache unloading cycle since the AOT method was created. >>>> * It is in the CompiledMethod state in_use. >>>> * The corresponding CodeToAMethod entry has _state equal to in_use, and the _aot pointer points right back to the AOT method as expected. >>>> >>>> So the conclusions I can draw from this is that we are looking at an AOTCompiledMethod that is in_use, alive, and published, but was created at least one unloading cycle earlier, yet was *not* found by the evident code cache unloading that has occurred between creating the AOT method and the crash. In fact it has not been found in any code cache unloading event at all. >>>> >>>> So I am certain about the problem being that the AOT method is not found during our code cache walk, and that the solution to that problem is to move the epoch business to nmethod instead. However, it would be desirable to know exactly why the AOT method isn't showing up in the code cache walk, but I'm afraid I can't see right now exactly why it has been missed out, only that it definitely has been missed out. :/ >>>> >>>> /Erik >>>> >>>>> On 2018-11-20 19:08, Vladimir Kozlov wrote: >>>>> To clarify. >>>>> >>>>> 'invalid' state is set only for AOT methods which never was registers (their state is 'not_set' and corresponding class was not initialized yet). Such methods are never visible to iterators and IC caches. >>>>> >>>>> If aot method is registered its state become 'in_use' and in such case it is treated and processed as normal nmethod. And AOTCompiledMethod structure is created only for such case. >>>>> >>>>> Vladimir >>>>> >>>>>> On 11/20/18 3:12 AM, Erik ?sterlund wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Yeah I think I misunderstood what I observed. So what I know is that in the code cache unloading, I'm not getting all is_alive() AOT methods into my iterator, which messes up the assumptions made by the epoch based scheme for AOT methods. >>>>>> >>>>>> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* indexes, int methods_cnt) we make AOTCompiledMethods "invalid" in the AOT heap, making it no longer observable from the iterators. Then it calls the VM_Deoptimize vm operation after. Throughout all this, the AOTCompiledMethod is alive(), yet when the iterators ask for all is_alive() AOTCompiledMethods, it won't be visible. But I suppose IC caches may still reference these methods and check if it is_unloading, and then we blow up. There may possibly be multiple ways for is_alive() AOTCompiledMethods to not be visible from iterators yet be visible through IC caches using the "invalid" state in the . >>>>>> >>>>>> Anyway, the fix is the same: stop doing the epoch state thingey for is_unloading() on AOTCompiledMethod where it isn't needed, and doesn't seem to play well with the rather different life cycle it has, and just return false instead. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>>>>>> Hi Erik, >>>>>>> >>>>>>>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>>>>>> ...except it looks like for AOTCompiledMethods when running with tiered compilation, may first be is_alive(), then become !is_alive() for a while, and then get resurrected to is_alive() using make_entrant(). >>>>>>> >>>>>>> this doesn't sounds quite right. AOTCompiledMethods aren't allowed to transition to zombie (!alive), only not_used or not_entrant, which should still have is_alive() returning true. Maybe some code is using is_not_entrant() instead of !is_alive()? >>>>>>> >>>>>>> dl >>>>>> >>>> > From jiangli.zhou at oracle.com Wed Nov 21 19:14:45 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 21 Nov 2018 11:14:45 -0800 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: References: Message-ID: <34bb0796-48e6-910f-5023-6ed8e46ecec4@oracle.com> Hi Stefan, The change looks good. Thanks for fixing this issue! Thanks, Jiangli On 11/21/18 6:43 AM, Stefan Johansson wrote: > Hi, > > Please review this fix to avoid leaving heap metadata in an > inconsistent state when CDS archive mapping fails. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 > Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ > > Summary > ------- > Since JDK 12b15 a default CDS archive is shipped with the JDK. If this > archive for some reason fails to be mapped into the heap, the cleanup > leaves the heap metadata in a inconsistent state. Instead of clearing > the range in the map storing which regions are archive, it once again > marks the range as archive. > > This information is used by the G1 Full collections and because a > regions is marked as archive even though it is not, we skip marking > through this region. This causes the heap to be corrupt and we can > crash in many different ways. > > Testing > ------- > Verified that the new code clears the map correctly and re-ran the > benchmark previously failing many times. Tier 1-3 of normal functional > testing also run through m5 without problems. > > Thanks, > Stefan From erik.osterlund at oracle.com Wed Nov 21 20:16:14 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 21:16:14 +0100 Subject: RFR: 8213486: SIGSEGV in CompiledMethod::cleanup_inline_caches_impl with AOT In-Reply-To: References: <2e1abc86-feca-6c2a-3dc4-02372a4913de@oracle.com> <5BF3EC1F.8060003@oracle.com> <5BF56AFC.70607@oracle.com> Message-ID: Hi Vladimir, Thanks for the review. /Erik On 2018-11-21 19:33, Vladimir Kozlov wrote: > Sounds good. > > Vladimir > > On 11/21/18 10:21 AM, Erik Osterlund wrote: >> Hi Vladimir, >> >> Thank you for the background. It sounds like we agree about the solution. >> >> Note though as I replied to Dean that we do need to call do_unloading >> on all AOT methods seen in iterators to clean their IC caches >> referring to unloading nmethods. Seems like the runtime stub AOT >> methods are hidden from iterators (according to an assert I added), >> and hence go under the radar due to not having IC caches. >> >> But still the solution is to just return false for is_unloading() on >> AOT methods and move the state fiddling to nmethod. >> >> /Erik >> >>> On 21 Nov 2018, at 18:49, Vladimir Kozlov >>> wrote: >>> >>> ?From the start AOT methods were never scanned for unloading because >>> they don't have embedded oops. >>> We don't need to explicitly "unload" aot methods to avoid GC to scan >>> them - they don't have oops. Oops and metadata referenced by aot >>> methods are kept in separate arrays. Scanning aot oops and metadata >>> arrays are done by separate code. >>> At least this is what I remember. If you look on original aot >>> unloading code - it did nothing: >>> http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/hotspot/share/aot/aotCompiledMethod.cpp#l78 >>> >>> >>> Based on that I think it is correct to move unloading logic to >>> nmethod as you suggested. >>> >>> I still can't remember how we proved that should not be any >>> "activations"/call to such aot methods if they are not marked as >>> 'unloaded'. >>> >>> Vladimir >>> >>>> On 11/21/18 8:32 AM, dean.long at oracle.com wrote: >>>> I think the problem is CompiledMethodIterator vs NMethodIterator. >>>> CompiledMethodIterator will visit AOT methods, but NMethodIterator >>>> will not.? As long as AOT methods can't be unloaded, it makes sense >>>> to continue to use NMethodIterator and move unloading logic from >>>> CompiledMethod to nmethod. >>>> dl >>>>> On 11/21/18 6:26 AM, Erik ?sterlund wrote: >>>>> Hi Vladimir, >>>>> >>>>> Thank you for the explanation. >>>>> >>>>> I inserted some debugging code to be able to draw better conclusions. >>>>> I put in a bool that tags CompiledMethods that are visited during >>>>> code cache unloading. >>>>> >>>>> When I crash, I find the AOTCompiledMethod in the following state: >>>>> >>>>> * The CodeCache::_unloading_cycle is 2, meaning there has been at >>>>> least one unloading that occurred since the VM started. >>>>> * My new boolean saying if it was found during code cache walking >>>>> said false; it has never been observed by GC code cache unloading. >>>>> * The _is_unloading_state of the AOT method is 2, meaning it was >>>>> from CodeCache::_unloading_cycle number 1. So there has been at >>>>> least one code cache unloading cycle since the AOT method was created. >>>>> * It is in the CompiledMethod state in_use. >>>>> * The corresponding CodeToAMethod entry has _state equal to in_use, >>>>> and the _aot pointer points right back to the AOT method as expected. >>>>> >>>>> So the conclusions I can draw from this is that we are looking at >>>>> an AOTCompiledMethod that is in_use, alive, and published, but was >>>>> created at least one unloading cycle earlier, yet was *not* found >>>>> by the evident code cache unloading that has occurred between >>>>> creating the AOT method and the crash. In fact it has not been >>>>> found in any code cache unloading event at all. >>>>> >>>>> So I am certain about the problem being that the AOT method is not >>>>> found during our code cache walk, and that the solution to that >>>>> problem is to move the epoch business to nmethod instead. However, >>>>> it would be desirable to know exactly why the AOT method isn't >>>>> showing up in the code cache walk, but I'm afraid I can't see right >>>>> now exactly why it has been missed out, only that it definitely has >>>>> been missed out. :/ >>>>> >>>>> /Erik >>>>> >>>>>> On 2018-11-20 19:08, Vladimir Kozlov wrote: >>>>>> To clarify. >>>>>> >>>>>> 'invalid' state is set only for AOT methods which never was >>>>>> registers (their state is 'not_set' and corresponding class was >>>>>> not initialized yet). Such methods are never visible to iterators >>>>>> and IC caches. >>>>>> >>>>>> If aot method is registered its state become 'in_use' and in such >>>>>> case it is treated and processed as normal nmethod. And >>>>>> AOTCompiledMethod structure is created only for such case. >>>>>> >>>>>> Vladimir >>>>>> >>>>>>> On 11/20/18 3:12 AM, Erik ?sterlund wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> Yeah I think I misunderstood what I observed. So what I know is >>>>>>> that in the code cache unloading, I'm not getting all is_alive() >>>>>>> AOT methods into my iterator, which messes up the assumptions >>>>>>> made by the epoch based scheme for AOT methods. >>>>>>> >>>>>>> I noticed that in AOTCodeHeap::sweep_dependent_methods(int* >>>>>>> indexes, int methods_cnt) we make AOTCompiledMethods "invalid" in >>>>>>> the AOT heap, making it no longer observable from the iterators. >>>>>>> Then it calls the VM_Deoptimize vm operation after. Throughout >>>>>>> all this, the AOTCompiledMethod is alive(), yet when the >>>>>>> iterators ask for all is_alive() AOTCompiledMethods, it won't be >>>>>>> visible. But I suppose IC caches may still reference these >>>>>>> methods and check if it is_unloading, and then we blow up. There >>>>>>> may possibly be multiple ways for is_alive() AOTCompiledMethods >>>>>>> to not be visible from iterators yet be visible through IC caches >>>>>>> using the "invalid" state in the . >>>>>>> >>>>>>> Anyway, the fix is the same: stop doing the epoch state thingey >>>>>>> for is_unloading() on AOTCompiledMethod where it isn't needed, >>>>>>> and doesn't seem to play well with the rather different life >>>>>>> cycle it has, and just return false instead. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>>> On 2018-11-20 00:00, dean.long at oracle.com wrote: >>>>>>>> Hi Erik, >>>>>>>> >>>>>>>>> On 11/19/18 12:42 PM, Erik ?sterlund wrote: >>>>>>>>> ...except it looks like for AOTCompiledMethods when running >>>>>>>>> with tiered compilation, may first be is_alive(), then become >>>>>>>>> !is_alive() for a while, and then get resurrected to is_alive() >>>>>>>>> using make_entrant(). >>>>>>>> >>>>>>>> this doesn't sounds quite right.? AOTCompiledMethods aren't >>>>>>>> allowed to transition to zombie (!alive), only not_used or >>>>>>>> not_entrant, which should still have is_alive() returning true. >>>>>>>> Maybe some code is using is_not_entrant() instead of !is_alive()? >>>>>>>> >>>>>>>> dl >>>>>>> >>>>> >> From per.liden at oracle.com Wed Nov 21 20:18:52 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 21 Nov 2018 21:18:52 +0100 Subject: RFR: 8214068: ZGC crashes with vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java In-Reply-To: References: Message-ID: On 2018-11-21 15:58, Daniel D. Daugherty wrote: > On 11/21/18 4:11 AM, Per Liden wrote: >> This patch adds a missing load barrier to the JVMTI heap walker. In >> VM_HeapWalkOperation::collect_simple_roots() the SimpleRootsClosure >> closure should not do raw loads of oops. Instead it should be loading >> them using NativeAccess::oop_load(). >> >> This problem only affects ZGC, since it's (so far) the only collector >> that scans JNIHandles concurrently. >> >> The problem was easily reproducible when running >> vmTestbase/nsk/jdi/ReferenceType/instances/instances004/TestDescription.java. >> However, for some reason this isn't reproducing in our CI pipeline, >> likely because of luck with timing. For example, this is harder to >> reproduce in a scenario where your using a smaller heap and hence have >> GC cycles happen more frequently. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214068 >> Webrev: http://cr.openjdk.java.net/~pliden/8214068/webrev.0 > > src/hotspot/share/prims/jvmtiTagMap.cpp > ??? No comments. > > Nice catch! Thumbs up. Thanks for reviewing, Dan! > > Since this is in the JVM/TI heap walker, I wonder if there are other > sightings in JBS that might be tracked back to this. Looks like > Zhengyu put a nice stack trace in the bug report so dup matching > should be easy (if there are any). I did a quick search, but didn't stumble onto anything relevant. This bug has "only" been in there since Oct 16 (which is when ZGC started to process JNIHandles concurrently). cheers, Per From erik.osterlund at oracle.com Wed Nov 21 20:16:55 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 21 Nov 2018 21:16:55 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> <5BF431EE.5000609@oracle.com> <8025dfcc-aa78-26c4-8cfd-395c388efd8c@oracle.com> <5BF5605F.70708@oracle.com> Message-ID: <04167cb1-e9c2-54c7-cac0-03d1dde3379c@oracle.com> Hi Vladimir, On 2018-11-21 19:30, Vladimir Kozlov wrote: > On 11/21/18 5:40 AM, Erik ?sterlund wrote: >> Hi Vladimir, >> >> Thanks for reviewing. >> >> On 2018-11-21 02:04, Vladimir Kozlov wrote: >>> I also thought about combining repeative checks is_alive() && >>> !is_unloading() - to match new enum: >>> >>> bool is_alive_and_not_unloading() { return is_alive() && >>> !is_unloading(); } >>> >>> But then I see only 5 cases and separate checks seems better for me. >>> >>> One question is where you clean up 'unloading' state when nmethod >>> state change to 'unloaded'? Is it possible that nmethod to have both >>> state at the same time? >> >> Yes. The answer to is_alive() monotonically goes from true to false >> and never back. Similarly, the answer to is_unloading() monotonically >> goes from false to true and never back. Once is_unloading - always >> is_unloading(). This is intentional to avoid races where reading >> is_unloading() again would race and give a different answer. > > Okay, so nmethod will be is_unloading() regardless _state value until it > is removed from CodeCache. Exactly. > I am confused by code in? bool nmethod::can_convert_to_zombie() which > from one side check only is_unloading() in assert but both > (!is_unloading() || is_unloaded()) in final condition. On other hand > can_convert_to_zombie() is only called when cm->is_not_entrant() is true > in NMethodSweeper::process_compiled_method(). What this code is dealing with is that the sweeper finds a not_entrant nmethod. But it could race with the GC unloading nmethods concurrently. And then by the time we call can_convert_to_zombie(), it might no longer be not_entrant(). If the nmethod is_unloading(), it can flip to unloaded at any time. So what this code is saying is that it is still okay to in this race convert the nmethod to zombie if it becomes unloaded. Then since it was not_entrant, we know it was a non-OSR nmethod, and that it should indeed convert over to zombie. Thanks, /Erik > Thanks, > Vladimir > >> >> Thanks, >> /Erik >> >>> Thanks, >>> Vladimir >>> >>> On 11/20/18 8:10 AM, Erik ?sterlund wrote: >>>> Hi Robbin, >>>> >>>> Thanks for the review. I think I will go without is_dying() now >>>> because I couldn't figure out how to put it in, in an intuitive way. >>>> Mostly because most of the code is checking for is_alive() && >>>> !is_unloading() compiled methods. In that state they are not >>>> is_dying(), according to your definition, but checking for >>>> !is_dying() doesn't imply that it is alive. So I think I will stick >>>> with being more explicit for now. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-11-20 10:57, Robbin Ehn wrote: >>>>> Looks good! >>>>> >>>>> I gave a suggestion IRL about an is_dying method, which covers the >>>>> is_alive >>>>> and is_unloading query. If choose to take it or not I'll leave it >>>>> up to you. >>>>> >>>>> Thanks, Robbin >>>>> >>>>> On 11/19/18 1:31 PM, Erik ?sterlund wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for having a look at this. >>>>>> >>>>>> New full webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>>>>> >>>>>> Incremental: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>>>>> >>>>>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>>>>> >>>>>>> How about instead of two bool arguments that are hard to read and >>>>>>> error prone and need comments, like: >>>>>>> >>>>>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* >>>>>>> only_not_unloading */); >>>>>>> >>>>>>> >>>>>>> enum FilterAlive { all_blobs, only_alive }; >>>>>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>>>>> >>>>>>> Then it can be something like: >>>>>>> >>>>>>> ?? CompiledMethodIterator iter(only_alive, all_blobs); >>>>>>> >>>>>>> Don't know if you can repeate all_blobs like that. >>>>>> >>>>>> You can't really have all_blobs in both. I tried out a variant >>>>>> with a single enum though, hope you like it. >>>>>> The observation was that we only need 3 different variations: >>>>>> all_blobs, only_alive or only_alive_and_not_unloading. So I made >>>>>> an enum out of that. >>>>>> In fact, these three modes are the only ones I support right now, >>>>>> so they are also the only valid options, which made the single >>>>>> enum look even more reasonable. >>>>>> >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>>>>> >>>>>>> >>>>>>> I'm not sure if metadata_do() should filter out unloading. It's >>>>>>> looking for "old" methods to stop from deallocating memory for >>>>>>> them (but it's in a safepoint).? If you're unloading, the IC >>>>>>> caches are already cleaned out? >>>>>> >>>>>> IC caches to unloading nmethods are not necessarily cleaned out. >>>>>> The concurrent GC threads will clean the IC caches concurrently, >>>>>> but a JavaThread can call the IC cache before the GC has gotten >>>>>> around to cleaning the IC cache. But any such call into an >>>>>> is_unloading() nmethod will be trapped by nmethod entry barriers, >>>>>> which will call the IC miss handler, which sorts things out >>>>>> lazily. So you can view this as the IC caches to is_unloading() >>>>>> nmethods being logically cleared, but not necessarily physically >>>>>> cleared yet. The effect is the same - the is_unloading() nmethods >>>>>> may not be entered from anywhere, including IC caches. >>>>>> >>>>>> So if you were to not filter is_unloading(), you would get a whole >>>>>> bunch of nmethods that are not really reachable by the >>>>>> application, i.e. they are not on any stacks, and may not be >>>>>> entered by the application. Therefore, I think it's perfectly fine >>>>>> to filter them out here, unless I missed something. And this makes >>>>>> the behaviour closer to as-if the nmethods were indeed unloaded in >>>>>> the safepoint. >>>>>> >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>>>>> >>>>>>> >>>>>>> 1108 // The release is only needed for compile-time ordering, as >>>>>>> accesses >>>>>>> 1109 // into the nmethod after the store is not safe, due to the >>>>>>> sweeper >>>>>>> 1110 // being allowed to free it then, in the case of concurrent >>>>>>> nmethod >>>>>>> 1111 // unloading. Therefore, there is no need for acquire on the >>>>>>> loader side. >>>>>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>>>>> >>>>>>> I tried to make sense out of this first sentence, but couldn't >>>>>>> really.? After the store to unloaded, can the sweeper free the >>>>>>> nmethod?? maybe remove "then, " and it would make more sense? >>>>>> >>>>>> Right, after the store is observed, the sweeper thread may >>>>>> concurrently observe the nmethod to be unloaded, and then it may >>>>>> immediately flip it over to zombie. And I don't want any >>>>>> concurrent make_unloaded code in that nmethod to still be racing >>>>>> after the sweeper turns it into zombie; a very unnecessary race to >>>>>> deal with. >>>>>> >>>>>> I updated the comment - hopefully it reads better now. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> Thanks, >>>>>>> Coleen >>>>>>> >>>>>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>>>>> ..put in bug number in subject to make mail filters happy. >>>>>>>> >>>>>>>> /Erik >>>>>>>> >>>>>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> All current GCs perform code cache unloading in safepoints. >>>>>>>>> Therefore, nmethods that are alive but is_unloading() are never >>>>>>>>> observed outside of safepoints. With concurrent class >>>>>>>>> unloading, nmethods that are alive but is_unloading() will >>>>>>>>> become observable outside of safepoints. This must be handled >>>>>>>>> appropriately. >>>>>>>>> >>>>>>>>> In this patch I changed the nmethod/compiled method iterators >>>>>>>>> to accept parameters describing whether they should filter out >>>>>>>>> not is_alive() or is_unloading() methods. Since there is no >>>>>>>>> obvious default (all combinations are used depending on call >>>>>>>>> site), you have to explicitly set what iteration mode you want. >>>>>>>>> >>>>>>>>> Other than that, I make sure that the sweeper stays away from >>>>>>>>> is_unloading() nmethods that are not yet is_unloaded(). To make >>>>>>>>> the interactions between the sweeper and concurrent GC threads >>>>>>>>> safe, I had to move down the store that sets the state to >>>>>>>>> unloaded, and use a release_store there, to make sure no >>>>>>>>> accesses float below it at compile-time. Once that store is >>>>>>>>> observed, nmethods may be deleted. >>>>>>>>> >>>>>>>>> In the IC miss handler, I also need to lazily clean stale IC >>>>>>>>> caches due to calling is_unloading nmethods using nmethod entry >>>>>>>>> barriers. >>>>>>>>> >>>>>>>>> Webrev: >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>>>>> >>>>>>>>> Bug: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> /Erik >>>>>>> >>>>>> >>>> >> From mark.reinhold at oracle.com Wed Nov 21 20:19:27 2018 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 21 Nov 2018 12:19:27 -0800 (PST) Subject: New candidate JEP: 347: Adopt C++14 Language Features in HotSpot Message-ID: <20181121201927.E6496230DA1@eggemoggin.niobe.net> https://openjdk.java.net/jeps/347 - Mark From thomas.schatzl at oracle.com Wed Nov 21 20:29:23 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 21 Nov 2018 21:29:23 +0100 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: References: Message-ID: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> Hi, On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: > Hi, > > Please review this fix to avoid leaving heap metadata in an > inconsistent state when CDS archive mapping fails. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 > Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ the change seems good. I have two comments on this though: - would it be useful to clear both maps in G1ArchiveAllocator::clear_range_archive()? I mean in the failure case these regions can't be used for one purpose or the other as soon as we fail the mapping, and saves passing the bool flag. - this is a cleanup request for the future: is there a reason to have separate tables for open and closed archives? I understand that the code happened this way because these types of archives were added at different times, but in my understanding a region in that table can either be one or the other, and by using a uint8_t as per-region entry we can even save space. Thanks, Thomas From jiangli.zhou at oracle.com Wed Nov 21 21:05:27 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 21 Nov 2018 13:05:27 -0800 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> References: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> Message-ID: <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> Hi Thomas, Just some comments for your comments. :) On 11/21/18 12:29 PM, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: >> Hi, >> >> Please review this fix to avoid leaving heap metadata in an >> inconsistent state when CDS archive mapping fails. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 >> Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ > the change seems good. I have two comments on this though: > > - would it be useful to clear both maps in > G1ArchiveAllocator::clear_range_archive()? I mean in the failure case > these regions can't be used for one purpose or the other as soon as we > fail the mapping, and saves passing the bool flag. We map the 'closed' archive regions before 'open' archive regions. The 'open' archive regions are only mapped if the 'closed' ones are successfully mapped. The 'open' archive regions have dependency on the 'closed' regions (objects in the open regions may have references to the closed objects). The 'closed' archive regions can be used independently however. If 'open' archive regions fail to map, we can still keep the 'closed' archives in use (no need to clear the closed map). > > - this is a cleanup request for the future: is there a reason to have > separate tables for open and closed archives? I understand that the > code happened this way because these types of archives were added at > different times, but in my understanding a region in that table can > either be one or the other, and by using a uint8_t as per-region entry > we can even save space. One benefit of two separate maps is that we can quickly find which range an object is in. It probably could be done without using the maps... Thanks, Jiangli > > Thanks, > Thomas > > From vladimir.kozlov at oracle.com Wed Nov 21 21:33:01 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 21 Nov 2018 13:33:01 -0800 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: <04167cb1-e9c2-54c7-cac0-03d1dde3379c@oracle.com> References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> <5BF431EE.5000609@oracle.com> <8025dfcc-aa78-26c4-8cfd-395c388efd8c@oracle.com> <5BF5605F.70708@oracle.com> <04167cb1-e9c2-54c7-cac0-03d1dde3379c@oracle.com> Message-ID: Good. Please, add/update comments to explain all cases in can_convert_to_zombie(). Thanks, Vladimir On 11/21/18 12:16 PM, Erik ?sterlund wrote: > Hi Vladimir, > > On 2018-11-21 19:30, Vladimir Kozlov wrote: >> On 11/21/18 5:40 AM, Erik ?sterlund wrote: >>> Hi Vladimir, >>> >>> Thanks for reviewing. >>> >>> On 2018-11-21 02:04, Vladimir Kozlov wrote: >>>> I also thought about combining repeative checks is_alive() && !is_unloading() - to match new enum: >>>> >>>> bool is_alive_and_not_unloading() { return is_alive() && !is_unloading(); } >>>> >>>> But then I see only 5 cases and separate checks seems better for me. >>>> >>>> One question is where you clean up 'unloading' state when nmethod state change to 'unloaded'? Is it possible that >>>> nmethod to have both state at the same time? >>> >>> Yes. The answer to is_alive() monotonically goes from true to false and never back. Similarly, the answer to >>> is_unloading() monotonically goes from false to true and never back. Once is_unloading - always is_unloading(). This >>> is intentional to avoid races where reading is_unloading() again would race and give a different answer. >> >> Okay, so nmethod will be is_unloading() regardless _state value until it is removed from CodeCache. > > Exactly. > >> I am confused by code in? bool nmethod::can_convert_to_zombie() which from one side check only is_unloading() in >> assert but both (!is_unloading() || is_unloaded()) in final condition. On other hand can_convert_to_zombie() is only >> called when cm->is_not_entrant() is true in NMethodSweeper::process_compiled_method(). > > What this code is dealing with is that the sweeper finds a not_entrant nmethod. But it could race with the GC unloading > nmethods concurrently. And then by the time we call can_convert_to_zombie(), it might no longer be not_entrant(). If the > nmethod is_unloading(), it can flip to unloaded at any time. So what this code is saying is that it is still okay to in > this race convert the nmethod to zombie if it becomes unloaded. Then since it was not_entrant, we know it was a non-OSR > nmethod, and that it should indeed convert over to zombie. > > Thanks, > /Erik > >> Thanks, >> Vladimir >> >>> >>> Thanks, >>> /Erik >>> >>>> Thanks, >>>> Vladimir >>>> >>>> On 11/20/18 8:10 AM, Erik ?sterlund wrote: >>>>> Hi Robbin, >>>>> >>>>> Thanks for the review. I think I will go without is_dying() now because I couldn't figure out how to put it in, in >>>>> an intuitive way. Mostly because most of the code is checking for is_alive() && !is_unloading() compiled methods. >>>>> In that state they are not is_dying(), according to your definition, but checking for !is_dying() doesn't imply >>>>> that it is alive. So I think I will stick with being more explicit for now. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-11-20 10:57, Robbin Ehn wrote: >>>>>> Looks good! >>>>>> >>>>>> I gave a suggestion IRL about an is_dying method, which covers the is_alive >>>>>> and is_unloading query. If choose to take it or not I'll leave it up to you. >>>>>> >>>>>> Thanks, Robbin >>>>>> >>>>>> On 11/19/18 1:31 PM, Erik ?sterlund wrote: >>>>>>> Hi Coleen, >>>>>>> >>>>>>> Thanks for having a look at this. >>>>>>> >>>>>>> New full webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>>>>>> >>>>>>> Incremental: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>>>>>> >>>>>>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>>>>>> >>>>>>>> How about instead of two bool arguments that are hard to read and error prone and need comments, like: >>>>>>>> >>>>>>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* only_not_unloading */); >>>>>>>> >>>>>>>> >>>>>>>> enum FilterAlive { all_blobs, only_alive }; >>>>>>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>>>>>> >>>>>>>> Then it can be something like: >>>>>>>> >>>>>>>> ?? CompiledMethodIterator iter(only_alive, all_blobs); >>>>>>>> >>>>>>>> Don't know if you can repeate all_blobs like that. >>>>>>> >>>>>>> You can't really have all_blobs in both. I tried out a variant with a single enum though, hope you like it. >>>>>>> The observation was that we only need 3 different variations: all_blobs, only_alive or >>>>>>> only_alive_and_not_unloading. So I made an enum out of that. >>>>>>> In fact, these three modes are the only ones I support right now, so they are also the only valid options, which >>>>>>> made the single enum look even more reasonable. >>>>>>> >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>>>>>> >>>>>>>> I'm not sure if metadata_do() should filter out unloading. It's looking for "old" methods to stop from >>>>>>>> deallocating memory for them (but it's in a safepoint).? If you're unloading, the IC caches are already cleaned >>>>>>>> out? >>>>>>> >>>>>>> IC caches to unloading nmethods are not necessarily cleaned out. The concurrent GC threads will clean the IC >>>>>>> caches concurrently, but a JavaThread can call the IC cache before the GC has gotten around to cleaning the IC >>>>>>> cache. But any such call into an is_unloading() nmethod will be trapped by nmethod entry barriers, which will >>>>>>> call the IC miss handler, which sorts things out lazily. So you can view this as the IC caches to is_unloading() >>>>>>> nmethods being logically cleared, but not necessarily physically cleared yet. The effect is the same - the >>>>>>> is_unloading() nmethods may not be entered from anywhere, including IC caches. >>>>>>> >>>>>>> So if you were to not filter is_unloading(), you would get a whole bunch of nmethods that are not really >>>>>>> reachable by the application, i.e. they are not on any stacks, and may not be entered by the application. >>>>>>> Therefore, I think it's perfectly fine to filter them out here, unless I missed something. And this makes the >>>>>>> behaviour closer to as-if the nmethods were indeed unloaded in the safepoint. >>>>>>> >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>>>>>> >>>>>>>> 1108 // The release is only needed for compile-time ordering, as accesses >>>>>>>> 1109 // into the nmethod after the store is not safe, due to the sweeper >>>>>>>> 1110 // being allowed to free it then, in the case of concurrent nmethod >>>>>>>> 1111 // unloading. Therefore, there is no need for acquire on the loader side. >>>>>>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>>>>>> >>>>>>>> I tried to make sense out of this first sentence, but couldn't really.? After the store to unloaded, can the >>>>>>>> sweeper free the nmethod?? maybe remove "then, " and it would make more sense? >>>>>>> >>>>>>> Right, after the store is observed, the sweeper thread may concurrently observe the nmethod to be unloaded, and >>>>>>> then it may immediately flip it over to zombie. And I don't want any concurrent make_unloaded code in that >>>>>>> nmethod to still be racing after the sweeper turns it into zombie; a very unnecessary race to deal with. >>>>>>> >>>>>>> I updated the comment - hopefully it reads better now. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>>> Thanks, >>>>>>>> Coleen >>>>>>>> >>>>>>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>>>>>> ..put in bug number in subject to make mail filters happy. >>>>>>>>> >>>>>>>>> /Erik >>>>>>>>> >>>>>>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> All current GCs perform code cache unloading in safepoints. Therefore, nmethods that are alive but >>>>>>>>>> is_unloading() are never observed outside of safepoints. With concurrent class unloading, nmethods that are >>>>>>>>>> alive but is_unloading() will become observable outside of safepoints. This must be handled appropriately. >>>>>>>>>> >>>>>>>>>> In this patch I changed the nmethod/compiled method iterators to accept parameters describing whether they >>>>>>>>>> should filter out not is_alive() or is_unloading() methods. Since there is no obvious default (all >>>>>>>>>> combinations are used depending on call site), you have to explicitly set what iteration mode you want. >>>>>>>>>> >>>>>>>>>> Other than that, I make sure that the sweeper stays away from is_unloading() nmethods that are not yet >>>>>>>>>> is_unloaded(). To make the interactions between the sweeper and concurrent GC threads safe, I had to move down >>>>>>>>>> the store that sets the state to unloaded, and use a release_store there, to make sure no accesses float below >>>>>>>>>> it at compile-time. Once that store is observed, nmethods may be deleted. >>>>>>>>>> >>>>>>>>>> In the IC miss handler, I also need to lazily clean stale IC caches due to calling is_unloading nmethods using >>>>>>>>>> nmethod entry barriers. >>>>>>>>>> >>>>>>>>>> Webrev: >>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>>>>>> >>>>>>>>>> Bug: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> /Erik >>>>>>>> >>>>>>> >>>>> >>> From thomas.schatzl at oracle.com Wed Nov 21 21:52:29 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 21 Nov 2018 22:52:29 +0100 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> References: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> Message-ID: <5330d48fd264753a3f556c5495771da2692eefca.camel@oracle.com> On Wed, 2018-11-21 at 13:05 -0800, Jiangli Zhou wrote: > Hi Thomas, > > Just some comments for your comments. :) > > On 11/21/18 12:29 PM, Thomas Schatzl wrote: > > Hi, > > > > On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: > > > Hi, > > > > > > Please review this fix to avoid leaving heap metadata in an > > > inconsistent state when CDS archive mapping fails. > > > > > > Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 > > > Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ > > > > the change seems good. I have two comments on this though: > > > > - would it be useful to clear both maps in > > G1ArchiveAllocator::clear_range_archive()? I mean in the failure > > case these regions can't be used for one purpose or the other as > > soon as we fail the mapping, and saves passing the bool flag. > > We map the 'closed' archive regions before 'open' archive regions. > The 'open' archive regions are only mapped if the 'closed' ones are > successfully mapped. > > The 'open' archive regions have dependency on the 'closed' regions > (objects in the open regions may have references to the closed > objects). > The 'closed' archive regions can be used independently however. If > 'open' archive regions fail to map, we can still keep the 'closed' > archives in use (no need to clear the closed map). Okay. Thanks for clarification. > > > > - this is a cleanup request for the future: is there a reason to > > have separate tables for open and closed archives? I understand > > that the code happened this way because these types of archives > > were added at different times, but in my understanding a region in > > that table can either be one or the other, and by using a uint8_t > > as per-region entry we can even save space. > > One benefit of two separate maps is that we can quickly find which > range an object is in. It probably could be done without using the > maps... Not against using a map to do that, my suggestion is to merge them into a single map carrying an enum value (none/closed/open) instead of two each being a bool. There should be (almost) no difference in comparing to true/false than comparing to a small integer value. Just an idea. Thanks, Thomas From jiangli.zhou at oracle.com Wed Nov 21 21:54:59 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 21 Nov 2018 13:54:59 -0800 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: <5330d48fd264753a3f556c5495771da2692eefca.camel@oracle.com> References: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> <5330d48fd264753a3f556c5495771da2692eefca.camel@oracle.com> Message-ID: On 11/21/18 1:52 PM, Thomas Schatzl wrote: > On Wed, 2018-11-21 at 13:05 -0800, Jiangli Zhou wrote: >> Hi Thomas, >> >> Just some comments for your comments. :) >> >> On 11/21/18 12:29 PM, Thomas Schatzl wrote: >>> Hi, >>> >>> On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: >>>> Hi, >>>> >>>> Please review this fix to avoid leaving heap metadata in an >>>> inconsistent state when CDS archive mapping fails. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 >>>> Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ >>> the change seems good. I have two comments on this though: >>> >>> - would it be useful to clear both maps in >>> G1ArchiveAllocator::clear_range_archive()? I mean in the failure >>> case these regions can't be used for one purpose or the other as >>> soon as we fail the mapping, and saves passing the bool flag. >> We map the 'closed' archive regions before 'open' archive regions. >> The 'open' archive regions are only mapped if the 'closed' ones are >> successfully mapped. >> >> The 'open' archive regions have dependency on the 'closed' regions >> (objects in the open regions may have references to the closed >> objects). >> The 'closed' archive regions can be used independently however. If >> 'open' archive regions fail to map, we can still keep the 'closed' >> archives in use (no need to clear the closed map). > Okay. Thanks for clarification. > >>> - this is a cleanup request for the future: is there a reason to >>> have separate tables for open and closed archives? I understand >>> that the code happened this way because these types of archives >>> were added at different times, but in my understanding a region in >>> that table can either be one or the other, and by using a uint8_t >>> as per-region entry we can even save space. >> One benefit of two separate maps is that we can quickly find which >> range an object is in. It probably could be done without using the >> maps... > Not against using a map to do that, my suggestion is to merge them into > a single map carrying an enum value (none/closed/open) instead of two > each being a bool. > > There should be (almost) no difference in comparing to true/false than > comparing to a small integer value. > > Just an idea. Sounds good. Thanks! Jiangli > > Thanks, > Thomas > > From erik.osterlund at oracle.com Wed Nov 21 22:00:45 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 21 Nov 2018 23:00:45 +0100 Subject: RFR: 8213755: Let nmethods be is_unloading() outside of safepoints In-Reply-To: References: <9edfa9fe-b3ef-ae55-7d48-fbcbf50f463c@oracle.com> <5BF2AD22.70100@oracle.com> <92bb5d9d-f8f2-ec7a-78d8-451ed18e35f4@oracle.com> <5BF431EE.5000609@oracle.com> <8025dfcc-aa78-26c4-8cfd-395c388efd8c@oracle.com> <5BF5605F.70708@oracle.com> <04167cb1-e9c2-54c7-cac0-03d1dde3379c@oracle.com> Message-ID: <0641332B-BE38-4B89-86CE-FB3786A8243A@oracle.com> Hi Vladimir, Thanks for the review. Will put in some comments before pushing. /Erik > On 21 Nov 2018, at 22:33, Vladimir Kozlov wrote: > > Good. Please, add/update comments to explain all cases in can_convert_to_zombie(). > > Thanks, > Vladimir > >> On 11/21/18 12:16 PM, Erik ?sterlund wrote: >> Hi Vladimir, >>> On 2018-11-21 19:30, Vladimir Kozlov wrote: >>>> On 11/21/18 5:40 AM, Erik ?sterlund wrote: >>>> Hi Vladimir, >>>> >>>> Thanks for reviewing. >>>> >>>>> On 2018-11-21 02:04, Vladimir Kozlov wrote: >>>>> I also thought about combining repeative checks is_alive() && !is_unloading() - to match new enum: >>>>> >>>>> bool is_alive_and_not_unloading() { return is_alive() && !is_unloading(); } >>>>> >>>>> But then I see only 5 cases and separate checks seems better for me. >>>>> >>>>> One question is where you clean up 'unloading' state when nmethod state change to 'unloaded'? Is it possible that nmethod to have both state at the same time? >>>> >>>> Yes. The answer to is_alive() monotonically goes from true to false and never back. Similarly, the answer to is_unloading() monotonically goes from false to true and never back. Once is_unloading - always is_unloading(). This is intentional to avoid races where reading is_unloading() again would race and give a different answer. >>> >>> Okay, so nmethod will be is_unloading() regardless _state value until it is removed from CodeCache. >> Exactly. >>> I am confused by code in bool nmethod::can_convert_to_zombie() which from one side check only is_unloading() in assert but both (!is_unloading() || is_unloaded()) in final condition. On other hand can_convert_to_zombie() is only called when cm->is_not_entrant() is true in NMethodSweeper::process_compiled_method(). >> What this code is dealing with is that the sweeper finds a not_entrant nmethod. But it could race with the GC unloading nmethods concurrently. And then by the time we call can_convert_to_zombie(), it might no longer be not_entrant(). If the nmethod is_unloading(), it can flip to unloaded at any time. So what this code is saying is that it is still okay to in this race convert the nmethod to zombie if it becomes unloaded. Then since it was not_entrant, we know it was a non-OSR nmethod, and that it should indeed convert over to zombie. >> Thanks, >> /Erik >>> Thanks, >>> Vladimir >>> >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> Thanks, >>>>> Vladimir >>>>> >>>>>> On 11/20/18 8:10 AM, Erik ?sterlund wrote: >>>>>> Hi Robbin, >>>>>> >>>>>> Thanks for the review. I think I will go without is_dying() now because I couldn't figure out how to put it in, in an intuitive way. Mostly because most of the code is checking for is_alive() && !is_unloading() compiled methods. In that state they are not is_dying(), according to your definition, but checking for !is_dying() doesn't imply that it is alive. So I think I will stick with being more explicit for now. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> On 2018-11-20 10:57, Robbin Ehn wrote: >>>>>>> Looks good! >>>>>>> >>>>>>> I gave a suggestion IRL about an is_dying method, which covers the is_alive >>>>>>> and is_unloading query. If choose to take it or not I'll leave it up to you. >>>>>>> >>>>>>> Thanks, Robbin >>>>>>> >>>>>>>> On 11/19/18 1:31 PM, Erik ?sterlund wrote: >>>>>>>> Hi Coleen, >>>>>>>> >>>>>>>> Thanks for having a look at this. >>>>>>>> >>>>>>>> New full webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.01/ >>>>>>>> >>>>>>>> Incremental: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00_01/ >>>>>>>> >>>>>>>>> On 2018-11-17 00:21, coleen.phillimore at oracle.com wrote: >>>>>>>>> >>>>>>>>> How about instead of two bool arguments that are hard to read and error prone and need comments, like: >>>>>>>>> >>>>>>>>> 76 CompiledMethodIterator iter(true /* only_alive */, false /* only_not_unloading */); >>>>>>>>> >>>>>>>>> >>>>>>>>> enum FilterAlive { all_blobs, only_alive }; >>>>>>>>> enum FilterUnloading { all_blobs, only_not_unloading }; >>>>>>>>> >>>>>>>>> Then it can be something like: >>>>>>>>> >>>>>>>>> CompiledMethodIterator iter(only_alive, all_blobs); >>>>>>>>> >>>>>>>>> Don't know if you can repeate all_blobs like that. >>>>>>>> >>>>>>>> You can't really have all_blobs in both. I tried out a variant with a single enum though, hope you like it. >>>>>>>> The observation was that we only need 3 different variations: all_blobs, only_alive or only_alive_and_not_unloading. So I made an enum out of that. >>>>>>>> In fact, these three modes are the only ones I support right now, so they are also the only valid options, which made the single enum look even more reasonable. >>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/codeCache.cpp.frames.html >>>>>>>>> >>>>>>>>> I'm not sure if metadata_do() should filter out unloading. It's looking for "old" methods to stop from deallocating memory for them (but it's in a safepoint). If you're unloading, the IC caches are already cleaned out? >>>>>>>> >>>>>>>> IC caches to unloading nmethods are not necessarily cleaned out. The concurrent GC threads will clean the IC caches concurrently, but a JavaThread can call the IC cache before the GC has gotten around to cleaning the IC cache. But any such call into an is_unloading() nmethod will be trapped by nmethod entry barriers, which will call the IC miss handler, which sorts things out lazily. So you can view this as the IC caches to is_unloading() nmethods being logically cleared, but not necessarily physically cleared yet. The effect is the same - the is_unloading() nmethods may not be entered from anywhere, including IC caches. >>>>>>>> >>>>>>>> So if you were to not filter is_unloading(), you would get a whole bunch of nmethods that are not really reachable by the application, i.e. they are not on any stacks, and may not be entered by the application. Therefore, I think it's perfectly fine to filter them out here, unless I missed something. And this makes the behaviour closer to as-if the nmethods were indeed unloaded in the safepoint. >>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/src/hotspot/share/code/nmethod.cpp.frames.html >>>>>>>>> >>>>>>>>> 1108 // The release is only needed for compile-time ordering, as accesses >>>>>>>>> 1109 // into the nmethod after the store is not safe, due to the sweeper >>>>>>>>> 1110 // being allowed to free it then, in the case of concurrent nmethod >>>>>>>>> 1111 // unloading. Therefore, there is no need for acquire on the loader side. >>>>>>>>> 1112 OrderAccess::release_store(&_state, (signed char)unloaded); >>>>>>>>> >>>>>>>>> I tried to make sense out of this first sentence, but couldn't really. After the store to unloaded, can the sweeper free the nmethod? maybe remove "then, " and it would make more sense? >>>>>>>> >>>>>>>> Right, after the store is observed, the sweeper thread may concurrently observe the nmethod to be unloaded, and then it may immediately flip it over to zombie. And I don't want any concurrent make_unloaded code in that nmethod to still be racing after the sweeper turns it into zombie; a very unnecessary race to deal with. >>>>>>>> >>>>>>>> I updated the comment - hopefully it reads better now. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>>> On 11/12/18 5:46 PM, Erik ?sterlund wrote: >>>>>>>>>> ..put in bug number in subject to make mail filters happy. >>>>>>>>>> >>>>>>>>>> /Erik >>>>>>>>>> >>>>>>>>>>> On 2018-11-12 23:44, Erik ?sterlund wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> All current GCs perform code cache unloading in safepoints. Therefore, nmethods that are alive but is_unloading() are never observed outside of safepoints. With concurrent class unloading, nmethods that are alive but is_unloading() will become observable outside of safepoints. This must be handled appropriately. >>>>>>>>>>> >>>>>>>>>>> In this patch I changed the nmethod/compiled method iterators to accept parameters describing whether they should filter out not is_alive() or is_unloading() methods. Since there is no obvious default (all combinations are used depending on call site), you have to explicitly set what iteration mode you want. >>>>>>>>>>> >>>>>>>>>>> Other than that, I make sure that the sweeper stays away from is_unloading() nmethods that are not yet is_unloaded(). To make the interactions between the sweeper and concurrent GC threads safe, I had to move down the store that sets the state to unloaded, and use a release_store there, to make sure no accesses float below it at compile-time. Once that store is observed, nmethods may be deleted. >>>>>>>>>>> >>>>>>>>>>> In the IC miss handler, I also need to lazily clean stale IC caches due to calling is_unloading nmethods using nmethod entry barriers. >>>>>>>>>>> >>>>>>>>>>> Webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213755/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Bug: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213755 >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> /Erik >>>>>>>>> >>>>>>>> >>>>>> >>>> From thomas.schatzl at oracle.com Wed Nov 21 22:00:47 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 21 Nov 2018 23:00:47 +0100 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> References: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> Message-ID: <7920f8a2a809086ace958012f6e2df60dc458a8b.camel@oracle.com> Hi, On Wed, 2018-11-21 at 13:05 -0800, Jiangli Zhou wrote: > Hi Thomas, > > Just some comments for your comments. :) > > On 11/21/18 12:29 PM, Thomas Schatzl wrote: > > Hi, > > > > On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: > > > Hi, > > > > > > Please review this fix to avoid leaving heap metadata in an > > > inconsistent state when CDS archive mapping fails. > > > > > > Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 > > > Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ > > > > the change seems good. I have two comments on this though: > > > > - would it be useful to clear both maps in > > G1ArchiveAllocator::clear_range_archive()? I mean in the failure > > case these regions can't be used for one purpose or the other as > > soon as we fail the mapping, and saves passing the bool flag. > > We map the 'closed' archive regions before 'open' archive regions. > The 'open' archive regions are only mapped if the 'closed' ones are > successfully mapped. > > The 'open' archive regions have dependency on the 'closed' regions > (objects in the open regions may have references to the closed > objects). > The 'closed' archive regions can be used independently however. If > 'open' archive regions fail to map, we can still keep the 'closed' > archives in use (no need to clear the closed map). G1ArchiveAllocator::clear_range_archive() does not clear the whole heap region range, just the given range which we failed to map the closed or open archive into as far as I understand. There can be no overlap, so clearing the range in both maps in that method should be fine. That's also why I suggested having a single map containing an enum value and not two maps with a bool each. But maybe I'm too tired already today, so feel free to ignore my suggestions :) Thanks, Thomas From jiangli.zhou at oracle.com Wed Nov 21 22:47:09 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 21 Nov 2018 14:47:09 -0800 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: <7920f8a2a809086ace958012f6e2df60dc458a8b.camel@oracle.com> References: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> <7920f8a2a809086ace958012f6e2df60dc458a8b.camel@oracle.com> Message-ID: <9fb4d0cc-5b8e-c45b-c6d6-1f92440bd563@oracle.com> Hi Thomas, On 11/21/18 2:00 PM, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-11-21 at 13:05 -0800, Jiangli Zhou wrote: >> Hi Thomas, >> >> Just some comments for your comments. :) >> >> On 11/21/18 12:29 PM, Thomas Schatzl wrote: >>> Hi, >>> >>> On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: >>>> Hi, >>>> >>>> Please review this fix to avoid leaving heap metadata in an >>>> inconsistent state when CDS archive mapping fails. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 >>>> Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ >>> the change seems good. I have two comments on this though: >>> >>> - would it be useful to clear both maps in >>> G1ArchiveAllocator::clear_range_archive()? I mean in the failure >>> case these regions can't be used for one purpose or the other as >>> soon as we fail the mapping, and saves passing the bool flag. >> We map the 'closed' archive regions before 'open' archive regions. >> The 'open' archive regions are only mapped if the 'closed' ones are >> successfully mapped. >> >> The 'open' archive regions have dependency on the 'closed' regions >> (objects in the open regions may have references to the closed >> objects). >> The 'closed' archive regions can be used independently however. If >> 'open' archive regions fail to map, we can still keep the 'closed' >> archives in use (no need to clear the closed map). > G1ArchiveAllocator::clear_range_archive() does not clear the whole heap > region range, just the given range which we failed to map the closed or > open archive into as far as I understand. There can be no overlap, so > clearing the range in both maps in that method should be fine. > > That's also why I suggested having a single map containing an enum > value and not two maps with a bool each. > > But maybe I'm too tired already today, so feel free to ignore my > suggestions :) That makes sense to me! If we use a single map for both 'open' and 'closed' archive regions, clearing is a 'safe' operation for a requested range since the heap archive ranges are guaranteed to not be overlapped. I'll create a RFE for your merged suggestion (if you haven't created one). Thanks! With the current separate maps, Stefan's fix that checks the boolean argument probably is slightly cleaner (reflecting to the G1ArchiveAllocator::set_range_archive() API)? Thanks! Jiangli > > Thanks, > Thomas > > From david.holmes at oracle.com Thu Nov 22 04:28:26 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Nov 2018 14:28:26 +1000 Subject: FYI: 32-bit builds are currently broken Message-ID: <3d663ad1-41a0-0c79-79be-cf7691151145@oracle.com> 32-bit builds are broken by JDK-8213419. Filed: JDK-8214206 David From goetz.lindenmaier at sap.com Thu Nov 22 08:51:49 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 22 Nov 2018 08:51:49 +0000 Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) In-Reply-To: References: Message-ID: <5784a2dbe5944f07b1feb03b7b1f87e7@sap.com> Hi, Doesn't this have to be posted to jdk8u-dev? I had a look at the backport. Including 7127191 confused me a bit. Is it good to hide the fact that this was backported in the repository? In os_linux one fix is missing, is this on purpose? I don't think this is a critical issue, though, so leaving it out is fine. > the dropping of the changes to ... > src/share/vm/runtime/task.cpp and > src/os/windows/vm/attachListener_windows.cpp These changes are included in the webrev ...? The webrev looks good to me. Best regards, Goetz. > -----Original Message----- > From: hotspot-dev On Behalf Of > Andrew Hughes > Sent: Mittwoch, 21. November 2018 07:45 > To: serviceability-dev ; hotspot-dev > > Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) > > Bug: https://bugs.openjdk.java.net/browse/JDK-8140482 > Original changeset: > https://hg.openjdk.java.net/jdk-updates/jdk9u/hotspot/rev/cd86b5699825 > Webrev: > https://cr.openjdk.java.net/~andrew/openjdk8/8140482/webrev.01/ > > The patch largely applies as is, with some adjustment for context and > the dropping of the changes to src/cpu/x86/vm/stubRoutines_x86.cpp, > src/share/vm/runtime/task.cpp and > src/os/windows/vm/attachListener_windows.cpp > which don't exist in 8u. A clean backport of 7127191 is included, which > allows the changes to agent/src/os/linux/libproc_impl.c to apply as-is. > > Applying the change to 8u improves the code quality there and aids > in backporting other changes, such as 8210836 [0]. > > Ok for 8u? > > [0] https://mail.openjdk.java.net/pipermail/serviceability-dev/2018- > November/025991.html > > Thanks, > -- > Andrew :) > > Senior Free Java Software Engineer > Red Hat, Inc. (http://www.redhat.com) > > Web Site: http://fuseyism.com > Twitter: https://twitter.com/gnu_andrew_java > PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) > Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From dalibor.topic at oracle.com Thu Nov 22 10:51:13 2018 From: dalibor.topic at oracle.com (dalibor topic) Date: Thu, 22 Nov 2018 11:51:13 +0100 Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) In-Reply-To: <5784a2dbe5944f07b1feb03b7b1f87e7@sap.com> References: <5784a2dbe5944f07b1feb03b7b1f87e7@sap.com> Message-ID: <14bc209e-647a-ba99-7ed1-ceead64f5bb4@oracle.com> On 22.11.2018 09:51, Lindenmaier, Goetz wrote: > Hi, > > Doesn't this have to be posted to jdk8u-dev? The approval requests need to go to jdk8u-dev. The reviews can happen on the appropriate list, which may or may not be jd8u-dev - it typically is the list where the initial change was discussed. cheers, dalibor topic > I had a look at the backport. > Including 7127191 confused me a bit. Is it good to hide the fact that > this was backported in the repository? > In os_linux one fix is missing, is this on purpose? I don't think this is a > critical issue, though, so leaving it out is fine. > >> the dropping of the changes to ... >> src/share/vm/runtime/task.cpp and >> src/os/windows/vm/attachListener_windows.cpp > These changes are included in the webrev ...? > > The webrev looks good to me. > > Best regards, > Goetz. > > > > > >> -----Original Message----- >> From: hotspot-dev On Behalf Of >> Andrew Hughes >> Sent: Mittwoch, 21. November 2018 07:45 >> To: serviceability-dev ; hotspot-dev >> >> Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8140482 >> Original changeset: >> https://hg.openjdk.java.net/jdk-updates/jdk9u/hotspot/rev/cd86b5699825 >> Webrev: >> https://cr.openjdk.java.net/~andrew/openjdk8/8140482/webrev.01/ >> >> The patch largely applies as is, with some adjustment for context and >> the dropping of the changes to src/cpu/x86/vm/stubRoutines_x86.cpp, >> src/share/vm/runtime/task.cpp and >> src/os/windows/vm/attachListener_windows.cpp >> which don't exist in 8u. A clean backport of 7127191 is included, which >> allows the changes to agent/src/os/linux/libproc_impl.c to apply as-is. >> >> Applying the change to 8u improves the code quality there and aids >> in backporting other changes, such as 8210836 [0]. >> >> Ok for 8u? >> >> [0] https://mail.openjdk.java.net/pipermail/serviceability-dev/2018- >> November/025991.html >> >> Thanks, >> -- >> Andrew :) >> >> Senior Free Java Software Engineer >> Red Hat, Inc. (http://www.redhat.com) >> >> Web Site: http://fuseyism.com >> Twitter: https://twitter.com/gnu_andrew_java >> PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) >> Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From shade at redhat.com Thu Nov 22 11:17:48 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 22 Nov 2018 12:17:48 +0100 Subject: FYI: 32-bit builds are currently broken In-Reply-To: <3d663ad1-41a0-0c79-79be-cf7691151145@oracle.com> References: <3d663ad1-41a0-0c79-79be-cf7691151145@oracle.com> Message-ID: <7e7ea198-5c83-9c18-bf17-4ccda65ce473@redhat.com> On 11/22/18 5:28 AM, David Holmes wrote: > 32-bit builds are broken by JDK-8213419. > Filed: JDK-8214206 Thanks! Are you working on a fix? Or anyone else is supposed to work on it? -Aleksey From goetz.lindenmaier at sap.com Thu Nov 22 11:34:21 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 22 Nov 2018 11:34:21 +0000 Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) In-Reply-To: <14bc209e-647a-ba99-7ed1-ceead64f5bb4@oracle.com> References: <5784a2dbe5944f07b1feb03b7b1f87e7@sap.com> <14bc209e-647a-ba99-7ed1-ceead64f5bb4@oracle.com> Message-ID: <7308f463711045aa9ce4a61818f17f7d@sap.com> Hi Dalibor, thanks for the info! Best regards, Goetz. > -----Original Message----- > From: dalibor topic > Sent: Donnerstag, 22. November 2018 11:51 > To: Lindenmaier, Goetz ; Andrew Hughes > ; serviceability-dev dev at openjdk.java.net>; hotspot-dev > Subject: Re: [8u] [RFR] 8140482: Various minor code improvements (runtime) > > > > On 22.11.2018 09:51, Lindenmaier, Goetz wrote: > > Hi, > > > > Doesn't this have to be posted to jdk8u-dev? > > The approval requests need to go to jdk8u-dev. The reviews can happen on > the appropriate list, which may or may not be jd8u-dev - it typically is > the list where the initial change was discussed. > > cheers, > dalibor topic > > > I had a look at the backport. > > Including 7127191 confused me a bit. Is it good to hide the fact that > > this was backported in the repository? > > In os_linux one fix is missing, is this on purpose? I don't think this is a > > critical issue, though, so leaving it out is fine. > > > >> the dropping of the changes to ... > >> src/share/vm/runtime/task.cpp and > >> src/os/windows/vm/attachListener_windows.cpp > > These changes are included in the webrev ...? > > > > The webrev looks good to me. > > > > Best regards, > > Goetz. > > > > > > > > > > > >> -----Original Message----- > >> From: hotspot-dev On Behalf > Of > >> Andrew Hughes > >> Sent: Mittwoch, 21. November 2018 07:45 > >> To: serviceability-dev ; hotspot-dev > >> > >> Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) > >> > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8140482 > >> Original changeset: > >> https://hg.openjdk.java.net/jdk- > updates/jdk9u/hotspot/rev/cd86b5699825 > >> Webrev: > >> https://cr.openjdk.java.net/~andrew/openjdk8/8140482/webrev.01/ > >> > >> The patch largely applies as is, with some adjustment for context and > >> the dropping of the changes to src/cpu/x86/vm/stubRoutines_x86.cpp, > >> src/share/vm/runtime/task.cpp and > >> src/os/windows/vm/attachListener_windows.cpp > >> which don't exist in 8u. A clean backport of 7127191 is included, which > >> allows the changes to agent/src/os/linux/libproc_impl.c to apply as-is. > >> > >> Applying the change to 8u improves the code quality there and aids > >> in backporting other changes, such as 8210836 [0]. > >> > >> Ok for 8u? > >> > >> [0] https://mail.openjdk.java.net/pipermail/serviceability-dev/2018- > >> November/025991.html > >> > >> Thanks, > >> -- > >> Andrew :) > >> > >> Senior Free Java Software Engineer > >> Red Hat, Inc. (http://www.redhat.com) > >> > >> Web Site: http://fuseyism.com > >> Twitter: https://twitter.com/gnu_andrew_java > >> PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) > >> Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 > > -- > Dalibor Topic | Principal Product Manager > Phone: +494089091214 | Mobile: +491737185961 > > > ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg > > ORACLE Deutschland B.V. & Co. KG > Hauptverwaltung: Riesstr. 25, D-80992 M?nchen > Registergericht: Amtsgericht M?nchen, HRA 95603 > > Komplement?rin: ORACLE Deutschland Verwaltung B.V. > Hertogswetering 163/167, 3543 AS Utrecht, Niederlande > Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 > Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher > > Oracle is committed to developing > practices and products that help protect the environment From dalibor.topic at oracle.com Thu Nov 22 11:40:08 2018 From: dalibor.topic at oracle.com (dalibor topic) Date: Thu, 22 Nov 2018 12:40:08 +0100 Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) In-Reply-To: <7308f463711045aa9ce4a61818f17f7d@sap.com> References: <5784a2dbe5944f07b1feb03b7b1f87e7@sap.com> <14bc209e-647a-ba99-7ed1-ceead64f5bb4@oracle.com> <7308f463711045aa9ce4a61818f17f7d@sap.com> Message-ID: The reason we picked that model for JDK 8 Updates at the time was to have a central place to document and discuss decisions about which changes get approved into 8u (jdk8u-dev, in this case), without requiring the developers most likely to be able to review backports to sign up for another mailing list where they wouldn't (need to) care about most of the on-list review & approval traffic. cheers, dalibor topic On 22.11.2018 12:34, Lindenmaier, Goetz wrote: > Hi Dalibor, > > thanks for the info! > > Best regards, > Goetz. > >> -----Original Message----- >> From: dalibor topic >> Sent: Donnerstag, 22. November 2018 11:51 >> To: Lindenmaier, Goetz ; Andrew Hughes >> ; serviceability-dev > dev at openjdk.java.net>; hotspot-dev >> Subject: Re: [8u] [RFR] 8140482: Various minor code improvements (runtime) >> >> >> >> On 22.11.2018 09:51, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> Doesn't this have to be posted to jdk8u-dev? >> >> The approval requests need to go to jdk8u-dev. The reviews can happen on >> the appropriate list, which may or may not be jd8u-dev - it typically is >> the list where the initial change was discussed. >> >> cheers, >> dalibor topic >> >>> I had a look at the backport. >>> Including 7127191 confused me a bit. Is it good to hide the fact that >>> this was backported in the repository? >>> In os_linux one fix is missing, is this on purpose? I don't think this is a >>> critical issue, though, so leaving it out is fine. >>> >>>> the dropping of the changes to ... >>>> src/share/vm/runtime/task.cpp and >>>> src/os/windows/vm/attachListener_windows.cpp >>> These changes are included in the webrev ...? >>> >>> The webrev looks good to me. >>> >>> Best regards, >>> Goetz. >>> >>> >>> >>> >>> >>>> -----Original Message----- >>>> From: hotspot-dev On Behalf >> Of >>>> Andrew Hughes >>>> Sent: Mittwoch, 21. November 2018 07:45 >>>> To: serviceability-dev ; hotspot-dev >>>> >>>> Subject: [8u] [RFR] 8140482: Various minor code improvements (runtime) >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8140482 >>>> Original changeset: >>>> https://hg.openjdk.java.net/jdk- >> updates/jdk9u/hotspot/rev/cd86b5699825 >>>> Webrev: >>>> https://cr.openjdk.java.net/~andrew/openjdk8/8140482/webrev.01/ >>>> >>>> The patch largely applies as is, with some adjustment for context and >>>> the dropping of the changes to src/cpu/x86/vm/stubRoutines_x86.cpp, >>>> src/share/vm/runtime/task.cpp and >>>> src/os/windows/vm/attachListener_windows.cpp >>>> which don't exist in 8u. A clean backport of 7127191 is included, which >>>> allows the changes to agent/src/os/linux/libproc_impl.c to apply as-is. >>>> >>>> Applying the change to 8u improves the code quality there and aids >>>> in backporting other changes, such as 8210836 [0]. >>>> >>>> Ok for 8u? >>>> >>>> [0] https://mail.openjdk.java.net/pipermail/serviceability-dev/2018- >>>> November/025991.html >>>> >>>> Thanks, >>>> -- >>>> Andrew :) >>>> >>>> Senior Free Java Software Engineer >>>> Red Hat, Inc. (http://www.redhat.com) >>>> >>>> Web Site: http://fuseyism.com >>>> Twitter: https://twitter.com/gnu_andrew_java >>>> PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) >>>> Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 >> >> -- >> Dalibor Topic | Principal Product Manager >> Phone: +494089091214 | Mobile: +491737185961 >> >> >> ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg >> >> ORACLE Deutschland B.V. & Co. KG >> Hauptverwaltung: Riesstr. 25, D-80992 M?nchen >> Registergericht: Amtsgericht M?nchen, HRA 95603 >> >> Komplement?rin: ORACLE Deutschland Verwaltung B.V. >> Hertogswetering 163/167, 3543 AS Utrecht, Niederlande >> Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 >> Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher >> >> Oracle is committed to developing >> practices and products that help protect the environment -- Dalibor Topic | Principal Product Manager Phone: +494089091214 | Mobile: +491737185961 ORACLE Deutschland B.V. & Co. KG | K?hneh?fe 5 | 22761 Hamburg ORACLE Deutschland B.V. & Co. KG Hauptverwaltung: Riesstr. 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From thomas.schatzl at oracle.com Thu Nov 22 11:48:11 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 22 Nov 2018 12:48:11 +0100 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: Message-ID: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> Hi Amith, welcome to the OpenJDK community! :) On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: > Hi all, > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, > S1 and S2 region (if any other ) memory areas on requested Numa nodes > and it should not configure itself to access other Numa nodes. This > issue is observed only when Java is allowed to run on fewer NUMA > nodes than available on the system with numactl membind and > interleave options. Running on all the nodes does not have any > effect. This will cause some applications (objects residing in old > gen and survivor region) to run slower on system with large Numa > nodes. > [... long explanation...] Is it possible to summarize the problem and the changes with the following few lines: "NUMA interleaving of memory of old gen and survivor spaces (for parallel GC) tells the OS to interleave memory across all nodes of a NUMA system. However the VM process may be configured to be limited to run only on a few nodes, which means that large parts of the heap will be located on foreign nodes. This can incurs a large performance penalty. The proposed solution is to tell the OS to interleave memory only across available nodes when enabling NUMA." We have had trouble understanding the problem statement and purpose of this patch when triaging (making sure the issue is understandable and can be worked on) as the text is rather technical. Having an easily understandable text also helps reviewers a lot. Assuming my summary is appropriate, I have several other unrelated questions: - could you propose a better subject for this work? "JDK NUMA Interleaving issue" seems very generic. Something like "NUMA heap allocation does not respect VM membind/interleave settings" maybe? - there have been other NUMA related patches from AMD recently, in particular JDK-what is the relation to JDK-8205051? The text there reads awfully similar to this one, but I *think* JDK-8205051 is actually about making sure that the parallel gc eden is not put on inactive nodes. Can you confirm this (talk to your colleague) so that we can fix the description too? - fyi, we are currently working on NUMA aware memory allocation support for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to not step on each other's toes (see below). [... broken patch...] I tried to apply the patch to the jdk/jdk tree, which failed; I then started to manually apply it but stopped after not being able to find the context of some hunks. I do not think this change applies to the latest source tree. Please make sure that the patch applies to the latest jdk/jdk tree with errors. All changes generally must first go into the latest dev tree before you can apply for backporting. Could you please send the patch as attachment (not copy&pasted) to this list and cc me? Then I can create a webrev out of it. I did look a bit over the patch as much as I could (it's very hard trying to review a diff), some comments: - the _numa_interleave_memory_v2 function pointer is never assigned to in the patch in the CR, so it will not be used. Please make sure the patch is complete. Actually it is never defined anywhere, ie. the patch unlikely actually compiles even if I could apply it somewhere. Please avoid frustrating reviewers by sending incomplete patches. - I am not sure set_numa_interleave, i.e. the interleaving, should be done without UseNUMAInterleaving enabled. Some collectors may do their own interleaving in the future (JDK-8210473 / JEP-345) so this would massively interfere in how they work. (That issue may be because I am looking at a potentially incomplete diff, so forgive me if the patch already handles this). - if some of the actions (interleaving/membind) fail, and it had been requested, it would be nice to print a log message. Actually it would be nice to print information about e.g. the bitmask anyway in the log so that the user can easily verify that what he specified on the command line has been properly picked up by the VM - instead of looking through the proc filesystem. Thanks, Thomas From david.holmes at oracle.com Thu Nov 22 12:06:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Nov 2018 22:06:01 +1000 Subject: FYI: 32-bit builds are currently broken In-Reply-To: <7e7ea198-5c83-9c18-bf17-4ccda65ce473@redhat.com> References: <3d663ad1-41a0-0c79-79be-cf7691151145@oracle.com> <7e7ea198-5c83-9c18-bf17-4ccda65ce473@redhat.com> Message-ID: <88bb56e9-392f-5dc9-142d-215337c29ac2@oracle.com> On 22/11/2018 9:17 pm, Aleksey Shipilev wrote: > On 11/22/18 5:28 AM, David Holmes wrote: >> 32-bit builds are broken by JDK-8213419. >> Filed: JDK-8214206 > > Thanks! Are you working on a fix? Or anyone else is supposed to work on it? I'm hoping Roland will be able to pick it up and fix it. If it's not fixed overnight (for me) (which given the holiday situation in the US seems a strong possibility) I'll look at backing out the change that broke things. David > -Aleksey > From erik.osterlund at oracle.com Thu Nov 22 13:59:29 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 22 Nov 2018 14:59:29 +0100 Subject: RFR: 8214231: Allow concurrent cleaning of TypeStackSlotEntries and ReturnTypeEntry Message-ID: <5BF6B641.60302@oracle.com> Hi, The TypeStackSlotEntries and ReturnTypeEntry contain weak metadata links that must be cleaned during class unloading. With concurrent class unloading, their ci counterparts must be protected from accessing unloading metadata. In particular, ciTypeStackSlotEntries and ciReturnTypeEntry counterparts copy the raw data from TypeStackSlotEntries and ReturnTypeEntry of the MDOs, and are subsequently queried by the compiler. The copied data may contain stale metadata. By checking for unloading metadata and replacing it with NULL during creation of the ci variants from their raw counterparts, we protect the compiler from seeing stale metadata that is being unloaded, before the GC has gotten around to cleaning up the MDOs. Webrev: http://cr.openjdk.java.net/~eosterlund/8214231/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8214231 Thanks, /Erik From rwestrel at redhat.com Thu Nov 22 14:14:28 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 22 Nov 2018 15:14:28 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit Message-ID: <878t1lp5bv.fsf@redhat.com> http://cr.openjdk.java.net/~roland/8214206/webrev.00/ The log2_intptr(int x) and log2_intptr(intptr_t x) conflict on 32 bit. Roland. From goetz.lindenmaier at sap.com Thu Nov 22 14:34:08 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 22 Nov 2018 14:34:08 +0000 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <878t1lp5bv.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> Message-ID: <80d75e3ac69745d8a3b1062de255a779@sap.com> Hi, isn't this wrong, too: inline int log2_long(jlong x) { return log2_long((unsigned long)x); } It casts the 64-bit x to 32 bit on 32 bit architectures. Further, we also see a build issue on s390x: ...jdk/src/hotspot/cpu/s390/c1_LIRAssembler_s390.cpp: In member function 'void LIR_Assembler::arithmetic_idiv(LIR_Code, LIR_Opr, LIR_Opr, LIR_Opr, LIR_Opr, CodeEmitInfo*)': ...jdk/src/hotspot/cpu/s390/c1_LIRAssembler_s390.cpp:1884:51: error: call of overloaded 'log2_long(jint)' is ambiguous jlong log_divisor = log2_long(right->as_jint()); ^ In file included from ...jdk/src/hotspot/share/utilities/align.hpp:28:0, from ...jdk /src/hotspot/share/runtime/globals.hpp:29, from ...jdk /hotspot/share/memory/allocation.hpp:28, from ...jdk /src/hotspot/share/classfile/classLoaderData.hpp:28, from ...jdk /src/hotspot/share/precompiled/precompiled.hpp:34: .../jdk/src/hotspot/share/utilities/globalDefinitions.hpp:1051:12: note: candidate: int log2_long(long unsigned int) inline int log2_long(unsigned long x) { ^~~~~~~~~ .../jdk/src/hotspot/share/utilities/globalDefinitions.hpp:1075:12: note: candidate: int log2_long(jlong) inline int log2_long(jlong x) { ^~~~~~~~~ Best regards, Goetz. > -----Original Message----- > From: hotspot-dev On Behalf Of > Roland Westrelin > Sent: Donnerstag, 22. November 2018 15:14 > To: hotspot-dev Source Developers > Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit > > > http://cr.openjdk.java.net/~roland/8214206/webrev.00/ > > The log2_intptr(int x) and log2_intptr(intptr_t x) conflict on 32 bit. > > Roland. From rwestrel at redhat.com Thu Nov 22 14:50:47 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 22 Nov 2018 15:50:47 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <80d75e3ac69745d8a3b1062de255a779@sap.com> References: <878t1lp5bv.fsf@redhat.com> <80d75e3ac69745d8a3b1062de255a779@sap.com> Message-ID: <87zhu1np2w.fsf@redhat.com> Hi Goetz, > isn't this wrong, too: > > inline int log2_long(jlong x) { > return log2_long((unsigned long)x); > } > > It casts the 64-bit x to 32 bit on 32 bit architectures. Yes, it's wrong. It also causes JDK-8214189. Which is the bug I intend to use it to fix it. Roland. From shade at redhat.com Thu Nov 22 15:00:28 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 22 Nov 2018 16:00:28 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <878t1lp5bv.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> Message-ID: <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> On 11/22/18 3:14 PM, Roland Westrelin wrote: > http://cr.openjdk.java.net/~roland/8214206/webrev.00/ > > The log2_intptr(int x) and log2_intptr(intptr_t x) conflict on 32 bit. I don't think conditional compilation is the answer here. It seems better to have declarations that are guaranteed not to clash. For example: http://cr.openjdk.java.net/~shade/8214206/webrev.00/ This builds on x86_32 and x86_64 without problems. STATIC_ASSERTS help to find the jlong downcast issue Goetz mentioned in his note, and I believe the answer is "unsigned long long". Roland, feel free to use this patch either partially or in full. -Aleksey From martin.doerr at sap.com Thu Nov 22 15:21:40 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 22 Nov 2018 15:21:40 +0000 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> Message-ID: <88265b9b7e864ee8936398abedd9e2ef@sap.com> +1 I guess we only need to use log2_jint in c1_LIRGenerator_s390.cpp to fix build on your new companies most expensive platform ?? Best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of Aleksey Shipilev Sent: Donnerstag, 22. November 2018 16:00 To: Roland Westrelin ; hotspot-dev Source Developers Subject: Re: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit On 11/22/18 3:14 PM, Roland Westrelin wrote: > http://cr.openjdk.java.net/~roland/8214206/webrev.00/ > > The log2_intptr(int x) and log2_intptr(intptr_t x) conflict on 32 bit. I don't think conditional compilation is the answer here. It seems better to have declarations that are guaranteed not to clash. For example: http://cr.openjdk.java.net/~shade/8214206/webrev.00/ This builds on x86_32 and x86_64 without problems. STATIC_ASSERTS help to find the jlong downcast issue Goetz mentioned in his note, and I believe the answer is "unsigned long long". Roland, feel free to use this patch either partially or in full. -Aleksey From aph at redhat.com Thu Nov 22 16:06:53 2018 From: aph at redhat.com (Andrew Haley) Date: Thu, 22 Nov 2018 16:06:53 +0000 Subject: Time to Safepoint and HotSpot intrinsics Message-ID: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> In several places in HotSpot we have intrinsics that can block for a long time. The most extreme cases of this are some crypto accelerators that are only bounded by the maximum size of a byte array, 2 gigabytes. This leads to extended time to safepoint, where every thread has to wait for the intrinsic to complete. It can be on the order of seconds. We could simply limit the maximum buffer size of the intrinsic and wrap it in a Java loop with (as usual) a safepoint check. This does not noticeably slow anything down, but it does solve the TTSP problem. This can be important when multi-megabyte messages are being encrypted. The question is, then, what is the maximum tolerable time that an intrinsic should block? I suggest that it should be on the order of a millisecond, particularly when garbage collectors are trying to drive down the safepoint time. This corresponds to AES/GCM encryption of about 128k - 256k batches on a decently fast 64-bit processor. Doing encryption in batches like this doesn't significantly slow down anything. It is, in a sense, like loop strip mining for primitives. So, I'm proposing two things: firstly we fix the intrinsics that can block for extended periods, and we then declare that no new intrinsic code will be accepted without the worst case safepoint time being measured and found to be acceptable. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rwestrel at redhat.com Thu Nov 22 16:20:19 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 22 Nov 2018 17:20:19 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> Message-ID: <87tvk9nkxo.fsf@redhat.com> > http://cr.openjdk.java.net/~shade/8214206/webrev.00/ It's better indeed. But isn't it julong instead of unsigned long long? Also the assert(x >= 0, "sanity") would fire for min int and min long. Roland. From goetz.lindenmaier at sap.com Thu Nov 22 16:37:10 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 22 Nov 2018 16:37:10 +0000 Subject: RFR(S): downport 8211326: add OS user related information to hs_err file Message-ID: Hi, I would like to downport 8211326. It helps us maintain systems that are (and most will be) running on jdk11. Unfortunately, in posix.hpp the context changed slightly, and thus the change does not apply. So I need a review. Webrev against jdk11u: http://cr.openjdk.java.net/~goetz/wr18/8211326-hs_err-user-jdk11u/01/ Original webrev: http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.4/ Best regards, Goetz. From rwestrel at redhat.com Thu Nov 22 16:53:33 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 22 Nov 2018 17:53:33 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <87tvk9nkxo.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> Message-ID: <87o9ahnjea.fsf@redhat.com> Based on Aleksey's patch: http://cr.openjdk.java.net/~roland/8214206/webrev.01/ This builds on all platform I can try. I pushed it through the submit repo but it seems it's currently unstable so it's quite likely to produce no useful result. Roland. From martin.doerr at sap.com Thu Nov 22 17:43:08 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 22 Nov 2018 17:43:08 +0000 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <87o9ahnjea.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> Message-ID: Hi Roland, please remove log2_intptr(intptr_t x). The sign conversion can be done implicitly. This fixes "call of overloaded 'log2_intptr(int)' is ambiguous". Best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of Roland Westrelin Sent: Donnerstag, 22. November 2018 17:54 To: Aleksey Shipilev ; hotspot-dev Source Developers Subject: Re: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit Based on Aleksey's patch: http://cr.openjdk.java.net/~roland/8214206/webrev.01/ This builds on all platform I can try. I pushed it through the submit repo but it seems it's currently unstable so it's quite likely to produce no useful result. Roland. From shade at redhat.com Thu Nov 22 17:47:13 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 22 Nov 2018 18:47:13 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <87o9ahnjea.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> Message-ID: <0effbdf1-e100-7526-755d-535d760e8e7b@redhat.com> On 11/22/18 5:53 PM, Roland Westrelin wrote: > Based on Aleksey's patch: > > http://cr.openjdk.java.net/~roland/8214206/webrev.01/ Looks good. I checked x86_64, x86_32, aarch64, armhf, s390x, ppc64el. You need this addition for ppc64el and s390x: diff -r c6a329a473b3 src/hotspot/cpu/ppc/c1_LIRGenerator_ppc.cpp --- a/src/hotspot/cpu/ppc/c1_LIRGenerator_ppc.cpp Thu Nov 22 18:18:58 2018 +0100 +++ b/src/hotspot/cpu/ppc/c1_LIRGenerator_ppc.cpp Thu Nov 22 18:46:47 2018 +0100 @@ -292,11 +292,11 @@ bool LIRGenerator::strength_reduce_multiply(LIR_Opr left, int c, LIR_Opr result, LIR_Opr tmp) { assert(left != result, "should be different registers"); if (is_power_of_2(c + 1)) { - __ shift_left(left, log2_intptr(c + 1), result); + __ shift_left(left, log2_int(c + 1), result); __ sub(result, left, result); return true; } else if (is_power_of_2(c - 1)) { - __ shift_left(left, log2_intptr(c - 1), result); + __ shift_left(left, log2_int(c - 1), result); __ add(result, left, result); return true; } diff -r c6a329a473b3 src/hotspot/cpu/s390/c1_LIRGenerator_s390.cpp --- a/src/hotspot/cpu/s390/c1_LIRGenerator_s390.cpp Thu Nov 22 18:18:58 2018 +0100 +++ b/src/hotspot/cpu/s390/c1_LIRGenerator_s390.cpp Thu Nov 22 18:46:47 2018 +0100 @@ -227,12 +227,12 @@ if (tmp->is_valid()) { if (is_power_of_2(c + 1)) { __ move(left, tmp); - __ shift_left(left, log2_intptr(c + 1), left); + __ shift_left(left, log2_int(c + 1), left); __ sub(left, tmp, result); return true; } else if (is_power_of_2(c - 1)) { __ move(left, tmp); - __ shift_left(left, log2_intptr(c - 1), left); + __ shift_left(left, log2_int(c - 1), left); __ add(left, tmp, result); return true; } Thanks, -Aleksey From rwestrel at redhat.com Thu Nov 22 19:53:00 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 22 Nov 2018 20:53:00 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <0effbdf1-e100-7526-755d-535d760e8e7b@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <0effbdf1-e100-7526-755d-535d760e8e7b@redhat.com> Message-ID: <877eh4zy77.fsf@redhat.com> Here is a new webrev with Aleksey's fixes and some sparc fixes as well. http://cr.openjdk.java.net/~roland/8214206/webrev.02/ Roland. From rwestrel at redhat.com Thu Nov 22 19:54:22 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 22 Nov 2018 20:54:22 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> Message-ID: <874lc8zy4x.fsf@redhat.com> > please remove log2_intptr(intptr_t x). The sign conversion can be done implicitly. > This fixes "call of overloaded 'log2_intptr(int)' is ambiguous". Aleksey's fixes to s390 are not sufficient (see latest webrev)? Roland. From david.holmes at oracle.com Thu Nov 22 21:30:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Nov 2018 07:30:17 +1000 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <877eh4zy77.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <0effbdf1-e100-7526-755d-535d760e8e7b@redhat.com> <877eh4zy77.fsf@redhat.com> Message-ID: On 23/11/2018 5:53 am, Roland Westrelin wrote: > > Here is a new webrev with Aleksey's fixes and some sparc fixes as well. > > http://cr.openjdk.java.net/~roland/8214206/webrev.02/ 1049 //* largest i such that 2^i <= x 1050 // A negative value of 'x' will return '63' 1051 inline int log2_long(julong x) { How can x be negative here? The name is also wrong but that's pre-existing :( Thanks, David > Roland. > From david.holmes at oracle.com Thu Nov 22 22:06:26 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Nov 2018 08:06:26 +1000 Subject: RFR(S): downport 8211326: add OS user related information to hs_err file In-Reply-To: References: Message-ID: <82f78262-17b8-ae00-d6d9-767f1362054a@oracle.com> Hi Goetz, Not sure why jdk8u-dev is cc'd here?? This depends on the backport of JDK-8211845, but Thomas has withdrawn that backport request. That process needs to be reinstated. Otherwise the actual backport looks accurate. Thanks, David On 23/11/2018 2:37 am, Lindenmaier, Goetz wrote: > Hi, > > I would like to downport 8211326. It helps us maintain systems > that are (and most will be) running on jdk11. > > Unfortunately, in posix.hpp the context changed slightly, and thus > the change does not apply. So I need a review. > Webrev against jdk11u: > http://cr.openjdk.java.net/~goetz/wr18/8211326-hs_err-user-jdk11u/01/ > Original webrev: > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.4/ > > Best regards, > Goetz. > From thomas.stuefe at gmail.com Thu Nov 22 22:16:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 22 Nov 2018 23:16:10 +0100 Subject: RFR(S): downport 8211326: add OS user related information to hs_err file In-Reply-To: <82f78262-17b8-ae00-d6d9-767f1362054a@oracle.com> References: <82f78262-17b8-ae00-d6d9-767f1362054a@oracle.com> Message-ID: Well, now that we have an actual use case for the ExtensiveErrorReport flag in 11u I will redraft the backport request. Not before next week though. Cheers, thomas On Thu 22. Nov 2018 at 23:06, David Holmes wrote: > Hi Goetz, > > Not sure why jdk8u-dev is cc'd here?? > > This depends on the backport of JDK-8211845, but Thomas has withdrawn > that backport request. That process needs to be reinstated. > > Otherwise the actual backport looks accurate. > > Thanks, > David > > On 23/11/2018 2:37 am, Lindenmaier, Goetz wrote: > > Hi, > > > > I would like to downport 8211326. It helps us maintain systems > > that are (and most will be) running on jdk11. > > > > Unfortunately, in posix.hpp the context changed slightly, and thus > > the change does not apply. So I need a review. > > Webrev against jdk11u: > > http://cr.openjdk.java.net/~goetz/wr18/8211326-hs_err-user-jdk11u/01/ > > Original webrev: > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.4/ > > > > Best regards, > > Goetz. > > > From rwestrel at redhat.com Fri Nov 23 08:16:44 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 23 Nov 2018 09:16:44 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <0effbdf1-e100-7526-755d-535d760e8e7b@redhat.com> <877eh4zy77.fsf@redhat.com> Message-ID: <87in0onr83.fsf@redhat.com> > 1049 //* largest i such that 2^i <= x > 1050 // A negative value of 'x' will return '63' > 1051 inline int log2_long(julong x) { > > How can x be negative here? It can't. The comment is wrong and should have been fixed by 8213419. Roland. From martin.doerr at sap.com Fri Nov 23 09:14:52 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 23 Nov 2018 09:14:52 +0000 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <874lc8zy4x.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <874lc8zy4x.fsf@redhat.com> Message-ID: <61afcbb6efff4f2fa4be6fb93e2101c7@sap.com> Hi Roland, I had already taken http://cr.openjdk.java.net/~roland/8214206/webrev.01/hs.patch when I got the error. The removal of inline int log2_intptr(intptr_t x) fixes it. I don't see any reason why it should exist. Best regards, Martin -----Original Message----- From: Roland Westrelin Sent: Donnerstag, 22. November 2018 20:54 To: Doerr, Martin ; Aleksey Shipilev ; hotspot-dev Source Developers Subject: RE: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit > please remove log2_intptr(intptr_t x). The sign conversion can be done implicitly. > This fixes "call of overloaded 'log2_intptr(int)' is ambiguous". Aleksey's fixes to s390 are not sufficient (see latest webrev)? Roland. From martin.doerr at sap.com Fri Nov 23 09:31:04 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 23 Nov 2018 09:31:04 +0000 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <61afcbb6efff4f2fa4be6fb93e2101c7@sap.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <874lc8zy4x.fsf@redhat.com> <61afcbb6efff4f2fa4be6fb93e2101c7@sap.com> Message-ID: <988d7ab2ecc34421a9082095379b213f@sap.com> Hi Roland, webrev.02 seems to work. But I still wonder why we should have a version which only casts to unsigned. Thanks and best regards, Martin -----Original Message----- From: hotspot-dev On Behalf Of Doerr, Martin Sent: Freitag, 23. November 2018 10:15 To: Roland Westrelin ; Aleksey Shipilev ; hotspot-dev Source Developers Subject: RE: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit Hi Roland, I had already taken http://cr.openjdk.java.net/~roland/8214206/webrev.01/hs.patch when I got the error. The removal of inline int log2_intptr(intptr_t x) fixes it. I don't see any reason why it should exist. Best regards, Martin -----Original Message----- From: Roland Westrelin Sent: Donnerstag, 22. November 2018 20:54 To: Doerr, Martin ; Aleksey Shipilev ; hotspot-dev Source Developers Subject: RE: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit > please remove log2_intptr(intptr_t x). The sign conversion can be done implicitly. > This fixes "call of overloaded 'log2_intptr(int)' is ambiguous". Aleksey's fixes to s390 are not sufficient (see latest webrev)? Roland. From robbin.ehn at oracle.com Fri Nov 23 12:45:00 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 23 Nov 2018 13:45:00 +0100 Subject: RFR: 8214231: Allow concurrent cleaning of TypeStackSlotEntries and ReturnTypeEntry In-Reply-To: <5BF6B641.60302@oracle.com> References: <5BF6B641.60302@oracle.com> Message-ID: <0dee808b-dfee-500d-ec57-ef96ce314fa3@oracle.com> Hi Erik, This is a reasonable approach, thanks! /Robbin On 2018-11-22 14:59, Erik ?sterlund wrote: > Hi, > > The TypeStackSlotEntries and ReturnTypeEntry contain weak metadata links that > must be cleaned during class unloading. With concurrent class unloading, their > ci counterparts must be protected from accessing unloading metadata. > > In particular, ciTypeStackSlotEntries and ciReturnTypeEntry counterparts copy > the raw data from TypeStackSlotEntries and ReturnTypeEntry of the MDOs, and are > subsequently queried by the compiler. The copied data may contain stale > metadata. By checking for unloading metadata and replacing it with NULL during > creation of the ci variants from their raw counterparts, we protect the compiler > from seeing stale metadata that is being unloaded, before the GC has gotten > around to cleaning up the MDOs. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8214231/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8214231 > > Thanks, > /Erik From rwestrel at redhat.com Fri Nov 23 13:35:35 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 23 Nov 2018 14:35:35 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <988d7ab2ecc34421a9082095379b213f@sap.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <874lc8zy4x.fsf@redhat.com> <61afcbb6efff4f2fa4be6fb93e2101c7@sap.com> <988d7ab2ecc34421a9082095379b213f@sap.com> Message-ID: <87d0qvor14.fsf@redhat.com> Hi Martin, > webrev.02 seems to work. But I still wonder why we should have a version which only casts to unsigned. Thanks for giving it a try. Not sure if a signed + an unsigned versions are needed. But not relying on implicit casting seems safer. webrev.02 passes the submit repo. So unless someone objects I'll push it later today (with a fix for the misleading comment that David pointed out). Roland. From martin.doerr at sap.com Fri Nov 23 14:13:28 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 23 Nov 2018 14:13:28 +0000 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <87d0qvor14.fsf@redhat.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <874lc8zy4x.fsf@redhat.com> <61afcbb6efff4f2fa4be6fb93e2101c7@sap.com> <988d7ab2ecc34421a9082095379b213f@sap.com> <87d0qvor14.fsf@redhat.com> Message-ID: <972fbe636a16425d897470c7c7dee380@sap.com> Ship it. Thanks, Martin -----Original Message----- From: Roland Westrelin Sent: Freitag, 23. November 2018 14:36 To: Doerr, Martin ; Doerr, Martin ; Aleksey Shipilev ; hotspot-dev Source Developers Subject: RE: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit Hi Martin, > webrev.02 seems to work. But I still wonder why we should have a version which only casts to unsigned. Thanks for giving it a try. Not sure if a signed + an unsigned versions are needed. But not relying on implicit casting seems safer. webrev.02 passes the submit repo. So unless someone objects I'll push it later today (with a fix for the misleading comment that David pointed out). Roland. From stefan.johansson at oracle.com Fri Nov 23 15:05:27 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 23 Nov 2018 16:05:27 +0100 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: <9fb4d0cc-5b8e-c45b-c6d6-1f92440bd563@oracle.com> References: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> <7920f8a2a809086ace958012f6e2df60dc458a8b.camel@oracle.com> <9fb4d0cc-5b8e-c45b-c6d6-1f92440bd563@oracle.com> Message-ID: <1f1fc6e2-69f9-9c29-e0a1-164063be62a7@oracle.com> Thanks for the reviews Jiangli and Thomas, I pushed this without any changes after talking to Thomas offline. Cheers, Stefan On 2018-11-21 23:47, Jiangli Zhou wrote: > Hi Thomas, > > > On 11/21/18 2:00 PM, Thomas Schatzl wrote: >> Hi, >> >> On Wed, 2018-11-21 at 13:05 -0800, Jiangli Zhou wrote: >>> Hi Thomas, >>> >>> Just some comments for your comments. :) >>> >>> On 11/21/18 12:29 PM, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: >>>>> Hi, >>>>> >>>>> Please review this fix to avoid leaving heap metadata in an >>>>> inconsistent state when CDS archive mapping fails. >>>>> >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 >>>>> Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ >>>> ??? the change seems good. I have two comments on this though: >>>> >>>> - would it be useful to clear both maps in >>>> G1ArchiveAllocator::clear_range_archive()? I mean in the failure >>>> case these regions can't be used for one purpose or the other as >>>> soon as we fail the mapping, and saves passing the bool flag. >>> We map the 'closed' archive regions before 'open' archive regions. >>> The 'open' archive regions are only mapped if the 'closed' ones are >>> successfully mapped. >>> >>> The 'open' archive regions have dependency on the 'closed' regions >>> (objects in the open regions may have references to the closed >>> objects). >>> The 'closed' archive regions can be used independently however. If >>> 'open' archive regions fail to map, we can still keep the 'closed' >>> archives in use (no need to clear the closed map). >> G1ArchiveAllocator::clear_range_archive() does not clear the whole heap >> region range, just the given range which we failed to map the closed or >> open archive into as far as I understand. There can be no overlap, so >> clearing the range in both maps in that method should be fine. >> >> That's also why I suggested having a single map containing an enum >> value and not two maps with a bool each. >> >> But maybe I'm too tired already today, so feel free to ignore my >> suggestions :) > > That makes sense to me! If we use a single map for both 'open' and > 'closed' archive regions, clearing is a 'safe' operation for a requested > range since the heap archive ranges are guaranteed to not be overlapped. > I'll create a RFE for your merged suggestion (if you haven't created > one). Thanks! > > With the current separate maps, Stefan's fix that checks the boolean > argument probably is slightly cleaner (reflecting to the > G1ArchiveAllocator::set_range_archive() API)? > > Thanks! > Jiangli >> >> Thanks, >> ?? Thomas >> >> > From erik.osterlund at oracle.com Fri Nov 23 15:16:01 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 23 Nov 2018 16:16:01 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <5BF423D2.1070800@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> <5BF423D2.1070800@oracle.com> Message-ID: <5BF819B1.7040908@oracle.com> Hi, Here is an updated version with some minor adjustments based on feedback from Robbin Ehn. Incremental: http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00_01/ Full: http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01/ Thanks, /Erik On 2018-11-20 16:10, Erik ?sterlund wrote: > Ping. > > /Erik > > On 2018-11-12 23:02, Erik ?sterlund wrote: >> Hi, >> >> In my change 8209189, I moved a lot of code around relating to code >> cache unloading. Unfortunately, I failed to see that when making >> nmethods unloaded, and their dependents are flushed... they are not >> immediately removed. Instead, they are tagged to have stale entries, >> that are later cleaned during weak metadata cleaning of >> InstanceKlass. As for method handles, they leak instead, with some >> hacks to reduce the leaking by expunging entries while adding and >> removing. >> >> Obviously, with InstanceKlass cleanup and code cache unloading now >> possibly running in parallel after 8209189, things can go wrong. >> These two phases used >> to be separated by a "wait barrier" for the worker threads preventing >> that parallelism. >> >> The reason dependency contexts were not cleaned when they were found >> during code cache cleaning, is because it was not thread safe when code >> cache unloading was made parallel instead of serial. But now that we are >> implementing concurrent class unloading, we are going to need concurrent >> cleanup of dependency contexts anyway. Therefore I will propose a bug >> fix that fixes the problem in a way that works for both serial, >> parallel and concurrent class unloading. >> >> The solution is to during code cache unloading claim cleaning of >> encountered stale dependency contexts, and clean them straight away. >> Entries are unlinked in a lock-free fashion, and placed on a purge >> list that is walked and deleted during ClassLoaderDataGraph::purge(). >> This follows the thinking of first unlinking, then syncing all >> threads, and then purging. >> >> New accessors for the head and next links, hide is_unloading entries >> and helps unlinking them and putting them into the purge list. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8213565 >> >> Thanks, >> /Erik > From erik.osterlund at oracle.com Fri Nov 23 15:24:44 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 23 Nov 2018 16:24:44 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <43818dbd-5008-b3a0-7045-2d314e47f5fa@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> <5BF423D2.1070800@oracle.com> <5BF819B1.7040908@oracle.com> <43818dbd-5008-b3a0-7045-2d314e47f5fa@oracle.com> Message-ID: <5BF81BBC.2060606@oracle.com> Hi Robbin, Thanks for the review. /Erik On 2018-11-23 16:26, Robbin Ehn wrote: > Looks good, thanks! > > /Robbin > > On 2018-11-23 16:16, Erik ?sterlund wrote: >> Hi, >> >> Here is an updated version with some minor adjustments based on >> feedback from Robbin Ehn. >> >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00_01/ >> >> Full: >> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01/ >> >> Thanks, >> /Erik >> >> On 2018-11-20 16:10, Erik ?sterlund wrote: >>> Ping. >>> >>> /Erik >>> >>> On 2018-11-12 23:02, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> In my change 8209189, I moved a lot of code around relating to code >>>> cache unloading. Unfortunately, I failed to see that when making >>>> nmethods unloaded, and their dependents are flushed... they are not >>>> immediately removed. Instead, they are tagged to have stale >>>> entries, that are later cleaned during weak metadata cleaning of >>>> InstanceKlass. As for method handles, they leak instead, with some >>>> hacks to reduce the leaking by expunging entries while adding and >>>> removing. >>>> >>>> Obviously, with InstanceKlass cleanup and code cache unloading now >>>> possibly running in parallel after 8209189, things can go wrong. >>>> These two phases used >>>> to be separated by a "wait barrier" for the worker threads >>>> preventing that parallelism. >>>> >>>> The reason dependency contexts were not cleaned when they were found >>>> during code cache cleaning, is because it was not thread safe when >>>> code >>>> cache unloading was made parallel instead of serial. But now that >>>> we are >>>> implementing concurrent class unloading, we are going to need >>>> concurrent >>>> cleanup of dependency contexts anyway. Therefore I will propose a >>>> bug fix that fixes the problem in a way that works for both serial, >>>> parallel and concurrent class unloading. >>>> >>>> The solution is to during code cache unloading claim cleaning of >>>> encountered stale dependency contexts, and clean them straight away. >>>> Entries are unlinked in a lock-free fashion, and placed on a purge >>>> list that is walked and deleted during >>>> ClassLoaderDataGraph::purge(). This follows the thinking of first >>>> unlinking, then syncing all threads, and then purging. >>>> >>>> New accessors for the head and next links, hide is_unloading >>>> entries and helps unlinking them and putting them into the purge list. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8213565 >>>> >>>> Thanks, >>>> /Erik >>> >> From robbin.ehn at oracle.com Fri Nov 23 15:26:55 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 23 Nov 2018 16:26:55 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <5BF819B1.7040908@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> <5BF423D2.1070800@oracle.com> <5BF819B1.7040908@oracle.com> Message-ID: <43818dbd-5008-b3a0-7045-2d314e47f5fa@oracle.com> Looks good, thanks! /Robbin On 2018-11-23 16:16, Erik ?sterlund wrote: > Hi, > > Here is an updated version with some minor adjustments based on feedback from > Robbin Ehn. > > Incremental: > http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00_01/ > > Full: > http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01/ > > Thanks, > /Erik > > On 2018-11-20 16:10, Erik ?sterlund wrote: >> Ping. >> >> /Erik >> >> On 2018-11-12 23:02, Erik ?sterlund wrote: >>> Hi, >>> >>> In my change 8209189, I moved a lot of code around relating to code cache >>> unloading. Unfortunately, I failed to see that when making nmethods unloaded, >>> and their dependents are flushed... they are not immediately removed. >>> Instead, they are tagged to have stale entries, that are later cleaned during >>> weak metadata cleaning of InstanceKlass. As for method handles, they leak >>> instead, with some hacks to reduce the leaking by expunging entries while >>> adding and removing. >>> >>> Obviously, with InstanceKlass cleanup and code cache unloading now >>> possibly running in parallel after 8209189, things can go wrong. These two >>> phases used >>> to be separated by a "wait barrier" for the worker threads preventing that >>> parallelism. >>> >>> The reason dependency contexts were not cleaned when they were found >>> during code cache cleaning, is because it was not thread safe when code >>> cache unloading was made parallel instead of serial. But now that we are >>> implementing concurrent class unloading, we are going to need concurrent >>> cleanup of dependency contexts anyway. Therefore I will propose a bug fix >>> that fixes the problem in a way that works for both serial, parallel and >>> concurrent class unloading. >>> >>> The solution is to during code cache unloading claim cleaning of >>> encountered stale dependency contexts, and clean them straight away. >>> Entries are unlinked in a lock-free fashion, and placed on a purge list that >>> is walked and deleted during ClassLoaderDataGraph::purge(). This follows the >>> thinking of first unlinking, then syncing all threads, and then purging. >>> >>> New accessors for the head and next links, hide is_unloading entries and >>> helps unlinking them and putting them into the purge list. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8213565 >>> >>> Thanks, >>> /Erik >> > From robbin.ehn at oracle.com Fri Nov 23 16:51:07 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 23 Nov 2018 17:51:07 +0100 Subject: 8214271: Fast primitive to wake many threads Message-ID: Hi all, please review. When a safepoint is ended we need a way to get back to 100% utilization as fast as possible. 100% utilization means no idle cpu in the system if there is a JavaThread that could be executed. The traditional ways to wake many, e.g. semaphore, pthread_cond, is not implemented with a single syscall instead they typical do one syscall per thread to wake. This change-set contains that primitive, the WaitBarrier, and a gtest for it. No actual users, which is in coming patches. The WaitBarrier solves by doing a cooperative semaphore posting, threads woken will also post. On Linux we can instead directly use a futex and with one syscall wake all. Depending on how many threads and cpus the performance vary, but a good utilization of the machine, just on the edge of saturated, the time to reach 100% utilization is around 3 times faster with the WaitBarrier (where futex is faster than semaphore). Webrev: http://cr.openjdk.java.net/~rehn/8214271/webrev/ CR: https://bugs.openjdk.java.net/browse/JDK-8214271 Passes 100 iterations of gtest on our platforms, both fastdebug and release. And have been stable when used in safepoints (t1-8) (coming patches). Thanks, Robbin From robbin.ehn at oracle.com Fri Nov 23 16:55:07 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 23 Nov 2018 17:55:07 +0100 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: References: Message-ID: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> Forgot RFR in subject. /Robbin On 2018-11-23 17:51, Robbin Ehn wrote: > Hi all, please review. > > When a safepoint is ended we need a way to get back to 100% utilization as fast > as possible. 100% utilization means no idle cpu in the system if there is a > JavaThread that could be executed. The traditional ways to wake many, e.g. > semaphore, pthread_cond, is not implemented with a single syscall instead they > typical do one syscall per thread to wake. > > This change-set contains that primitive, the WaitBarrier, and a gtest for it. > No actual users, which is in coming patches. > > The WaitBarrier solves by doing a cooperative semaphore posting, threads woken > will also post. On Linux we can instead directly use a futex and with one > syscall wake all. Depending on how many threads and cpus the performance vary, > but a good utilization of the machine, just on the edge of saturated, the time > to reach 100% utilization is around 3 times faster with the WaitBarrier (where > futex is faster than semaphore). > > Webrev: > http://cr.openjdk.java.net/~rehn/8214271/webrev/ > > CR: > https://bugs.openjdk.java.net/browse/JDK-8214271 > > Passes 100 iterations of gtest on our platforms, both fastdebug and release. > And have been stable when used in safepoints (t1-8) (coming patches). > > Thanks, Robbin From jiangli.zhou at oracle.com Sat Nov 24 02:30:17 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Fri, 23 Nov 2018 18:30:17 -0800 Subject: RFR: 8214118: HeapRegions marked as archive even if CDS mapping fails In-Reply-To: <9fb4d0cc-5b8e-c45b-c6d6-1f92440bd563@oracle.com> References: <6fc6e308ba6ec28a97a93857546a4b944cb13a03.camel@oracle.com> <7efd2a30-58e9-8a27-144c-2773cde6cbed@oracle.com> <7920f8a2a809086ace958012f6e2df60dc458a8b.camel@oracle.com> <9fb4d0cc-5b8e-c45b-c6d6-1f92440bd563@oracle.com> Message-ID: <677bdf6e-8c3c-4eb5-be32-f09b6c950e7d@oracle.com> Hi Thomas and Stefan, I've filed https://bugs.openjdk.java.net/browse/JDK-8214277 for merging the open and closed maps. Thanks! Jiangli On 11/21/18 2:47 PM, Jiangli Zhou wrote: > Hi Thomas, > > > On 11/21/18 2:00 PM, Thomas Schatzl wrote: >> Hi, >> >> On Wed, 2018-11-21 at 13:05 -0800, Jiangli Zhou wrote: >>> Hi Thomas, >>> >>> Just some comments for your comments. :) >>> >>> On 11/21/18 12:29 PM, Thomas Schatzl wrote: >>>> Hi, >>>> >>>> On Wed, 2018-11-21 at 15:43 +0100, Stefan Johansson wrote: >>>>> Hi, >>>>> >>>>> Please review this fix to avoid leaving heap metadata in an >>>>> inconsistent state when CDS archive mapping fails. >>>>> >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8214118 >>>>> Webrev: http://cr.openjdk.java.net/~sjohanss/8214118/00/ >>>> ??? the change seems good. I have two comments on this though: >>>> >>>> - would it be useful to clear both maps in >>>> G1ArchiveAllocator::clear_range_archive()? I mean in the failure >>>> case these regions can't be used for one purpose or the other as >>>> soon as we fail the mapping, and saves passing the bool flag. >>> We map the 'closed' archive regions before 'open' archive regions. >>> The 'open' archive regions are only mapped if the 'closed' ones are >>> successfully mapped. >>> >>> The 'open' archive regions have dependency on the 'closed' regions >>> (objects in the open regions may have references to the closed >>> objects). >>> The 'closed' archive regions can be used independently however. If >>> 'open' archive regions fail to map, we can still keep the 'closed' >>> archives in use (no need to clear the closed map). >> G1ArchiveAllocator::clear_range_archive() does not clear the whole heap >> region range, just the given range which we failed to map the closed or >> open archive into as far as I understand. There can be no overlap, so >> clearing the range in both maps in that method should be fine. >> >> That's also why I suggested having a single map containing an enum >> value and not two maps with a bool each. >> >> But maybe I'm too tired already today, so feel free to ignore my >> suggestions :) > > That makes sense to me! If we use a single map for both 'open' and > 'closed' archive regions, clearing is a 'safe' operation for a > requested range since the heap archive ranges are guaranteed to not be > overlapped. I'll create a RFE for your merged suggestion (if you > haven't created one). Thanks! > > With the current separate maps, Stefan's fix that checks the boolean > argument probably is slightly cleaner (reflecting to the > G1ArchiveAllocator::set_range_archive() API)? > > Thanks! > Jiangli >> >> Thanks, >> ?? Thomas >> >> > From aph at redhat.com Sat Nov 24 11:12:43 2018 From: aph at redhat.com (Andrew Haley) Date: Sat, 24 Nov 2018 11:12:43 +0000 Subject: 8214271: Fast primitive to wake many threads In-Reply-To: References: Message-ID: <0a930a4b-3db9-dcaa-6f7c-0ba5b268bf5f@redhat.com> On 11/23/18 4:51 PM, Robbin Ehn wrote: > Webrev: > http://cr.openjdk.java.net/~rehn/8214271/webrev/ The code is rather lightly commented. In particular, it's not clear that a wait() might return spuriously from a futex and how you deal with that. Is it the case that users of this API are to be exposed to spurious wakeups? If so, you must say so in the comments. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From robbin.ehn at oracle.com Sat Nov 24 22:10:11 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Sat, 24 Nov 2018 23:10:11 +0100 Subject: 8214271: Fast primitive to wake many threads In-Reply-To: <0a930a4b-3db9-dcaa-6f7c-0ba5b268bf5f@redhat.com> References: <0a930a4b-3db9-dcaa-6f7c-0ba5b268bf5f@redhat.com> Message-ID: <9ff7188e-a8e3-a360-cbef-197855cfe084@oracle.com> Hi Andrew, thanks for having a look. On 2018-11-24 12:12, Andrew Haley wrote: > On 11/23/18 4:51 PM, Robbin Ehn wrote: >> Webrev: >> http://cr.openjdk.java.net/~rehn/8214271/webrev/ > > The code is rather lightly commented. In particular, it's not > clear that a wait() might return spuriously from a futex and how > you deal with that. Is it the case that users of this API are to be > exposed to spurious wakeups? If so, you must say so in the comments. > In my usage the futex is in the same memory location the entire processes lifetime, therefore spurious wake ups with retval 0, AFAIK, do not happen? But you are absolute right we should handle that. And EINTR totally slipped my mind, so thanks! Inc: http://cr.openjdk.java.net/~rehn/8214271/2/inc/webrev/ Full: http://cr.openjdk.java.net/~rehn/8214271/2/full/webrev/ /Robbin From david.holmes at oracle.com Mon Nov 26 06:12:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 26 Nov 2018 16:12:45 +1000 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> Message-ID: Hi Robbin, On 24/11/2018 2:55 am, Robbin Ehn wrote: > Forgot RFR in subject. Yep and now you have two different review threads happening in parallel unfortunately :( - src/hotspot/share/utilities/waitBarrier.hpp I'm studying just the WaitBarrierType API. Is this inherently tied to safepoint usage or intended as a general synchronization tool? As a general tool the API does not have clear semantics on how it should be used: - How do you communicate the current tag between the arming thread and the waiting threads? There would seem to be inherent races between arm(tag) and wait(tag) unless access to the tag itself is synchronized via another mechanism. - What happens if wake() is called before disarm()? Should it be disallowed? (For other readers disarm() and wake() are distinct operations so that they fit better into the existing safepoint protocol which disarms the safepoint polling page at one spot and wakes blocked threads at another.) - Should there be constraints that the same thread must arm/disarm/wake? It doesn't really make sense to allow these operations to happen in arbitrary order from multiple threads. The semantics for re-arming with the same tag should be clearly set out not "implementation-defined". This should probably be a usage error IMHO - but it comes back to how the tag is expected to be used. As Andrew mentioned there needs to be documentation regarding spurious wakeups or other "interruptions" at the API level. And I assume a blocked wait() only ever returns in response to a wake(). Nothwithstanding clarification of the above may I suggest the following rewrite of the API documentation for further clarity: /* Platform independent WaitBarrier API. An armed WaitBarrier prevents threads from advancing until the barrier is disarmed and the waiting threads woken. The barrier is armed by setting a non-zero value - the tag. Expected Usage: - Arming thread: tag = ...; // non-zero value barrier.arm(tag); barrier.disarm(); barrier.wake(); - After arm(tag) returns any thread calling wait(tag) will block - After disarm() returns any subsequent calls to wait(tag) will not block - After wake() returns all blocked threads are unblocked and eligible to execute again - After calling disarm() and wake() the barrier is ready to be re-armed with a new tag - Waiting threads wait(tag); // don't execute following code unless 'safe' - A call to wait(tag) will block if the barrier is armed with the value 'tag'; else it will return immediately. - A blocked thread is eligible to execute again once the barrier is disarmed and wake() has been called. A primary goal of the WaitBarrier implementation is to wake all waiting threads as fast, and as concurrently, as possible. */ Looking at the "implementation" in this file I'm unclear on the way the Linux specialization is being handled here. Why do we need the template? Can't this just be done the same way we do Semaphore? More to follow. Thanks, David ----- > /Robbin > > On 2018-11-23 17:51, Robbin Ehn wrote: >> Hi all, please review. >> >> When a safepoint is ended we need a way to get back to 100% >> utilization as fast >> as possible. 100% utilization means no idle cpu in the system if there >> is a >> JavaThread that could be executed. The traditional ways to wake many, >> e.g. >> semaphore, pthread_cond, is not implemented with a single syscall >> instead they >> typical do one syscall per thread to wake. >> >> This change-set contains that primitive, the WaitBarrier, and a gtest >> for it. >> No actual users, which is in coming patches. >> >> The WaitBarrier solves by doing a cooperative semaphore posting, >> threads woken >> will also post. On Linux we can instead directly use a futex and with one >> syscall wake all. Depending on how many threads and cpus the >> performance vary, >> but a good utilization of the machine, just on the edge of saturated, >> the time to reach 100% utilization is around 3 times faster with the >> WaitBarrier (where futex is faster than semaphore). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8214271 >> >> Passes 100 iterations of gtest on our platforms, both fastdebug and >> release. >> And have been stable when used in safepoints (t1-8) (coming patches). >> >> Thanks, Robbin From tobias.hartmann at oracle.com Mon Nov 26 09:08:42 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Mon, 26 Nov 2018 10:08:42 +0100 Subject: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit In-Reply-To: <972fbe636a16425d897470c7c7dee380@sap.com> References: <878t1lp5bv.fsf@redhat.com> <773b49f6-3788-a86d-7758-a03027894a09@redhat.com> <87tvk9nkxo.fsf@redhat.com> <87o9ahnjea.fsf@redhat.com> <874lc8zy4x.fsf@redhat.com> <61afcbb6efff4f2fa4be6fb93e2101c7@sap.com> <988d7ab2ecc34421a9082095379b213f@sap.com> <87d0qvor14.fsf@redhat.com> <972fbe636a16425d897470c7c7dee380@sap.com> Message-ID: Hi Roland, the test was problem listed in the meantime (JDK-8214276). The fix should therefore remove the test from that list. Best regards, Tobias On 23.11.18 15:13, Doerr, Martin wrote: > Ship it. > > Thanks, Martin > > > -----Original Message----- > From: Roland Westrelin > Sent: Freitag, 23. November 2018 14:36 > To: Doerr, Martin ; Doerr, Martin ; Aleksey Shipilev ; hotspot-dev Source Developers > Subject: RE: RFR(T): 8214206: Fix for JDK-8213419 is broken on 32-bit > > > Hi Martin, > >> webrev.02 seems to work. But I still wonder why we should have a version which only casts to unsigned. > > Thanks for giving it a try. Not sure if a signed + an unsigned versions > are needed. But not relying on implicit casting seems safer. > > webrev.02 passes the submit repo. So unless someone objects I'll push it > later today (with a fix for the misleading comment that David pointed > out). > > Roland. > From goetz.lindenmaier at sap.com Mon Nov 26 12:57:22 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 26 Nov 2018 12:57:22 +0000 Subject: RFR(S): downport 8211326: add OS user related information to hs_err file In-Reply-To: <82f78262-17b8-ae00-d6d9-767f1362054a@oracle.com> References: <82f78262-17b8-ae00-d6d9-767f1362054a@oracle.com> Message-ID: <685dd2229c0d41029cca834ac8ea938a@sap.com> Hi David, thanks for reviewing. I know about the dependency on Thomas' change. He'll take charge. 8u-dev was meant to be jdk-updates-dev. Best regards, Goetz. > -----Original Message----- > From: David Holmes > Sent: Donnerstag, 22. November 2018 23:06 > To: Lindenmaier, Goetz ; hotspot-dev Source > Developers ; jdk8u-dev at openjdk.java.net > Subject: Re: RFR(S): downport 8211326: add OS user related information to > hs_err file > > Hi Goetz, > > Not sure why jdk8u-dev is cc'd here?? > > This depends on the backport of JDK-8211845, but Thomas has withdrawn > that backport request. That process needs to be reinstated. > > Otherwise the actual backport looks accurate. > > Thanks, > David > > On 23/11/2018 2:37 am, Lindenmaier, Goetz wrote: > > Hi, > > > > I would like to downport 8211326. It helps us maintain systems > > that are (and most will be) running on jdk11. > > > > Unfortunately, in posix.hpp the context changed slightly, and thus > > the change does not apply. So I need a review. > > Webrev against jdk11u: > > http://cr.openjdk.java.net/~goetz/wr18/8211326-hs_err-user-jdk11u/01/ > > Original webrev: > > http://cr.openjdk.java.net/~mbaesken/webrevs/8211326.4/ > > > > Best regards, > > Goetz. > > From sgehwolf at redhat.com Mon Nov 26 13:02:55 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 26 Nov 2018 14:02:55 +0100 Subject: [PING] [11u] RFR: 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: <675c77930b9ba3c7f724c74f9372582a9080b81e.camel@redhat.com> References: <5cd90d21-5fa8-4289-c242-5cca6f9ead8e@oracle.com> <675c77930b9ba3c7f724c74f9372582a9080b81e.camel@redhat.com> Message-ID: <47628217e09770c32097984a2959d6eb55140605.camel@redhat.com> On Mon, 2018-11-19 at 17:23 +0100, Severin Gehwolf wrote: > On Fri, 2018-11-16 at 10:11 -0500, Lois Foltan wrote: > > On 11/16/2018 10:09 AM, Severin Gehwolf wrote: > > > > > On Thu, 2018-11-15 at 20:09 +0100, Severin Gehwolf wrote: > > > > Hi, > > > > > > > > Please review this 11u backport for JDK-8212937. While the JDK 12 patch > > > > applies cleanly it doesn't quite do the right thing. The JDK 12 test > > > > would still crash the JVM due to the lack of the changes in > > > > src/hotspot/share/classfile/javaClasses.cpp as describe_external is > > > > being used in JDK 11. That function has been removed with JDK- > > > > 8205611[1] in JDK 12. > > > > > > > > In addition, the exception messages aren't the same between JDK 12 and > > > > JDK 11, so Test.java needed an update to account for that. Also due to > > > > lack of JDK-8205611[1] in JDK 11. Everything else is the same as for > > > > JDK 12. > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8212937 > > > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8212937/jdk11/webrev.01/ > > > > > > > > Testing: Added regression test, internal test. > > > > > > Also: > > > > > > tier1 testing showed no regressions with this on linux x86_64. > > > > Hi Severin, > > > > Thank you for doing this testing. I think your change looks good! > > Thanks, Lois. Could I get a second review for this, please? Anyone willing to review this? Thanks, Severin > > > > Thoughts? > > > > > > > > Thanks, > > > > Severin > > > > > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8205611 > > > > From matthias.baesken at sap.com Mon Nov 26 13:11:43 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 26 Nov 2018 13:11:43 +0000 Subject: RFR : [XXS] 8214297: testbug : missing fclose in jvmti_aod.cpp Message-ID: Please review this small test fix. It adds an fclose in an early return case to function nsk_jvmti_aod_redefineClass . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8214297 http://cr.openjdk.java.net/~mbaesken/webrevs/8214297.0/ Thanks ,Matthias From thomas.stuefe at gmail.com Mon Nov 26 13:15:35 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 26 Nov 2018 14:15:35 +0100 Subject: RFR : [XXS] 8214297: testbug : missing fclose in jvmti_aod.cpp In-Reply-To: References: Message-ID: Looks fine Matthias. ..Thomas On Mon, Nov 26, 2018 at 2:12 PM Baesken, Matthias wrote: > > Please review this small test fix. > It adds an fclose in an early return case to function nsk_jvmti_aod_redefineClass . > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8214297 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8214297.0/ > > > Thanks ,Matthias From coleen.phillimore at oracle.com Mon Nov 26 14:20:45 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 26 Nov 2018 09:20:45 -0500 Subject: RFR: 8213209: [REDO] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <5BF2B1F2.8070004@oracle.com> References: <5BF2B1F2.8070004@oracle.com> Message-ID: <8b5280bc-09ff-4069-a49b-5edf1d9284da@oracle.com> Looks good! Coleen On 11/19/18 7:52 AM, Erik ?sterlund wrote: > Hi Coleen, > > Thanks for the review. > > Full webrev: > http://cr.openjdk.java.net/~eosterlund/8213209/webrev.01/ > > Incremental webrev: > http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00_01/ > > On 2018-11-16 22:08, coleen.phillimore at oracle.com wrote: >> >> Erik, thank you for taking this change and making it lock-free. >> >> On 11/16/18 1:12 PM, Erik ?sterlund wrote: >>> Hi, >>> >>> This is a redo of Coleen's earlier patch to clean weak metadata >>> links using the Compile_lock. >>> >>> We noticed the proposed solution could cause deadlocks. At least one >>> of the reasons for that is that the Compile_lock was taken during >>> printing and verification in safepoints. And the Compile_lock is >>> sometimes held while issuing safepoints. The latter problem also >>> forces us to take the Compile_lock before joining the suspendible >>> thread set, to avoid another deadlock situation. But that is really >>> nasty for other reasons. And there have been a few more problems as >>> well. >>> >>> So in this case, we think it is both safer, less error prone, and >>> better performing (although less important) to perform this cleaning >>> in a lock-free fashion rather than trying to dodge all the issues >>> related to the locking protocol. >>> >>> Inserts still require a lock. Therefore, we are dealing with >>> lock-free reads, and lock-free cleaning. The lock-free cleaning is >>> interacting with multiple lock-free readers and a single writer. >>> >>> The single inserter always helps with cleaning the subclass list >>> head before prepending new nodes to the chain. That allows an >>> invariant that the siblings link is never inserted pointing at a >>> Klass that is unloading, which simplifies things a lot. The head is >>> inserted in a classic load + CAS in a loop. >>> >>> Readers require load_acquire when reading the head, due to competing >>> inserts putting new Klass pointers there. The siblings, however, >>> only need relaxed consistency, because they are linked to data >>> strictly older than the head, which has already been acquired. >>> >>> Unlinked entries are all inserted into a purge list, because freeing >>> them immediately is not safe. A global handshake operation is >>> performed, and after that ClassLoaderDataGraph::purge() will go >>> through the list and delete the entries safely. >> >> I don't see the purge list in your change, and it's not needed since >> this is unlinking in place.? I think this is your other change you're >> thinking of. > > Yes you are right, of course. It's the unloading list and not a > separate purge list... > >> Should the Compile_lock be removed in ciInstanceKlass.cpp >> ciInstanceKlass* ciInstanceKlass::implementor() { >> >> and in jvmciCompilerToVM.cpp >> C2V_VMENTRY(jobject, getImplementor, (JNIEnv *, jobject, jobject >> jvmci_type)) >> >> The former does a bit more than calling implementor().? Or do you >> want to leave these locks to be conservative? > > While I think your analysis is correct, I would indeed like to leave > them in this RFE to be conservative. > >> ? JVMCIKlassHandle handle(THREAD); >> ? { >> ??? // Need Compile_lock around implementor() >> ??? MutexLocker locker(Compile_lock); >> ??? handle = iklass->implementor(); >> ? } >> >> >> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/oops/instanceKlass.cpp.udiff.html >> >> >> nof_implementors() has an assert_locked_or_safepoint(Compile_lock); >> which it no longer needs.? You'll hit this from printing because >> you've removed the locking here: >> >> 3093?? if (is_interface()) { >> 3094 MutexLocker ml(Compile_lock); >> 3095???? st->print_cr(BULLET"nof implementors:? %d", >> nof_implementors()); >> 3096???? if (nof_implementors() == 1) { >> 3097?????? st->print_cr(BULLET"implementor:??? "); >> 3098?????? st->print("?? "); >> 3099?????? implementor()->print_value_on(st); >> 3100?????? st->cr(); >> 3101???? } >> >> >> We don't test printing very well. >> > > Ahh yes - good catch. I removed the assert as it is no longer required. > >> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/utilities/vmError.cpp.udiff.html >> >> >> This file is unchanged except a line removed. > > I put it back the way it was before. > >> This change looks good.? I've run through the tests with it and >> didn't find any problems that doomed my earlier version. > > Thank you for running additional tests on this patch! > > Thanks, > /Erik > >> Thanks!! >> Coleen >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8213209 >>> >>> Thanks, >>> /Erik >> > From coleen.phillimore at oracle.com Mon Nov 26 14:28:44 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 26 Nov 2018 09:28:44 -0500 Subject: RFR : [XXS] 8214297: testbug : missing fclose in jvmti_aod.cpp In-Reply-To: References: Message-ID: <53fbebce-714b-6396-2e3a-d796d663af81@oracle.com> I agree.? Looks good and trivial to push now. thanks, Coleen On 11/26/18 8:15 AM, Thomas St?fe wrote: > Looks fine Matthias. > > ..Thomas > On Mon, Nov 26, 2018 at 2:12 PM Baesken, Matthias > wrote: >> Please review this small test fix. >> It adds an fclose in an early return case to function nsk_jvmti_aod_redefineClass . >> >> Bug/webrev : >> >> https://bugs.openjdk.java.net/browse/JDK-8214297 >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8214297.0/ >> >> >> Thanks ,Matthias From alexey.menkov at oracle.com Mon Nov 26 19:14:00 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Mon, 26 Nov 2018 11:14:00 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp Message-ID: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> Hi all, Please review the fix for https://bugs.openjdk.java.net/browse/JDK-8195639 webrev: http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ Description: The test suspends a thread, turns on single stepping and then calls PopFrame. SingleStep event is expected as soon as the thread is resumed and PopFrame is processed (so we have call stack with the depth 1 less than it was before PopFrame). Instead SingleStep event is received with much deeper call stack (and PopFrame processes wrong frame). Research shown that this is caused by missed deoptimization of the current frame. As far as I understand CompilationPolicy::event should handle the case when the thread has is_interp_only_mode() set, but TieredThresholdPolicy::event checks this only then CompLevel is CompLevel_none. CompilerRuntime always calls policy()->event with CompLevel == CompLevel_aot. The fix looks quite simple, but I'd appreciate feedback from runtime and compiler teams as I'm not sure I completely understand all the details of the "PopFrame dance". --alex From coleen.phillimore at oracle.com Mon Nov 26 20:21:40 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 26 Nov 2018 15:21:40 -0500 Subject: [PING] [11u] RFR: 8212937: Parent class loader may not have a referred ClassLoaderData instance when obtained in Klass::class_in_module_of_loader In-Reply-To: <47628217e09770c32097984a2959d6eb55140605.camel@redhat.com> References: <5cd90d21-5fa8-4289-c242-5cca6f9ead8e@oracle.com> <675c77930b9ba3c7f724c74f9372582a9080b81e.camel@redhat.com> <47628217e09770c32097984a2959d6eb55140605.camel@redhat.com> Message-ID: <3e29de39-9bf9-e6be-7459-1857ef4842f9@oracle.com> I reviewed this.? This looks good. Thanks for the backport. Coleen On 11/26/18 8:02 AM, Severin Gehwolf wrote: > On Mon, 2018-11-19 at 17:23 +0100, Severin Gehwolf wrote: >> On Fri, 2018-11-16 at 10:11 -0500, Lois Foltan wrote: >>> On 11/16/2018 10:09 AM, Severin Gehwolf wrote: >>> >>>> On Thu, 2018-11-15 at 20:09 +0100, Severin Gehwolf wrote: >>>>> Hi, >>>>> >>>>> Please review this 11u backport for JDK-8212937. While the JDK 12 patch >>>>> applies cleanly it doesn't quite do the right thing. The JDK 12 test >>>>> would still crash the JVM due to the lack of the changes in >>>>> src/hotspot/share/classfile/javaClasses.cpp as describe_external is >>>>> being used in JDK 11. That function has been removed with JDK- >>>>> 8205611[1] in JDK 12. >>>>> >>>>> In addition, the exception messages aren't the same between JDK 12 and >>>>> JDK 11, so Test.java needed an update to account for that. Also due to >>>>> lack of JDK-8205611[1] in JDK 11. Everything else is the same as for >>>>> JDK 12. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8212937 >>>>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8212937/jdk11/webrev.01/ >>>>> >>>>> Testing: Added regression test, internal test. >>>> Also: >>>> >>>> tier1 testing showed no regressions with this on linux x86_64. >>> Hi Severin, >>> >>> Thank you for doing this testing. I think your change looks good! >> Thanks, Lois. Could I get a second review for this, please? > Anyone willing to review this? > > Thanks, > Severin > >>>>> Thoughts? >>>>> >>>>> Thanks, >>>>> Severin >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8205611 >>> From thomas.stuefe at gmail.com Mon Nov 26 20:43:21 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 26 Nov 2018 21:43:21 +0100 Subject: Extend NMT to JDK native libraries? In-Reply-To: <22785794-bee7-98e9-2151-c8487f65744a@redhat.com> References: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> <22785794-bee7-98e9-2151-c8487f65744a@redhat.com> Message-ID: Hi Zhengyu, On Wed, Nov 21, 2018 at 5:26 PM Zhengyu Gu wrote: > > > > > But note the unbalanced-malloc issue. The more we expose NMT to the > > JDK wild, the more probable are such bugs. Within the hotspot all is > > nice and tidy. > > Hotspot also has a few unbalanced-malloc instances. Oh, where? Really, raw malloc with os::free, or the other way around? Would they not create the problem I described, corrupted C-heap etc? > And yes, we will > find more in library, I would consider it as initial investment to fix > them, especially, if we can do them module-by-module, no? Yes, they should be fixed if possible. My concern is that it is realistic to assume that we miss some, or even if not that new ones will creep in if the code changes. And that the penalties for these unbalanced mallocs is severe - crashes and heap corruptions - if we use malloc headers. Even worse, they would only show up, in release builds, if NMT is switched on, since only NMT adds malloc headers in release builds. That would be annoying, telling a customer to switch on NMT just for him to hit such a code path and crash the VM. I don't know, I seem to miss something here. Do you not see this as serious problem preventing NMT from covering code outside hotspot? > > Mismatched statistics is quite annoying ... Did see people actually > counting bytes and expecting to match :-) JDK-8191369 actually was > driven by customers, who tried to match smap. > I remember :) and yes, we have many curious customers too. But that is fine. We want NMT to be as exact as possible. Cheers, Thomas > > Thanks, > > -Zhengyu > > > > > If we wanted to avoid these bugs, we would have to remove malloc > > headers from both os::malloc() and NMT MallocTracker and add a malloc > > pointer hashtable of some sorts to the latter. This is not very > > difficult, but would still need an initial investment. > > > > > > > Thanks, Thomas > > > >> > >> Thanks, > >> > >> -Zhengyu > >> > >>> > >>> (I think even if we were to instrument parts of the JDK - e.g. just > >>> NIO - this would already be very helpful. In parts we do this already > >>> for mtOther.). > >>> > >>> On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu wrote: > >>>> > >>>> FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) > >>>> https://bugs.openjdk.java.net/browse/JDK-6995789 > >>>> > >>>> -Zhengyu > >>>> > >>>> On 11/21/18 9:28 AM, Thomas St?fe wrote: > >>>>> Hi all, > >>>>> > >>>>> (yet again not sure if this is serviceablity-dev or not - I start at > >>>>> hs-dev, feel free to move this mail around.) > >>>>> > >>>>> Do we have any plans to extend NMT to cover native JDK libaries too? > >>>>> That would be a really cool feature. > >>>>> > >>>>> -- > >>>>> > >>>>> We at SAP have done a similar thing in the past: > >>>>> > >>>>> We have a monitoring facility in our port which tracks C-heap > >>>>> allocations, non-imaginatively called "malloc statistic". This feature > >>>>> predates NMT somewhat - had we had NMT at that time, we would not have > >>>>> bothered. Our Malloc statistic is less powerful than NMT and > >>>>> implementation-wise completely at odds with it, so I never felt the > >>>>> urge to bring it upstream. However, one thing we did do is we extended > >>>>> its coverage to the JDK native code. > >>>>> > >>>>> This has been quite helpful in the past to find leaks in JDK, see > >>>>> e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 > >>>>> > >>>>> We did this by exposing os::malloc, os::free etc from libjvm.so > >>>>> ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we > >>>>> then either manually replaced calls to raw ::malloc(), ::free() etc > >>>>> with JVM_malloc(), JVM_free(). Or, in places where this was possible, > >>>>> we did this replacement stuff wholesale by employing a header which > >>>>> re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, > >>>>> we also had to add a number of linkage dependencies to the libjvm.so. > >>>>> > >>>>> All this is pretty standard stuff. > >>>>> > >>>>> One detail stood out: malloc headers are evil. In our experience, JDK > >>>>> native code was more difficult to control and "unbalanced > >>>>> malloc/frees" kept creeping in - especially with the > >>>>> wholesale-redefinition technique. Unbalanced mallocs/frees means cases > >>>>> where malloc() is instrumented but ::free() stays raw, or the other > >>>>> way around. Both combinations are catastrophic since os::malloc uses > >>>>> malloc headers. We typically corrupted the C-Heap and crashed, often > >>>>> much later in completely unrelated places. > >>>>> > >>>>> These types of bugs were very hard to spot and hence very expensive. > >>>>> And they can creep in in many ways. One example, there exist a > >>>>> surprising number of system APIs which return results in C-heap and > >>>>> require the user to free that, which of course must happen with raw > >>>>> ::free(), not os::free(). > >>>>> > >>>>> We fixed this by not using malloc headers. That means a pointer > >>>>> returned by os::malloc() is compatible with raw ::free() and vice > >>>>> versa. The only bad thing happening would be our statistic numbers > >>>>> being slightly off. > >>>>> > >>>>> Instead of malloc headers we use a hand-groomed hash table to track > >>>>> the malloced memory. It is actually quite fast, fast enough that this > >>>>> malloc statistic feature is on-by-default in our port. > >>>>> > >>>>> -- > >>>>> > >>>>> Of course, if we extend NMT to JDK native code we also would want to > >>>>> extend it to mmap() etc - we never did this with our statistic, since > >>>>> it only tracked malloc. > >>>>> > >>>>> What do you think? Did anyone else play with similar ideas? Would it > >>>>> be worth the effort? > >>>>> > >>>>> Cheers, Thomas > >>>>> From rkennke at redhat.com Mon Nov 26 21:39:30 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 26 Nov 2018 22:39:30 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector Message-ID: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Hi, This is the first round of changes for including Shenandoah GC into mainline. I divided the review into parts that roughly correspond to the mailing lists that would normally review it, and I divided it into 'shared' code changes and 'shenandoah' code changes (actually, mostly additions). The intend is to eventually push them as single 'combined' changeset, once reviewed. JEP: ? https://openjdk.java.net/jeps/189 Bug entry: ?https://bugs.openjdk.java.net/browse/JDK-8214259 Webrevs: ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ For those who want to see the full change, have a look at the shenandoah-complete directory, it contains the full combined webrev. Alternatively, there is the file shenandoah-master.patch , which is what I intend to commit (and which should be equivalent to the 'shenandoah-complete' webrev). Sections to review (at this point) are the following: ?*) shenandoah-gc ??? - Actual Shenandoah implementation, almost completely residing in gc/shenandoah ?*) shared-gc ??? - This is mostly boilerplate that is common to any GC ??? - referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1) ??? - taskqueue.hpp has some small adjustments to enable subclassing ?*) shared-serviceability ??? - The usual code to support another GC ?*) shared-runtime ??? - A number of friends declarations to allow Shenandoah iterators to hook up with, ????? e.g. ClassLoaderData, CodeCache, etc ??? - Warning and disabling JFR LeakProfiler ??? - fieldDescriptor.hpp added is_stable() accessor, for use in Shenandoah C2 optimizations ??? - Locks initialization in mutexLocker.cpp as usual ??? - VM operations defines for Shenandoah's VM ops ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in Shenandoah's logging ??? - The usual macros in macro.hpp ?*) shared-build ??? - Add shenandoah feature, enabled by default, as agreed with Vladimir K. beforehand ??? - Some flags for shenandoah-enabled compilation to get SUPPORT_BARRIER_ON_PRIMITIVES ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for Shenandoah's barriers ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source files, which is ????? useful to get the whole marking loop inlined (observed significant regression if we ????? don't) ?*) shared-tests ??? - Test infrastructure to support Shenandoah ??? - Shenandoah test groups ??? - Exclude Shenandoah in various tests that can be run with selected GC ??? - Enable/add configure for Shenandoah for tests that make sense to run with it ?*) shenandoah-tests ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory ??? - A couple of tests configurations have been added, e.g. TestGCBasherWithShenandoah.java I intentionally left out shared-compiler for now, because we have some work left to do there, but if you click around you'll find the patch anyway, in case you want to take a peek at it. We have regular builds on: ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} ? - {Windows} x {x86_64}, ? - {MacOS X} x {x86_64} This also routinely passes: ? - the new Shenandoah tests ? - jcstress with/without aggressive Shenandoah verification ? - specjvm2008 with/without aggressive Shenandoah verification I'd like to thank my collegues at Red Hat: Christine Flood, she deserves the credit for being the original inventor of Shenandoah, Aleksey Shipl?v, Roland Westrelin & Zhengyu Gu for their countless contributions, everybody else in Red Hat's OpenJDK team for testing, advice and support, my collegues in Oracle's GC, runtime and compiler teams for tirelessly helping with and reviewing all the GC interface and related changes, and of course the many early adopters for reporting bugs and success stories and feature requests: we wouldn't be here without any of you! Best regards, Roman From hohensee at amazon.com Mon Nov 26 22:01:01 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 26 Nov 2018 22:01:01 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> Message-ID: <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> The compiler could estimate the cost of an iteration using some rough pessimistic heuristic and strip mine based on that. Measuring the safepoint time directly is fragile due to inability to find all the hardware platforms that might run the JVM. Paul ?On 11/22/18, 8:07 AM, "hotspot-dev on behalf of Andrew Haley" wrote: In several places in HotSpot we have intrinsics that can block for a long time. The most extreme cases of this are some crypto accelerators that are only bounded by the maximum size of a byte array, 2 gigabytes. This leads to extended time to safepoint, where every thread has to wait for the intrinsic to complete. It can be on the order of seconds. We could simply limit the maximum buffer size of the intrinsic and wrap it in a Java loop with (as usual) a safepoint check. This does not noticeably slow anything down, but it does solve the TTSP problem. This can be important when multi-megabyte messages are being encrypted. The question is, then, what is the maximum tolerable time that an intrinsic should block? I suggest that it should be on the order of a millisecond, particularly when garbage collectors are trying to drive down the safepoint time. This corresponds to AES/GCM encryption of about 128k - 256k batches on a decently fast 64-bit processor. Doing encryption in batches like this doesn't significantly slow down anything. It is, in a sense, like loop strip mining for primitives. So, I'm proposing two things: firstly we fix the intrinsics that can block for extended periods, and we then declare that no new intrinsic code will be accepted without the worst case safepoint time being measured and found to be acceptable. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From hohensee at amazon.com Mon Nov 26 22:16:15 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Mon, 26 Nov 2018 22:16:15 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> Message-ID: <9D626E11-3115-47C8-BF70-A09B14B7FC52@amazon.com> Also, it's not just intrinsics that have this problem. All counted loops should be "strip-mined" this way. Android JVMs have been doing it for time out of mind because they budget 2-3ms max per 16.67ms frame computation time for stw events. ?On 11/26/18, 2:01 PM, "Hohensee, Paul" wrote: The compiler could estimate the cost of an iteration using some rough pessimistic heuristic and strip mine based on that. Measuring the safepoint time directly is fragile due to inability to find all the hardware platforms that might run the JVM. Paul On 11/22/18, 8:07 AM, "hotspot-dev on behalf of Andrew Haley" wrote: In several places in HotSpot we have intrinsics that can block for a long time. The most extreme cases of this are some crypto accelerators that are only bounded by the maximum size of a byte array, 2 gigabytes. This leads to extended time to safepoint, where every thread has to wait for the intrinsic to complete. It can be on the order of seconds. We could simply limit the maximum buffer size of the intrinsic and wrap it in a Java loop with (as usual) a safepoint check. This does not noticeably slow anything down, but it does solve the TTSP problem. This can be important when multi-megabyte messages are being encrypted. The question is, then, what is the maximum tolerable time that an intrinsic should block? I suggest that it should be on the order of a millisecond, particularly when garbage collectors are trying to drive down the safepoint time. This corresponds to AES/GCM encryption of about 128k - 256k batches on a decently fast 64-bit processor. Doing encryption in batches like this doesn't significantly slow down anything. It is, in a sense, like loop strip mining for primitives. So, I'm proposing two things: firstly we fix the intrinsics that can block for extended periods, and we then declare that no new intrinsic code will be accepted without the worst case safepoint time being measured and found to be acceptable. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.joelsson at oracle.com Mon Nov 26 22:47:46 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 26 Nov 2018 14:47:46 -0800 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> Build changes look ok to me. /Erik On 2018-11-26 13:39, Roman Kennke wrote: > Hi, > > This is the first round of changes for including Shenandoah GC into > mainline. > I divided the review into parts that roughly correspond to the mailing lists > that would normally review it, and I divided it into 'shared' code > changes and > 'shenandoah' code changes (actually, mostly additions). The intend is to > eventually > push them as single 'combined' changeset, once reviewed. > > JEP: > ? https://openjdk.java.net/jeps/189 > Bug entry: > > ?https://bugs.openjdk.java.net/browse/JDK-8214259 > > Webrevs: > ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ > > For those who want to see the full change, have a look at the > shenandoah-complete > > directory, > it contains the full combined webrev. Alternatively, there is the file > shenandoah-master.patch > , > which is what I intend to commit (and which should be equivalent to the > 'shenandoah-complete' webrev). > > Sections to review (at this point) are the following: > ?*) shenandoah-gc > > ??? - Actual Shenandoah implementation, almost completely residing in > gc/shenandoah > > ?*) shared-gc > > ??? - This is mostly boilerplate that is common to any GC > ??? - referenceProcessor.cpp has a little change to make one assert not > fail (next to CMS and G1) > ??? - taskqueue.hpp has some small adjustments to enable subclassing > > ?*) shared-serviceability > > ??? - The usual code to support another GC > > ?*) shared-runtime > > ??? - A number of friends declarations to allow Shenandoah iterators to > hook up with, > ????? e.g. ClassLoaderData, CodeCache, etc > ??? - Warning and disabling JFR LeakProfiler > ??? - fieldDescriptor.hpp added is_stable() accessor, for use in > Shenandoah C2 optimizations > ??? - Locks initialization in mutexLocker.cpp as usual > ??? - VM operations defines for Shenandoah's VM ops > ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in > Shenandoah's logging > ??? - The usual macros in macro.hpp > > ?*) shared-build > > ??? - Add shenandoah feature, enabled by default, as agreed with > Vladimir K. beforehand > ??? - Some flags for shenandoah-enabled compilation to get > SUPPORT_BARRIER_ON_PRIMITIVES > ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for > Shenandoah's barriers > ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source > files, which is > ????? useful to get the whole marking loop inlined (observed significant > regression if we > ????? don't) > > ?*) shared-tests > > ??? - Test infrastructure to support Shenandoah > ??? - Shenandoah test groups > ??? - Exclude Shenandoah in various tests that can be run with selected GC > ??? - Enable/add configure for Shenandoah for tests that make sense to > run with it > > ?*) shenandoah-tests > > ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory > ??? - A couple of tests configurations have been added, e.g. > TestGCBasherWithShenandoah.java > > I intentionally left out shared-compiler for now, because we have some > work left to do > there, but if you click around you'll find the patch anyway, in case you > want to take > a peek at it. > > We have regular builds on: > ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} > ? - {Windows} x {x86_64}, > ? - {MacOS X} x {x86_64} > > This also routinely passes: > ? - the new Shenandoah tests > ? - jcstress with/without aggressive Shenandoah verification > ? - specjvm2008 with/without aggressive Shenandoah verification > > > I'd like to thank my collegues at Red Hat: Christine Flood, she deserves > the credit for being the original inventor of Shenandoah, Aleksey > Shipl?v, Roland Westrelin & Zhengyu Gu for their countless > contributions, everybody else in Red Hat's OpenJDK team for testing, > advice and support, my collegues in Oracle's GC, runtime and compiler > teams for tirelessly helping with and reviewing all the GC interface and > related changes, and of course the many early adopters for reporting > bugs and success stories and feature requests: we wouldn't be here > without any of you! > > Best regards, > Roman > From vladimir.kozlov at oracle.com Mon Nov 26 23:12:46 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 26 Nov 2018 15:12:46 -0800 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> Message-ID: <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> Hi Roman, You need to check if shenandoahgc is disabled first - check DISABLED_JVM_FEATURES list (see code for jvmci). Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check. Do you support all x86 OSs? Why you set VM CFLAGS when shenandoahgc is not enabled? It is in JvmFeatures.gmk. I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc specific nodes. This looks intrusive. I hope you clean it up. Thanks, Vladimir On 11/26/18 2:47 PM, Erik Joelsson wrote: > Build changes look ok to me. > > /Erik > > On 2018-11-26 13:39, Roman Kennke wrote: >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the mailing lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is to >> eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> ?? https://openjdk.java.net/jeps/189 >> Bug entry: >> >> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> which is what I intend to commit (and which should be equivalent to the >> 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> ??*) shenandoah-gc >> >> ???? - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> ??*) shared-gc >> >> ???? - This is mostly boilerplate that is common to any GC >> ???? - referenceProcessor.cpp has a little change to make one assert not >> fail (next to CMS and G1) >> ???? - taskqueue.hpp has some small adjustments to enable subclassing >> >> ??*) shared-serviceability >> >> ???? - The usual code to support another GC >> >> ??*) shared-runtime >> >> ???? - A number of friends declarations to allow Shenandoah iterators to >> hook up with, >> ?????? e.g. ClassLoaderData, CodeCache, etc >> ???? - Warning and disabling JFR LeakProfiler >> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> ???? - Locks initialization in mutexLocker.cpp as usual >> ???? - VM operations defines for Shenandoah's VM ops >> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> ???? - The usual macros in macro.hpp >> >> ??*) shared-build >> >> ???? - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> ???? - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> ?????? useful to get the whole marking loop inlined (observed significant >> regression if we >> ?????? don't) >> >> ??*) shared-tests >> >> ???? - Test infrastructure to support Shenandoah >> ???? - Shenandoah test groups >> ???? - Exclude Shenandoah in various tests that can be run with selected GC >> ???? - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> ??*) shenandoah-tests >> >> ???? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >> ???? - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case you >> want to take >> a peek at it. >> >> We have regular builds on: >> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> ?? - {Windows} x {x86_64}, >> ?? - {MacOS X} x {x86_64} >> >> This also routinely passes: >> ?? - the new Shenandoah tests >> ?? - jcstress with/without aggressive Shenandoah verification >> ?? - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >> the credit for being the original inventor of Shenandoah, Aleksey >> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface and >> related changes, and of course the many early adopters for reporting >> bugs and success stories and feature requests: we wouldn't be here >> without any of you! >> >> Best regards, >> Roman >> From kim.barrett at oracle.com Tue Nov 27 00:07:02 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 26 Nov 2018 19:07:02 -0500 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: <46CBF9C2-7684-4EED-80F8-7103D48FA3A0@oracle.com> A few quick comments (not a real review). > On Nov 26, 2018, at 4:39 PM, Roman Kennke wrote: > > Sections to review (at this point) are the following: > > *) shared-gc > - This is mostly boilerplate that is common to any GC > - referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1) Ick. But I don?t have a better suggestion that doesn?t involve adding a new API to CollectedHeap for use by this assertion, which seems a bit excessive if there are no other uses. > - taskqueue.hpp has some small adjustments to enable subclassing Why this change instead of JDK-8204947? As the description from that RFE says: "The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.? From dean.long at oracle.com Tue Nov 27 00:05:30 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 26 Nov 2018 16:05:30 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> Message-ID: <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> How does this solve the problem of HotSpotJVMCIRuntime.adjustCompilationLevel being called? I don't think this fix is the right approach.? Doesn't is_interp_only_mode() only apply to the current thread?? I don't think it's safe to assume no compiles will happen in other threads, or that a method call target is not already compiled, because as far as I can tell, JVMTI only deoptimizes the active frames.? The bug report describes a case where the caller has been deoptimized while resolving a call.? I don't see how this fix prevents the target from being a previously compiled method.? But we do need to make sure not to call into compiled code, so I think the fix needs to be in code like SharedRuntime::resolve_static_call_C(), where it returns get_c2i_entry() if is_interp_only_mode() is true.? However, there is still another problem.? By allowing JVMTI to suspend the thread during call setup, but reporting the frame as still in the caller instead of the callee, we confuse JVMTI into thinking that execution will resume in the caller instead of the callee.? We may want to restrict where we offer JVMTI suspend points, and not offer a JVMTI suspend point in SharedRuntime::resolve_static_call_C and friends at all. dl On 11/26/18 11:14 AM, Alex Menkov wrote: > Hi all, > > Please review the fix for > https://bugs.openjdk.java.net/browse/JDK-8195639 > webrev: > http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ > > Description: > The test suspends a thread, turns on single stepping and then calls > PopFrame. SingleStep event is expected as soon as the thread is > resumed and PopFrame is processed (so we have call stack with the > depth 1 less than it was before PopFrame). Instead SingleStep event is > received with much deeper call stack (and PopFrame processes wrong > frame). > Research shown that this is caused by missed deoptimization of the > current frame. > As far as I understand CompilationPolicy::event should handle the case > when the thread has is_interp_only_mode() set, but > TieredThresholdPolicy::event checks this only then CompLevel is > CompLevel_none. > CompilerRuntime always calls policy()->event with CompLevel == > CompLevel_aot. > > The fix looks quite simple, but I'd appreciate feedback from runtime > and compiler teams as I'm not sure I completely understand all the > details of the "PopFrame dance". > > --alex From vladimir.kozlov at oracle.com Tue Nov 27 00:24:37 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 26 Nov 2018 16:24:37 -0800 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> Message-ID: <5d5cfdeb-6aaa-4576-a26d-b79efe2efdd5@oracle.com> I am fine with changes for copy constructor. Do we have other cases where it is used? I think the original use in [1] could be removed because I doubt the hack will have performance benefits for such big methods as stated in comment. May be we would not need it copy constructor then. For assignment operator I have reservation. You can't use original allocation type if you can't guaranty that destination is the same type. Thanks, Vladimir http://hg.openjdk.java.net/jdk/jdk/file/3db8758f0f79/src/hotspot/share/classfile/classFileParser.cpp#l135 On 11/18/18 10:14 PM, Kim Barrett wrote: > Please review this fix of the debug-only copy constructor and > assignment operator for ResourceObj, along with adding some missing > uses of the copy constructor. > > The missing copy constructor uses were found by enabling -Wextra. > > The ResourceObj debug-only copy constructor should behave exactly the > same as the corresponding default constructor. That is, the setup for > checking the allocation state in the destructor and operator delete > should be the same whether in the normal or copy constructor. That > previously wasn't true, resulting in assert failures. > > The ResourceObj assignment operator also should not be assuming the > target of the assignment was stack/embeded allocated; that's just not > a valid assumption. The corresponding assertion has been removed. > > Note that the ResourceObj allocation/construction/deletion code is > oddly indented, using an initial 4 space indentation and 2 spaces for > further indenting, unlike the usual consistent 2 space indentation > used elsewhere in HotSpot, including elsewhere in allocation.cpp. > I've left that indentation intact to minimize diffs in some views for > review. Unless there are objections, I plan to fix the indentation. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8213481 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ > > Testing: > mach5 tier1-3. There were many failures in these tiers with just the > addition of the missing copy constructor calls (JDK-8213414). > From zgu at redhat.com Tue Nov 27 01:11:21 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 26 Nov 2018 20:11:21 -0500 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <46CBF9C2-7684-4EED-80F8-7103D48FA3A0@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <46CBF9C2-7684-4EED-80F8-7103D48FA3A0@oracle.com> Message-ID: <3b06a452-2119-d2d2-20b4-b8fdc75fec67@redhat.com> Hi Kim, > >> - taskqueue.hpp has some small adjustments to enable subclassing > > Why this change instead of JDK-8204947? As the description from that RFE says: > "The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.? > We were busy on preparing Shenandoah for upstreaming, this did take a back seat. I am going to start to work on moving ShenandoahTaskTerminator to share tomorrow, but it should not hold up the review. Thanks, -Zhengyu From kim.barrett at oracle.com Tue Nov 27 01:36:59 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 26 Nov 2018 20:36:59 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <5d5cfdeb-6aaa-4576-a26d-b79efe2efdd5@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <5d5cfdeb-6aaa-4576-a26d-b79efe2efdd5@oracle.com> Message-ID: <16AF8FFF-738C-4AD7-B8DC-6690B0DCD402@oracle.com> > On Nov 26, 2018, at 7:24 PM, Vladimir Kozlov wrote: Thanks for looking at this. > I am fine with changes for copy constructor. > > Do we have other cases where it is used? I think the original use in [1] could be removed because I doubt the hack will have performance benefits for such big methods as stated in comment. May be we would not need it copy constructor then. There are other uses. For example, StackMapFrame gets copied (see StackMapFrame::copy). That's why I deleted the "Used in" comment. I don't have an informed opinion about the benefit (or lack thereof) of the copy in parse_constant_pool_entries. This copy is using the implicit copier for the stream class, but is being allocated on the stack so the old copier worked. > For assignment operator I have reservation. You can't use original allocation type if you can't guaranty that destination is the same type. I'm not sure what you mean here. The assignment destination was allocated somewhere, and the assignment doesn't change where it was allocated. And that location has nothing to do with where the assignment source was allocated. So copying the allocation information would be wrong, and was not being done by the old code; I didn't change that. What I changed was removing the assertion that the assignment destination was stack/member allocated, since I don't think that's a correct or reasonable assumption (though apparently there aren't any counter examples in current usage). > Thanks, > Vladimir > > http://hg.openjdk.java.net/jdk/jdk/file/3db8758f0f79/src/hotspot/share/classfile/classFileParser.cpp#l135 > > On 11/18/18 10:14 PM, Kim Barrett wrote: >> [?] CR: >> https://bugs.openjdk.java.net/browse/JDK-8213481 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ >> Testing: >> mach5 tier1-3. There were many failures in these tiers with just the >> addition of the missing copy constructor calls (JDK-8213414). From david.holmes at oracle.com Tue Nov 27 01:39:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 27 Nov 2018 11:39:15 +1000 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> Message-ID: <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> Hi Kim, First thanks for the offline education on the nuances of copy constructors! On 19/11/2018 4:14 pm, Kim Barrett wrote: > Please review this fix of the debug-only copy constructor and > assignment operator for ResourceObj, along with adding some missing > uses of the copy constructor. > > The missing copy constructor uses were found by enabling -Wextra. > > The ResourceObj debug-only copy constructor should behave exactly the > same as the corresponding default constructor. That is, the setup for > checking the allocation state in the destructor and operator delete > should be the same whether in the normal or copy constructor. That > previously wasn't true, resulting in assert failures. That all seems fine to me. > The ResourceObj assignment operator also should not be assuming the > target of the assignment was stack/embeded allocated; that's just not > a valid assumption. The corresponding assertion has been removed. Isn't that enforcing a usage requirement of ResourceObj? C-heap/resource-area/arena allocated ResourceObj should only be used via pointers - assigning one to another makes no sense to me. Thanks, David ----- > Note that the ResourceObj allocation/construction/deletion code is > oddly indented, using an initial 4 space indentation and 2 spaces for > further indenting, unlike the usual consistent 2 space indentation > used elsewhere in HotSpot, including elsewhere in allocation.cpp. > I've left that indentation intact to minimize diffs in some views for > review. Unless there are objections, I plan to fix the indentation. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8213481 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ > > Testing: > mach5 tier1-3. There were many failures in these tiers with just the > addition of the missing copy constructor calls (JDK-8213414). > From kim.barrett at oracle.com Tue Nov 27 03:15:26 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 26 Nov 2018 22:15:26 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> Message-ID: > On Nov 26, 2018, at 8:39 PM, David Holmes wrote: > > Hi Kim, > > First thanks for the offline education on the nuances of copy constructors! Happy to help. > On 19/11/2018 4:14 pm, Kim Barrett wrote: >> Please review this fix of the debug-only copy constructor and >> assignment operator for ResourceObj, along with adding some missing >> uses of the copy constructor. >> The missing copy constructor uses were found by enabling -Wextra. >> The ResourceObj debug-only copy constructor should behave exactly the >> same as the corresponding default constructor. That is, the setup for >> checking the allocation state in the destructor and operator delete >> should be the same whether in the normal or copy constructor. That >> previously wasn't true, resulting in assert failures. > > That all seems fine to me. Thanks. >> The ResourceObj assignment operator also should not be assuming the >> target of the assignment was stack/embeded allocated; that's just not >> a valid assumption. The corresponding assertion has been removed. > > Isn't that enforcing a usage requirement of ResourceObj? C-heap/resource-area/arena allocated ResourceObj should only be used via pointers - assigning one to another makes no sense to me. I don?t see any reason for such an artificial restriction, and think it might prevent reasonable uses. I don?t have a specific use-case in mind, but the natural result is observed from doing the natural thing. From david.holmes at oracle.com Tue Nov 27 03:21:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 27 Nov 2018 13:21:07 +1000 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> Message-ID: <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> On 27/11/2018 1:15 pm, Kim Barrett wrote: >> On Nov 26, 2018, at 8:39 PM, David Holmes wrote: >> >> Hi Kim, >> >> First thanks for the offline education on the nuances of copy constructors! > > Happy to help. > >> On 19/11/2018 4:14 pm, Kim Barrett wrote: >>> Please review this fix of the debug-only copy constructor and >>> assignment operator for ResourceObj, along with adding some missing >>> uses of the copy constructor. >>> The missing copy constructor uses were found by enabling -Wextra. >>> The ResourceObj debug-only copy constructor should behave exactly the >>> same as the corresponding default constructor. That is, the setup for >>> checking the allocation state in the destructor and operator delete >>> should be the same whether in the normal or copy constructor. That >>> previously wasn't true, resulting in assert failures. >> >> That all seems fine to me. > > Thanks. > >>> The ResourceObj assignment operator also should not be assuming the >>> target of the assignment was stack/embeded allocated; that's just not >>> a valid assumption. The corresponding assertion has been removed. >> >> Isn't that enforcing a usage requirement of ResourceObj? C-heap/resource-area/arena allocated ResourceObj should only be used via pointers - assigning one to another makes no sense to me. > > I don?t see any reason for such an artificial restriction, and think it might prevent reasonable uses. > I don?t have a specific use-case in mind, but the natural result is observed from doing the natural thing. What is the "natural result" here? I don't see (and Vladimir seems to share a similar view) how it makes any sense to assign non stack/embedded resource objects to each other? And mixing assignment across different types of ResourceObj makes no sense to me at all. I'm not even sure there's a usecase for stack/embedded but that at least seem a consistent usage. David From jcbeyler at google.com Tue Nov 27 03:27:53 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 26 Nov 2018 19:27:53 -0800 Subject: RFR : [XXS] 8214297: testbug : missing fclose in jvmti_aod.cpp In-Reply-To: <53fbebce-714b-6396-2e3a-d796d663af81@oracle.com> References: <53fbebce-714b-6396-2e3a-d796d663af81@oracle.com> Message-ID: Looks good to me as well, in a similar style of bug for the same method; it seems we are leaking classBytes if we go down two of the error paths. Different bug and different fix of course :) Jc On Mon, Nov 26, 2018 at 6:29 AM wrote: > I agree. Looks good and trivial to push now. > thanks, > Coleen > > On 11/26/18 8:15 AM, Thomas St?fe wrote: > > Looks fine Matthias. > > > > ..Thomas > > On Mon, Nov 26, 2018 at 2:12 PM Baesken, Matthias > > wrote: > >> Please review this small test fix. > >> It adds an fclose in an early return case to function > nsk_jvmti_aod_redefineClass . > >> > >> Bug/webrev : > >> > >> https://bugs.openjdk.java.net/browse/JDK-8214297 > >> > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8214297.0/ > >> > >> > >> Thanks ,Matthias > > -- Thanks, Jc From dean.long at oracle.com Tue Nov 27 06:11:38 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Mon, 26 Nov 2018 22:11:38 -0800 Subject: RFR: 8213209: [REDO] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: <5BF2B1F2.8070004@oracle.com> References: <5BF2B1F2.8070004@oracle.com> Message-ID: Looks OK to me too.? The lazy initialization of _has_subklass probably isn't needed anymore, but it does seem to be an improvement. dl On 11/19/18 4:52 AM, Erik ?sterlund wrote: > Hi Coleen, > > Thanks for the review. > > Full webrev: > http://cr.openjdk.java.net/~eosterlund/8213209/webrev.01/ > > Incremental webrev: > http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00_01/ > > On 2018-11-16 22:08, coleen.phillimore at oracle.com wrote: >> >> Erik, thank you for taking this change and making it lock-free. >> >> On 11/16/18 1:12 PM, Erik ?sterlund wrote: >>> Hi, >>> >>> This is a redo of Coleen's earlier patch to clean weak metadata >>> links using the Compile_lock. >>> >>> We noticed the proposed solution could cause deadlocks. At least one >>> of the reasons for that is that the Compile_lock was taken during >>> printing and verification in safepoints. And the Compile_lock is >>> sometimes held while issuing safepoints. The latter problem also >>> forces us to take the Compile_lock before joining the suspendible >>> thread set, to avoid another deadlock situation. But that is really >>> nasty for other reasons. And there have been a few more problems as >>> well. >>> >>> So in this case, we think it is both safer, less error prone, and >>> better performing (although less important) to perform this cleaning >>> in a lock-free fashion rather than trying to dodge all the issues >>> related to the locking protocol. >>> >>> Inserts still require a lock. Therefore, we are dealing with >>> lock-free reads, and lock-free cleaning. The lock-free cleaning is >>> interacting with multiple lock-free readers and a single writer. >>> >>> The single inserter always helps with cleaning the subclass list >>> head before prepending new nodes to the chain. That allows an >>> invariant that the siblings link is never inserted pointing at a >>> Klass that is unloading, which simplifies things a lot. The head is >>> inserted in a classic load + CAS in a loop. >>> >>> Readers require load_acquire when reading the head, due to competing >>> inserts putting new Klass pointers there. The siblings, however, >>> only need relaxed consistency, because they are linked to data >>> strictly older than the head, which has already been acquired. >>> >>> Unlinked entries are all inserted into a purge list, because freeing >>> them immediately is not safe. A global handshake operation is >>> performed, and after that ClassLoaderDataGraph::purge() will go >>> through the list and delete the entries safely. >> >> I don't see the purge list in your change, and it's not needed since >> this is unlinking in place.? I think this is your other change you're >> thinking of. > > Yes you are right, of course. It's the unloading list and not a > separate purge list... > >> Should the Compile_lock be removed in ciInstanceKlass.cpp >> ciInstanceKlass* ciInstanceKlass::implementor() { >> >> and in jvmciCompilerToVM.cpp >> C2V_VMENTRY(jobject, getImplementor, (JNIEnv *, jobject, jobject >> jvmci_type)) >> >> The former does a bit more than calling implementor().? Or do you >> want to leave these locks to be conservative? > > While I think your analysis is correct, I would indeed like to leave > them in this RFE to be conservative. > >> ? JVMCIKlassHandle handle(THREAD); >> ? { >> ??? // Need Compile_lock around implementor() >> ??? MutexLocker locker(Compile_lock); >> ??? handle = iklass->implementor(); >> ? } >> >> >> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/oops/instanceKlass.cpp.udiff.html >> >> >> nof_implementors() has an assert_locked_or_safepoint(Compile_lock); >> which it no longer needs.? You'll hit this from printing because >> you've removed the locking here: >> >> 3093?? if (is_interface()) { >> 3094 MutexLocker ml(Compile_lock); >> 3095???? st->print_cr(BULLET"nof implementors:? %d", >> nof_implementors()); >> 3096???? if (nof_implementors() == 1) { >> 3097?????? st->print_cr(BULLET"implementor:??? "); >> 3098?????? st->print("?? "); >> 3099?????? implementor()->print_value_on(st); >> 3100?????? st->cr(); >> 3101???? } >> >> >> We don't test printing very well. >> > > Ahh yes - good catch. I removed the assert as it is no longer required. > >> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/utilities/vmError.cpp.udiff.html >> >> >> This file is unchanged except a line removed. > > I put it back the way it was before. > >> This change looks good.? I've run through the tests with it and >> didn't find any problems that doomed my earlier version. > > Thank you for running additional tests on this patch! > > Thanks, > /Erik > >> Thanks!! >> Coleen >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8213209 >>> >>> Thanks, >>> /Erik >> > From erik.osterlund at oracle.com Tue Nov 27 07:15:39 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 27 Nov 2018 08:15:39 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <5BF81BBC.2060606@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> <5BF423D2.1070800@oracle.com> <5BF819B1.7040908@oracle.com> <43818dbd-5008-b3a0-7045-2d314e47f5fa@oracle.com> <5BF81BBC.2060606@oracle.com> Message-ID: <86d894d1-6a32-3a1a-01e9-1919f1efceae@oracle.com> Hi, Small fix to add new constructor for nmethod to allow making dummy nmethods for the test_dependencyContext.cpp that no longer works with not properly constructed dummy nmethods due to requiring a vtable to call is_unloading() on them. Incremental: http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01_02/ Full: http://cr.openjdk.java.net/~eosterlund/8213565/webrev.02/ Still need one more reviewer for this change. /Erik On 2018-11-23 16:24, Erik ?sterlund wrote: > Hi Robbin, > > Thanks for the review. > > /Erik > > On 2018-11-23 16:26, Robbin Ehn wrote: >> Looks good, thanks! >> >> /Robbin >> >> On 2018-11-23 16:16, Erik ?sterlund wrote: >>> Hi, >>> >>> Here is an updated version with some minor adjustments based on >>> feedback from Robbin Ehn. >>> >>> Incremental: >>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00_01/ >>> >>> Full: >>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01/ >>> >>> Thanks, >>> /Erik >>> >>> On 2018-11-20 16:10, Erik ?sterlund wrote: >>>> Ping. >>>> >>>> /Erik >>>> >>>> On 2018-11-12 23:02, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> In my change 8209189, I moved a lot of code around relating to code >>>>> cache unloading. Unfortunately, I failed to see that when making >>>>> nmethods unloaded, and their dependents are flushed... they are not >>>>> immediately removed. Instead, they are tagged to have stale >>>>> entries, that are later cleaned during weak metadata cleaning of >>>>> InstanceKlass. As for method handles, they leak instead, with some >>>>> hacks to reduce the leaking by expunging entries while adding and >>>>> removing. >>>>> >>>>> Obviously, with InstanceKlass cleanup and code cache unloading now >>>>> possibly running in parallel after 8209189, things can go wrong. >>>>> These two phases used >>>>> to be separated by a "wait barrier" for the worker threads >>>>> preventing that parallelism. >>>>> >>>>> The reason dependency contexts were not cleaned when they were found >>>>> during code cache cleaning, is because it was not thread safe when >>>>> code >>>>> cache unloading was made parallel instead of serial. But now that >>>>> we are >>>>> implementing concurrent class unloading, we are going to need >>>>> concurrent >>>>> cleanup of dependency contexts anyway. Therefore I will propose a >>>>> bug fix that fixes the problem in a way that works for both serial, >>>>> parallel and concurrent class unloading. >>>>> >>>>> The solution is to during code cache unloading claim cleaning of >>>>> encountered stale dependency contexts, and clean them straight away. >>>>> Entries are unlinked in a lock-free fashion, and placed on a purge >>>>> list that is walked and deleted during >>>>> ClassLoaderDataGraph::purge(). This follows the thinking of first >>>>> unlinking, then syncing all threads, and then purging. >>>>> >>>>> New accessors for the head and next links, hide is_unloading >>>>> entries and helps unlinking them and putting them into the purge list. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8213565 >>>>> >>>>> Thanks, >>>>> /Erik >>>> >>> > From goetz.lindenmaier at sap.com Tue Nov 27 08:09:57 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 27 Nov 2018 08:09:57 +0000 Subject: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> Message-ID: Hi Kim, I put this into our build system to assure our compilers grok this code. AIX is building already, all other platforms will build tonight (European night??). I think xlc is the most likely to cause issues, On linux we are on gcc 7.3 anyways, which should be fine. Best regards, Goetz. > -----Original Message----- > From: hotspot-dev On Behalf Of > Kim Barrett > Sent: Monday, November 19, 2018 7:14 AM > To: hotspot-dev developers > Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot > > Please review this fix of the debug-only copy constructor and > assignment operator for ResourceObj, along with adding some missing > uses of the copy constructor. > > The missing copy constructor uses were found by enabling -Wextra. > > The ResourceObj debug-only copy constructor should behave exactly the > same as the corresponding default constructor. That is, the setup for > checking the allocation state in the destructor and operator delete > should be the same whether in the normal or copy constructor. That > previously wasn't true, resulting in assert failures. > > The ResourceObj assignment operator also should not be assuming the > target of the assignment was stack/embeded allocated; that's just not > a valid assumption. The corresponding assertion has been removed. > > Note that the ResourceObj allocation/construction/deletion code is > oddly indented, using an initial 4 space indentation and 2 spaces for > further indenting, unlike the usual consistent 2 space indentation > used elsewhere in HotSpot, including elsewhere in allocation.cpp. > I've left that indentation intact to minimize diffs in some views for > review. Unless there are objections, I plan to fix the indentation. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8213481 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ > > Testing: > mach5 tier1-3. There were many failures in these tiers with just the > addition of the missing copy constructor calls (JDK-8213414). From rwestrel at redhat.com Tue Nov 27 08:36:29 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Tue, 27 Nov 2018 09:36:29 +0100 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <9D626E11-3115-47C8-BF70-A09B14B7FC52@amazon.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> <9D626E11-3115-47C8-BF70-A09B14B7FC52@amazon.com> Message-ID: <87zhtunche.fsf@redhat.com> > Also, it's not just intrinsics that have this problem. All counted > loops should be "strip-mined" this way. Android JVMs have been doing > it for time out of mind because they budget 2-3ms max per 16.67ms > frame computation time for stw events. C2 has loop strip mining now: https://bugs.openjdk.java.net/browse/JDK-8186027 Roland. From rkennke at redhat.com Tue Nov 27 09:38:31 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Nov 2018 10:38:31 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> Message-ID: Thanks for reviewing, Erik! Roman > Build changes look ok to me. > > /Erik > > On 2018-11-26 13:39, Roman Kennke wrote: >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the mailing >> lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is to >> eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> ?? https://openjdk.java.net/jeps/189 >> Bug entry: >> >> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> >> which is what I intend to commit (and which should be equivalent to the >> 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> ??*) shenandoah-gc >> >> >> ???? - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> ??*) shared-gc >> >> ???? - This is mostly boilerplate that is common to any GC >> ???? - referenceProcessor.cpp has a little change to make one assert not >> fail (next to CMS and G1) >> ???? - taskqueue.hpp has some small adjustments to enable subclassing >> >> ??*) shared-serviceability >> >> >> ???? - The usual code to support another GC >> >> ??*) shared-runtime >> >> >> ???? - A number of friends declarations to allow Shenandoah iterators to >> hook up with, >> ?????? e.g. ClassLoaderData, CodeCache, etc >> ???? - Warning and disabling JFR LeakProfiler >> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> ???? - Locks initialization in mutexLocker.cpp as usual >> ???? - VM operations defines for Shenandoah's VM ops >> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> ???? - The usual macros in macro.hpp >> >> ??*) shared-build >> >> >> ???? - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> ???? - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> ?????? useful to get the whole marking loop inlined (observed significant >> regression if we >> ?????? don't) >> >> ??*) shared-tests >> >> >> ???? - Test infrastructure to support Shenandoah >> ???? - Shenandoah test groups >> ???? - Exclude Shenandoah in various tests that can be run with >> selected GC >> ???? - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> ??*) shenandoah-tests >> >> >> ???? - Shenandoah specific tests, most reside in gc/shenandoah >> subdirectory >> ???? - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case you >> want to take >> a peek at it. >> >> We have regular builds on: >> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> ?? - {Windows} x {x86_64}, >> ?? - {MacOS X} x {x86_64} >> >> This also routinely passes: >> ?? - the new Shenandoah tests >> ?? - jcstress with/without aggressive Shenandoah verification >> ?? - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >> the credit for being the original inventor of Shenandoah, Aleksey >> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface and >> related changes, and of course the many early adopters for reporting >> bugs and success stories and feature requests: we wouldn't be here >> without any of you! >> >> Best regards, >> Roman >> From rkennke at redhat.com Tue Nov 27 09:37:57 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Nov 2018 10:37:57 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> Message-ID: <4650c622-c937-4a76-3b9f-10f93c677dcc@redhat.com> Hi Vladimir, > You need to check if shenandoahgc is disabled first - check > DISABLED_JVM_FEATURES list (see code for jvmci). Why? If Shenandoah is disabled, it will be on this list, and therefore not be built (see JvmFeatures.gmk). > Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check. > Do you support all x86 OSs? We can build with 32bit, but it will not run. It seems indeed curious to enable it. It's probably only there to allow 32bit builds with Shenandoah enabled, just to verify that we have all the relevant LP64 checks in code in place. I will check it with my collegues. > Why you set VM CFLAGS when shenandoahgc is not enabled? It is in > JvmFeatures.gmk. This *disables* and excludes Shenandoah if not enabled. +ifneq ($(call check-jvm-feature, shenandoahgc), true) + JVM_CFLAGS_FEATURES += -DINCLUDE_SHENANDOAHGC=0 + JVM_EXCLUDE_PATTERNS += gc/shenandoah pretty much the same pattern as zgc and other features. and then we add some flags when Shenandoah is enabled: +else + JVM_CFLAGS_FEATURES += -DSUPPORT_BARRIER_ON_PRIMITIVES -DSUPPORT_NOT_TO_SPACE_INVARIANT +endif ... which are required for building with Shenandoah enabled. > I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc > specific nodes. This looks intrusive. I hope you clean it up. There are a few places with INCLUDE_SHENANDOAHGC checks where it seemed excessive to add a whole API just for a tiny, very GC specific check. We still do have intrusive changes in loop*, which we are working on to resolve. We declare+define all our GC specific nodes in shenandoahBarrierSetC2.hpp and related Shenandoah-specific files. The only things in shared code is the usual declarations in classes.hpp/cpp and node.hpp to get is_ShenandoahBarrier() an similar queries. This seems in-line with what other GCs do (look for e.g. LoadBarrier). Please be specific which changes in opto you'd like to see resolved (and don't look at loop* source files at this point ;-) ). Thanks for reviewing! Roman > Thanks, > Vladimir > > On 11/26/18 2:47 PM, Erik Joelsson wrote: >> Build changes look ok to me. >> >> /Erik >> >> On 2018-11-26 13:39, Roman Kennke wrote: >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the >>> mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ?? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ??*) shenandoah-gc >>> >>> >>> ???? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ??*) shared-gc >>> >>> ???? - This is mostly boilerplate that is common to any GC >>> ???? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >>> >>> ??*) shared-runtime >>> >>> >>> ???? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ?????? e.g. ClassLoaderData, CodeCache, etc >>> ???? - Warning and disabling JFR LeakProfiler >>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ???? - Locks initialization in mutexLocker.cpp as usual >>> ???? - VM operations defines for Shenandoah's VM ops >>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ???? - The usual macros in macro.hpp >>> >>> ??*) shared-build >>> >>> >>> ???? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ???? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ?????? useful to get the whole marking loop inlined (observed >>> significant >>> regression if we >>> ?????? don't) >>> >>> ??*) shared-tests >>> >>> >>> ???? - Test infrastructure to support Shenandoah >>> ???? - Shenandoah test groups >>> ???? - Exclude Shenandoah in various tests that can be run with >>> selected GC >>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ??*) shenandoah-tests >>> >>> >>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>> subdirectory >>> ???? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ?? - {Windows} x {x86_64}, >>> ?? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ?? - the new Shenandoah tests >>> ?? - jcstress with/without aggressive Shenandoah verification >>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> From rkennke at redhat.com Tue Nov 27 09:46:18 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Nov 2018 10:46:18 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <46CBF9C2-7684-4EED-80F8-7103D48FA3A0@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <46CBF9C2-7684-4EED-80F8-7103D48FA3A0@oracle.com> Message-ID: Hi Kim, >> Sections to review (at this point) are the following: >> >> *) shared-gc >> - This is mostly boilerplate that is common to any GC >> - referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1) > > Ick. But I don?t have a better suggestion that doesn?t involve adding a new API > to CollectedHeap for use by this assertion, which seems a bit excessive if there > are no other uses. Yeah. I guess we could add a config _discovery_is_concurrent or similar in RP, and check that. Or maybe one of _discovery_is_mt or _discovery_is_atomic already covers that? I couldn't immediately tell/100% understand their semantics. Seems worthy to look at after this? >> - taskqueue.hpp has some small adjustments to enable subclassing > > Why this change instead of JDK-8204947? As the description from that RFE says: > "The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.? Yeah, see Zhengyu's comment. Let's ignore those changes for this review (for now), expect our impl ported to taskqueue.hpp/cpp soon. Roman From aph at redhat.com Tue Nov 27 10:41:31 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 27 Nov 2018 10:41:31 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> Message-ID: On 11/26/18 10:01 PM, Hohensee, Paul wrote: > The compiler could estimate the cost of an iteration using some rough pessimistic heuristic and strip mine based on that. But the compiler doesn't see the intrinsics: they are hand-coded assembly language. > Measuring the safepoint time directly is fragile due to inability to find all the hardware platforms that might run the JVM. A671 Sure, but all I'm suggesting is that we take an order-of-magnitude guess and apply that to our intrinsics. It does not have to be accurate: in the case of encryption the speed difference caused by limiting the number of iterations is infinitesimal, but the time to safepoint difference is huge. IMO, this is a true no brainer. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rkennke at redhat.com Tue Nov 27 10:52:21 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Nov 2018 11:52:21 +0100 Subject: Call for reviews: JEP 189: Shenandoah: A Low-Pause Garbage Collector Message-ID: Hi there, I just noticed that we have only one 'Reviewed-by:' entry in the Shenandoah JEP. I know that several people have reviewed it in the past, as I've incorporated several suggestions into the JEP. I believe the JEP cannot move forward to 'proposed to target' without at least one more added reviewer. https://bugs.openjdk.java.net/browse/JDK-8046179 If you have reviewed the JEP in the past, and/or feel like reviewing it now, please don't forget to add your 'Reviewed-by:' entry. I believe any Reviewer familiar with the relevant field of expertise (GC and/or Hotspot in this case) can review it and add his name to it. Thanks! Roman From goetz.lindenmaier at sap.com Tue Nov 27 11:51:18 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 27 Nov 2018 11:51:18 +0000 Subject: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> Message-ID: Hi, I built aix slowdebug, fastdebug and product builds successfully. The other builds run tonight. Best regards, Goetz. > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Tuesday, November 27, 2018 9:10 AM > To: 'Kim Barrett' ; hotspot-dev developers > > Subject: RE: 8213481: [REDO] Fix incorrect copy constructors in hotspot > > Hi Kim, > > I put this into our build system to assure our compilers grok this code. > AIX is building already, all other platforms will build tonight > (European night??). I think xlc is the most likely to cause issues, > On linux we are on gcc 7.3 anyways, which should be fine. > > Best regards, > Goetz. > > > > > > > -----Original Message----- > > From: hotspot-dev On Behalf > Of > > Kim Barrett > > Sent: Monday, November 19, 2018 7:14 AM > > To: hotspot-dev developers > > Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot > > > > Please review this fix of the debug-only copy constructor and > > assignment operator for ResourceObj, along with adding some missing > > uses of the copy constructor. > > > > The missing copy constructor uses were found by enabling -Wextra. > > > > The ResourceObj debug-only copy constructor should behave exactly the > > same as the corresponding default constructor. That is, the setup for > > checking the allocation state in the destructor and operator delete > > should be the same whether in the normal or copy constructor. That > > previously wasn't true, resulting in assert failures. > > > > The ResourceObj assignment operator also should not be assuming the > > target of the assignment was stack/embeded allocated; that's just not > > a valid assumption. The corresponding assertion has been removed. > > > > Note that the ResourceObj allocation/construction/deletion code is > > oddly indented, using an initial 4 space indentation and 2 spaces for > > further indenting, unlike the usual consistent 2 space indentation > > used elsewhere in HotSpot, including elsewhere in allocation.cpp. > > I've left that indentation intact to minimize diffs in some views for > > review. Unless there are objections, I plan to fix the indentation. > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8213481 > > > > Webrev: > > http://cr.openjdk.java.net/~kbarrett/8213481/open.00/ > > > > Testing: > > mach5 tier1-3. There were many failures in these tiers with just the > > addition of the missing copy constructor calls (JDK-8213414). From rkennke at redhat.com Tue Nov 27 12:19:43 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Nov 2018 13:19:43 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> Message-ID: <17d4fb47-c5c3-5d13-e318-ac1b62a88815@redhat.com> > Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check. > > Do you support all x86 OSs? We enable this because we can actually run on such hardware. We fall back to 'passive' mode though, which means Shenandoah acts more or less like Parallel GC, and doesn't involve any barriers at all, and doesn't do any concurrent processing. This has been useful for footprint experiments. Roman > Why you set VM CFLAGS when shenandoahgc is not enabled? It is in > JvmFeatures.gmk. > > I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc > specific nodes. This looks intrusive. I hope you clean it up. > > Thanks, > Vladimir > > On 11/26/18 2:47 PM, Erik Joelsson wrote: >> Build changes look ok to me. >> >> /Erik >> >> On 2018-11-26 13:39, Roman Kennke wrote: >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the >>> mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ?? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ??*) shenandoah-gc >>> >>> >>> ???? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ??*) shared-gc >>> >>> ???? - This is mostly boilerplate that is common to any GC >>> ???? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >>> >>> ??*) shared-runtime >>> >>> >>> ???? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ?????? e.g. ClassLoaderData, CodeCache, etc >>> ???? - Warning and disabling JFR LeakProfiler >>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ???? - Locks initialization in mutexLocker.cpp as usual >>> ???? - VM operations defines for Shenandoah's VM ops >>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ???? - The usual macros in macro.hpp >>> >>> ??*) shared-build >>> >>> >>> ???? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ???? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ?????? useful to get the whole marking loop inlined (observed >>> significant >>> regression if we >>> ?????? don't) >>> >>> ??*) shared-tests >>> >>> >>> ???? - Test infrastructure to support Shenandoah >>> ???? - Shenandoah test groups >>> ???? - Exclude Shenandoah in various tests that can be run with >>> selected GC >>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ??*) shenandoah-tests >>> >>> >>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>> subdirectory >>> ???? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ?? - {Windows} x {x86_64}, >>> ?? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ?? - the new Shenandoah tests >>> ?? - jcstress with/without aggressive Shenandoah verification >>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> From erik.osterlund at oracle.com Tue Nov 27 13:00:57 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 27 Nov 2018 14:00:57 +0100 Subject: RFR: 8214257: IC cache not clean after cleaning assertion failure Message-ID: Hi, Back in 8212681, an abstract CompiledICLocker was introduce to protect inline caches from concurrent patching. Unfortunately, not all patching of ICs started using the CompiledICLocker. There were still some callsites using the Patching_lock. Therefore, the assert may trigger sometimes saying the IC cache is not clean right after cleaning, due to other concurrent modifications under the Patching_lock. Also, in CompiledIC::set_to_clean(), the condition for safe_transition was altered incorrectly from is_at_safepoint() to CompiledICLocker::is_safe(), causing cleaning to not used transition, so I reinstated that in this patch. This patch depends on 8214338 by assuming that the CompiledICLocker does not use safepoint checks, as it is now used from leaf calls where safepointing is not allowed. Bug: https://bugs.openjdk.java.net/browse/JDK-8214257 Webrev: http://cr.openjdk.java.net/~eosterlund/8214257/webrev.00/ Thanks, /Erik From erik.osterlund at oracle.com Tue Nov 27 13:00:15 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 27 Nov 2018 14:00:15 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions Message-ID: Hi, IC stub refilling currently requires a safepoint operation. When done right at the point where an CompiledIC is about to get patched to a transitional state using an IC stub, locks may already be held, causing a bunch of locking issues - especially for concurrent class unloading. Therefore, the IC stub refilling ought to be moved out so that IC cache cleaning and transitioning may be done without any safepoints, and the locks in the path ought to not perform safepoint checking. This is implemented by allowing IC transitions to fail when they require IC stubs, and we run out of them. This propages back to a higher level where IC stubs are refilled after having released the IC locker. Webrev: http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8214338 Thanks, /Erik From jcbeyler at google.com Tue Nov 27 14:56:10 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 27 Nov 2018 06:56:10 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> Message-ID: Thanks Chris, Anybody else motivated to look at this and review it? :) Jc On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer wrote: > Hi JC, > > I'm ok with the FatalError approach, but would like to hear opinions from > others also. > > thanks, > > Chris > > On 11/21/18 8:19 AM, JC Beyler wrote: > > Hi Chris, > > Thanks for taking the time to look at it and yes you have raised exactly > why the webrev is between two worlds: in cases where a fatal error on > failure is wanted, should we simplify the code to remove the return tests > since we do them internally? Now that I've looked around for non-fatal > cases, I think the answer is yes, it simplifies the code while maintaining > the checks. > > I looked a bit and it seems that I can't find easily a case where the test > accepts a JNI failure to then move on. Therefore, perhaps, for now, the > fail with a Fatal is enough and we can work on the tests to clean them up? > > That means that this is the new webrev with only Fatal and cleans up the > tests so that it is no longer in between two worlds: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 > > (This passes testing on my dev machine for all the modified tests) > > with the example you provided, it now looks like: > > http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html > > Where it does, to me at least, seem cleaner and less "noisy". > > Let me know what you think, > Jc > > > On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer > wrote: > >> Hi JC, >> >> Sorry about the delay. I had to go back an look at the initial 8210842 >> webrev and RFR thread to see what this was initially all about. >> >> In general the changes look good. >> >> I don't have a good answer to your FatalError/NonFatalError question. It >> makes the code a lot cleaner to use FatalError, but then it is a behavior >> change, and you also need to deal with tests that intentionally induce >> errors (do you have an example of that). >> >> In any case, right now your webrev seems to be between two worlds. You >> are producing FatalError, but still checking results. Here's a good example: >> >> >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> >> I'm not sure if this is just a temporary state until it was decided which >> approach to take. >> >> thanks, >> >> Chris >> >> >> On 11/20/18 2:14 PM, JC Beyler wrote: >> >> Hi all, >> >> Chris thought it made sense to have more eyes on this change than just >> serviceability as it will modify to tests that are not only serviceability >> tests so I've moved this to conversation here :) >> >> For convenience, I've copy-pasted the initial RFR: >> >> Could I have a review for the extension and usage of the >> ExceptionJniWrapper. This adds lines and filenames to the end of the >> wrapper JNI methods, adds tracing, and throws an error if need be. I've >> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >> ported a few of the tests that were already changed for the assignment >> webrev for JDK-8212884. >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> >> For illustration, if I force an error to the AP04/ap04t03 test and set >> the verbosity on, I get something like: >> >> >> Calling JNI method FindClass from ap04t003.cpp:343 >> >> Calling with these parameter(s): >> java/lang/Threadd >> Wait for thread to finish >> << Called JNI method FindClass from ap04t003.cpp:343 >> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >> java/lang/Threadd >> at >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> Method) >> at >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> at >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >> at >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >> at >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >> ... 3 more >> FATAL ERROR in native method: JNI method FindClass : internal error from >> ap04t003.cpp:343 >> at >> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >> Method) >> at >> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >> at >> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >> >> Questions/comments I have about this are: >> - Do we want to force fatal errors when a JNI call fails in general? >> Most of these tests do the right thing and test the return of the JNI >> calls, for example: >> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >> if (thrClass == NULL) { >> >> but now the wrapper actually would do a fatal if the FindClass call would >> return a nullptr, so we could remove that test altogether. What do you >> think? >> - I prefer to leave them as the tests then become closer to what real >> users would have in their code and is the "recommended" way of doing it >> >> >> - The alternative is to use the NonFatalError I added which then just >> prints out that something went wrong, letting the test continue. Question >> will be what should be the default? The fatal or the non-fatal error >> handling? >> >> On a different subject: >> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >> wrapper handles the tracing and the verify in almost the same way; only >> difference I can really tell is that the complain method from NSK has a max >> complain before stopping to "complain"; I have not added that part of the >> code in this webrev >> >> Once we decide on these, I can continue on the files from JDK-8212884 and >> then do both the assignment in an if extraction followed-by this type of >> webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can be >> deprecated as well as we go forward. >> >> Thanks! >> Jc >> >> On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer >> wrote: >> >>> On 11/19/18 10:07 AM, JC Beyler wrote: >>> >>> Hi all, >>> >>> @David/Chris: should I then push this RFR to the hotspot mailing or the >>> runtime one? For what it's worth, a lot of the tests under the vmTestbase >>> are jvmti so the review also affects serviceability; it just turns out I >>> started with the GC originally and then hit some other tests I had touched >>> via the assignment extraction. >>> >>> I think hotspot would be best. >>> >>> Chris >>> >>> >>> @Serguei: Done for the method renaming, for the indent, are you talking >>> about going from the 8-indent to 4-indent? If so, would it not just be >>> better to do a new JBS bug and do the whole files in one go? I ask because >>> otherwise, it will look a bit weird to have parts of the file as 8-indent >>> and others 4-indent? >>> >>> Thanks for looking at it! >>> Jc >>> >>> On Mon, Nov 19, 2018 at 1:25 AM serguei.spitsyn at oracle.com < >>> serguei.spitsyn at oracle.com> wrote: >>> >>>> Hi Jc, >>>> >>>> We have to start this review anyway. :) >>>> It looks good to me in general. >>>> Thank you for your consistency in this refactoring! >>>> >>>> Some minor comments. >>>> >>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>> >>>> +static const char* remove_folders(const char* fullname) { >>>> >>>> I'd suggest to rename the function name to something traditional like get_basename. >>>> Otherwise, it sounds like this function has to really remove folders. :) >>>> >>>> >>>> Also, all *Locker.cpp have wrong indent in the bodies of if and while statements. >>>> Could this be fixed with the refactoring? >>>> >>>> I did not look on how this impacts the tests other than serviceability. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 11/16/18 19:43, JC Beyler wrote: >>>> >>>> Hi all, >>>> >>>> Anybody motivated to review this? :) >>>> Jc >>>> >>>> On Wed, Nov 7, 2018 at 9:53 PM JC Beyler wrote: >>>> >>>>> Hi all, >>>>> >>>>> Could I have a review for the extension and usage of the >>>>> ExceptionJniWrapper. This adds lines and filenames to the end of the >>>>> wrapper JNI methods, adds tracing, and throws an error if need be. I've >>>>> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >>>>> ported a few of the tests that were already changed for the assignment >>>>> webrev for JDK-8212884. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>> >>>>> For illustration, if I force an error to the AP04/ap04t03 test and set >>>>> the verbosity on, I get something like: >>>>> >>>>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>>>> >> Calling with these parameter(s): >>>>> java/lang/Threadd >>>>> Wait for thread to finish >>>>> << Called JNI method FindClass from ap04t003.cpp:343 >>>>> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >>>>> java/lang/Threadd >>>>> at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> Method) >>>>> at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >>>>> at >>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>> at >>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>> at >>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>> ... 3 more >>>>> FATAL ERROR in native method: JNI method FindClass : internal error >>>>> from ap04t003.cpp:343 >>>>> at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>> Method) >>>>> at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>> at >>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>> >>>>> Questions/comments I have about this are: >>>>> - Do we want to force fatal errors when a JNI call fails in general? >>>>> Most of these tests do the right thing and test the return of the JNI >>>>> calls, for example: >>>>> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >>>>> if (thrClass == NULL) { >>>>> >>>>> but now the wrapper actually would do a fatal if the FindClass call >>>>> would return a nullptr, so we could remove that test altogether. What do >>>>> you think? >>>>> - I prefer to leave them as the tests then become closer to what >>>>> real users would have in their code and is the "recommended" way of doing it >>>>> >>>>> - The alternative is to use the NonFatalError I added which then >>>>> just prints out that something went wrong, letting the test continue. >>>>> Question will be what should be the default? The fatal or the non-fatal >>>>> error handling? >>>>> >>>>> On a different subject: >>>>> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >>>>> wrapper handles the tracing and the verify in almost the same way; only >>>>> difference I can really tell is that the complain method from NSK has a max >>>>> complain before stopping to "complain"; I have not added that part of the >>>>> code in this webrev >>>>> >>>>> Once we decide on these, I can continue on the files from JDK-8212884 >>>>> and then do both the assignment in an if extraction followed-by this type >>>>> of webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can >>>>> be deprecated as well as we go forward. >>>>> >>>>> Thank you for the reviews/comments :) >>>>> Jc >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>>> >>>> >>>> >>> >>> -- >>> >>> Thanks, >>> Jc >>> >>> >>> >> >> -- >> >> Thanks, >> Jc >> >> >> > > -- > > Thanks, > Jc > > > -- Thanks, Jc From matthias.baesken at sap.com Tue Nov 27 15:08:32 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 27 Nov 2018 15:08:32 +0000 Subject: RFR : [XS] 8214373: adjust usage of ReleaseLongArrayElements in MacosxDebuggerLocal Message-ID: Hello, please review this small fix . In MacosxDebuggerLocal.m , at a few places the functions GetLongArrayElements and ReleaseLongArrayElements are used. However there is one place where in an early return the ReleaseLongArrayElements call has been forgotten . Additionally at another place ReleaseLongArrayElements is at the wrong place (although this is probably not a big issue currently). Bug/webrev: https://bugs.openjdk.java.net/browse/JDK-8214373 http://cr.openjdk.java.net/~mbaesken/webrevs/8214373.0/ Thanks, Matthias From jcbeyler at google.com Tue Nov 27 15:47:42 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 27 Nov 2018 07:47:42 -0800 Subject: RFR : [XS] 8214373: adjust usage of ReleaseLongArrayElements in MacosxDebuggerLocal In-Reply-To: References: Message-ID: Hi Matthias, Not a reviewer but looks good to me :). I checked if other releases were not in the right place and they all seemed good as well. Jc On Tue, Nov 27, 2018 at 7:09 AM Baesken, Matthias wrote: > Hello, please review this small fix . > > In MacosxDebuggerLocal.m , at a few places the functions > GetLongArrayElements and ReleaseLongArrayElements are used. > > However there is one place where in an early return the > ReleaseLongArrayElements call has been forgotten . > > Additionally at another place ReleaseLongArrayElements is at the wrong > place (although this is probably not a big issue currently). > > > Bug/webrev: > > https://bugs.openjdk.java.net/browse/JDK-8214373 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8214373.0/ > > > > Thanks, Matthias > -- Thanks, Jc From ludwig.mark at siemens.com Tue Nov 27 16:24:47 2018 From: ludwig.mark at siemens.com (Ludwig, Mark) Date: Tue, 27 Nov 2018 16:24:47 +0000 Subject: Java AWT, Swing, threads, anon classes, and garbage collection Message-ID: Greetings, Abstract: I'd appreciate pointers to any papers or discussions about how anonymous classes are garbage-collected in current JVMs (Java 9+). Hopefully someone will respond that the historical problem with garbage collection of anonymous classes has been solved. TL;DR My information is over 10 years old, when almost all of the JVMs were 32-bit; we ran into a problem with a coding pattern that constructed anonymous classes more quickly than the garbage collection could handle, and the permanent generation ran out of space, bringing down the VM. At the time, it was running as an Applet, in a browser started by one of our thousands of users, owned by the user's employer (our customer), on a machine tightly configured by our customer's IT department to run multiple applications, where we do not have any ability to inject JVM launch options to, for example, control the size of the permanent generation. It seemed at the time that the garbage collectors didn't even look in the permanent generation very frequently, so there as an architectural mis-match with high-speed construction of anonymous classes. My understanding is that 64-bit JVMs don't have the concept of a permanent generation. Is there a replacement in current JVMs where anonymous classes might still overflow? While the application was usually launched as an Applet, it also has a main() and can run as an application. That's the way it will run, going forward, obviously. Now that we can influence the JVM startup options, I want to know if I should be considering this. The reason is that I want to use this pattern to fix another problem with GUI activity (Java AWT). This fix will spike the construction of anonymous classes significantly.** If there is any likelihood of crashing the JVM (as it would, historically), I want to understand what the risk is. ** The new pattern would use: SwingUtilities.invokeLater(new Runnable() {public void run() {doGUI();}}); Where doGUI() might be as trivial as painting a dot. This thing could resemble a denial-of-service attack on the thread constructor and GUI event thread scheduler, if such a DoS concept existed. As long as garbage collection works, I expect it all should work...? Thanks! Mark Ludwig From thomas.stuefe at gmail.com Tue Nov 27 16:46:38 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 27 Nov 2018 17:46:38 +0100 Subject: RFR : [XS] 8214373: adjust usage of ReleaseLongArrayElements in MacosxDebuggerLocal In-Reply-To: References: Message-ID: Hi Matthias, looks fine. How did you find this, via Coverity? ... Thomas On Tue, Nov 27, 2018 at 4:09 PM Baesken, Matthias wrote: > > Hello, please review this small fix . > > > > In MacosxDebuggerLocal.m , at a few places the functions GetLongArrayElements and ReleaseLongArrayElements are used. > > > > However there is one place where in an early return the ReleaseLongArrayElements call has been forgotten . > > > > Additionally at another place ReleaseLongArrayElements is at the wrong place (although this is probably not a big issue currently). > > > > > > Bug/webrev: > > > > https://bugs.openjdk.java.net/browse/JDK-8214373 > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8214373.0/ > > > > > > > > Thanks, Matthias From adinn at redhat.com Tue Nov 27 17:07:43 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 27 Nov 2018 17:07:43 +0000 Subject: Call for reviews: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: Message-ID: <72d322c7-c997-06cd-5bda-2238b4a46d63@redhat.com> Hi Roman, On 27/11/2018 10:52, Roman Kennke wrote: > I just noticed that we have only one 'Reviewed-by:' entry in the > Shenandoah JEP. I know that several people have reviewed it in the past, > as I've incorporated several suggestions into the JEP. I believe the JEP > cannot move forward to 'proposed to target' without at least one more > added reviewer. > > https://bugs.openjdk.java.net/browse/JDK-8046179 > > If you have reviewed the JEP in the past, and/or feel like reviewing it > now, please don't forget to add your 'Reviewed-by:' entry. I believe any > Reviewer familiar with the relevant field of expertise (GC and/or > Hotspot in this case) can review it and add his name to it. I have reviewed this JEP. It would be nice to have a 3rd review from one of of the Oracle GC devs but I don't suppose that is critical. regards, Andrew Dinn ----------- From vladimir.kozlov at oracle.com Tue Nov 27 17:31:57 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 27 Nov 2018 09:31:57 -0800 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <4650c622-c937-4a76-3b9f-10f93c677dcc@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> <4650c622-c937-4a76-3b9f-10f93c677dcc@redhat.com> Message-ID: On 11/27/18 1:37 AM, Roman Kennke wrote: > Hi Vladimir, > >> You need to check if shenandoahgc is disabled first - check >> DISABLED_JVM_FEATURES list (see code for jvmci). > > Why? If Shenandoah is disabled, it will be on this list, and therefore > not be built (see JvmFeatures.gmk). Did you test with --with-jvm-variants=-shenandoahgc to make sure it is disabled? May be I needed explicitly check jvmci because I need it early to check it for enable Graal build. So it is different from your case. > >> Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check. >> Do you support all x86 OSs? > > We can build with 32bit, but it will not run. It seems indeed curious to > enable it. It's probably only there to allow 32bit builds with > Shenandoah enabled, just to verify that we have all the relevant LP64 > checks in code in place. I will check it with my collegues. What about OS? Do you support Windows, MacOS? I did not see any OS specific changes. So may be it is alright. > >> Why you set VM CFLAGS when shenandoahgc is not enabled? It is in >> JvmFeatures.gmk. > > This *disables* and excludes Shenandoah if not enabled. > > +ifneq ($(call check-jvm-feature, shenandoahgc), true) > + JVM_CFLAGS_FEATURES += -DINCLUDE_SHENANDOAHGC=0 > + JVM_EXCLUDE_PATTERNS += gc/shenandoah > > pretty much the same pattern as zgc and other features. > > and then we add some flags when Shenandoah is enabled: > > +else > + JVM_CFLAGS_FEATURES += -DSUPPORT_BARRIER_ON_PRIMITIVES > -DSUPPORT_NOT_TO_SPACE_INVARIANT > +endif > > ... which are required for building with Shenandoah enabled. My bad. Somehow I thought it was reverse. Too much coffee at the morning. :( Looks good. > >> I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc >> specific nodes. This looks intrusive. I hope you clean it up. > > There are a few places with INCLUDE_SHENANDOAHGC checks where it seemed > excessive to add a whole API just for a tiny, very GC specific check. We > still do have intrusive changes in loop*, which we are working on to > resolve. We declare+define all our GC specific nodes in > shenandoahBarrierSetC2.hpp and related Shenandoah-specific files. The > only things in shared code is the usual declarations in classes.hpp/cpp > and node.hpp to get is_ShenandoahBarrier() an similar queries. This > seems in-line with what other GCs do (look for e.g. LoadBarrier). Please > be specific which changes in opto you'd like to see resolved (and don't > look at loop* source files at this point ;-) ). Is it possible to reuse LoadBarrier by adding GC specific flag to it? I am not against adding new nodes if really needed. My concern is about using GC name in node's names. Yes, I am fine with very few INCLUDE_SHENANDOAHGC. Thanks, Vladimir > > Thanks for reviewing! > Roman > >> Thanks, >> Vladimir >> >> On 11/26/18 2:47 PM, Erik Joelsson wrote: >>> Build changes look ok to me. >>> >>> /Erik >>> >>> On 2018-11-26 13:39, Roman Kennke wrote: >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the >>>> mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ?? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ??*) shenandoah-gc >>>> >>>> >>>> ???? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ??*) shared-gc >>>> >>>> ???? - This is mostly boilerplate that is common to any GC >>>> ???? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ??*) shared-serviceability >>>> >>>> >>>> ???? - The usual code to support another GC >>>> >>>> ??*) shared-runtime >>>> >>>> >>>> ???? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ?????? e.g. ClassLoaderData, CodeCache, etc >>>> ???? - Warning and disabling JFR LeakProfiler >>>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ???? - Locks initialization in mutexLocker.cpp as usual >>>> ???? - VM operations defines for Shenandoah's VM ops >>>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ???? - The usual macros in macro.hpp >>>> >>>> ??*) shared-build >>>> >>>> >>>> ???? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ???? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ?????? useful to get the whole marking loop inlined (observed >>>> significant >>>> regression if we >>>> ?????? don't) >>>> >>>> ??*) shared-tests >>>> >>>> >>>> ???? - Test infrastructure to support Shenandoah >>>> ???? - Shenandoah test groups >>>> ???? - Exclude Shenandoah in various tests that can be run with >>>> selected GC >>>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ??*) shenandoah-tests >>>> >>>> >>>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>>> subdirectory >>>> ???? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ?? - {Windows} x {x86_64}, >>>> ?? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ?? - the new Shenandoah tests >>>> ?? - jcstress with/without aggressive Shenandoah verification >>>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> > From Michael.Rasmussen at roguewave.com Tue Nov 27 17:44:32 2018 From: Michael.Rasmussen at roguewave.com (Michael Rasmussen) Date: Tue, 27 Nov 2018 17:44:32 +0000 Subject: Java AWT, Swing, threads, anon classes, and garbage collection In-Reply-To: References: Message-ID: The concept of an anonymous inner class as it exists on the language level doesn't really exist in the JVM. At compile time an anonymous inner class gets compiled into a .class file usually with a name like OuterClass$1, OuterClass$2 etc. So there is only one Class for the anonymous class (including in permgen/metaspace), but as many instances as instantiated -- and the GCs today are very good at handling these very short-lived objects. Even if using a lambda, the lambda meta factory will create just one class for the lambda, and instances of that class with be instantiated as needed (for a non-capturing lambda, it can even reuse the same instance). /Michael ________________________________ From: hotspot-dev on behalf of Ludwig, Mark Sent: 27 November 2018 18:24:47 To: hotspot-dev at openjdk.java.net Subject: Java AWT, Swing, threads, anon classes, and garbage collection Greetings, Abstract: I'd appreciate pointers to any papers or discussions about how anonymous classes are garbage-collected in current JVMs (Java 9+). Hopefully someone will respond that the historical problem with garbage collection of anonymous classes has been solved. TL;DR My information is over 10 years old, when almost all of the JVMs were 32-bit; we ran into a problem with a coding pattern that constructed anonymous classes more quickly than the garbage collection could handle, and the permanent generation ran out of space, bringing down the VM. At the time, it was running as an Applet, in a browser started by one of our thousands of users, owned by the user's employer (our customer), on a machine tightly configured by our customer's IT department to run multiple applications, where we do not have any ability to inject JVM launch options to, for example, control the size of the permanent generation. It seemed at the time that the garbage collectors didn't even look in the permanent generation very frequently, so there as an architectural mis-match with high-speed construction of anonymous classes. My understanding is that 64-bit JVMs don't have the concept of a permanent generation. Is there a replacement in current JVMs where anonymous classes might still overflow? While the application was usually launched as an Applet, it also has a main() and can run as an application. That's the way it will run, going forward, obviously. Now that we can influence the JVM startup options, I want to know if I should be considering this. The reason is that I want to use this pattern to fix another problem with GUI activity (Java AWT). This fix will spike the construction of anonymous classes significantly.** If there is any likelihood of crashing the JVM (as it would, historically), I want to understand what the risk is. ** The new pattern would use: SwingUtilities.invokeLater(new Runnable() {public void run() {doGUI();}}); Where doGUI() might be as trivial as painting a dot. This thing could resemble a denial-of-service attack on the thread constructor and GUI event thread scheduler, if such a DoS concept existed. As long as garbage collection works, I expect it all should work...? Thanks! Mark Ludwig From rkennke at redhat.com Tue Nov 27 17:43:42 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Nov 2018 18:43:42 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> <4650c622-c937-4a76-3b9f-10f93c677dcc@redhat.com> Message-ID: Hi Vladimir, > On 11/27/18 1:37 AM, Roman Kennke wrote: >> Hi Vladimir, >> >>> You need to check if shenandoahgc is disabled first - check >>> DISABLED_JVM_FEATURES list (see code for jvmci). >> >> Why? If Shenandoah is disabled, it will be on this list, and therefore >> not be built (see JvmFeatures.gmk). > > Did you test with --with-jvm-variants=-shenandoahgc to make sure it is > disabled? I tested with --with-jvm-features=-shenandoahgc (and will do so again right after finishing this email, just to be sure). (Note: shenandoahgc is a feature, not a variant). > May be I needed explicitly check jvmci because I need it early to check > it for enable Graal build. So it is different from your case. Probably. >>> Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check. >>> Do you support all x86 OSs? >> >> We can build with 32bit, but it will not run. It seems indeed curious to >> enable it. It's probably only there to allow 32bit builds with >> Shenandoah enabled, just to verify that we have all the relevant LP64 >> checks in code in place. I will check it with my collegues. > > What about OS? Do you support Windows, MacOS? I did not see any OS > specific changes. So may be it is alright. Shenandoah is OS agnostic and we compile + run it on Windows successfully. Adopters tell us it's fine on MacOS too. >> >>> Why you set VM CFLAGS when shenandoahgc is not enabled? It is in >>> JvmFeatures.gmk. >> >> This *disables* and excludes Shenandoah if not enabled. >> >> +ifneq ($(call check-jvm-feature, shenandoahgc), true) >> +? JVM_CFLAGS_FEATURES += -DINCLUDE_SHENANDOAHGC=0 >> +? JVM_EXCLUDE_PATTERNS += gc/shenandoah >> >> pretty much the same pattern as zgc and other features. >> >> and then we add some flags when Shenandoah is enabled: >> >> +else >> +? JVM_CFLAGS_FEATURES += -DSUPPORT_BARRIER_ON_PRIMITIVES >> -DSUPPORT_NOT_TO_SPACE_INVARIANT >> +endif >> >> ... which are required for building with Shenandoah enabled. > > My bad. Somehow I thought it was reverse. Too much coffee at the > morning. :( Yeah, maybe it's written the wrong way around (double-negation), but that's how it is for other similar blocks too. > Looks good. Thanks! >>> I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc >>> specific nodes. This looks intrusive. I hope you clean it up. >> >> There are a few places with INCLUDE_SHENANDOAHGC checks where it seemed >> excessive to add a whole API just for a tiny, very GC specific check. We >> still do have intrusive changes in loop*, which we are working on to >> resolve. We declare+define all our GC specific nodes in >> shenandoahBarrierSetC2.hpp and related Shenandoah-specific files. The >> only things in shared code is the usual declarations in classes.hpp/cpp >> and node.hpp to get is_ShenandoahBarrier() an similar queries. This >> seems in-line with what other GCs do (look for e.g. LoadBarrier). Please >> be specific which changes in opto you'd like to see resolved (and don't >> look at loop* source files at this point ;-) ). > > Is it possible to reuse LoadBarrier by adding GC specific flag to it? > I am not against adding new nodes if really needed. My concern is about > using GC name in node's names. That would be very weird. Shenandoah's barrier is *not* a load barrier. GC's names in node name makes sense (to me) because the generated code is GC specific, and it's used in .ad files to match. I guess we could give it a more generic names (ReadBarrier, WriteBarrier, GCCompareAndSwap, ..) and then match them in .ad and call via BarrierSetAssembler to generate GC specific code. But it seems odd and hard to read+understand to me. > Yes, I am fine with very few INCLUDE_SHENANDOAHGC. Ok. Let's see how it looks like after Roland's latest changes are in. > Thanks, > Vladimir Thanks for reviewing! Roman >> Thanks for reviewing! >> Roman >> >>> Thanks, >>> Vladimir >>> >>> On 11/26/18 2:47 PM, Erik Joelsson wrote: >>>> Build changes look ok to me. >>>> >>>> /Erik >>>> >>>> On 2018-11-26 13:39, Roman Kennke wrote: >>>>> Hi, >>>>> >>>>> This is the first round of changes for including Shenandoah GC into >>>>> mainline. >>>>> I divided the review into parts that roughly correspond to the >>>>> mailing lists >>>>> that would normally review it, and I divided it into 'shared' code >>>>> changes and >>>>> 'shenandoah' code changes (actually, mostly additions). The intend >>>>> is to >>>>> eventually >>>>> push them as single 'combined' changeset, once reviewed. >>>>> >>>>> JEP: >>>>> ??? https://openjdk.java.net/jeps/189 >>>>> Bug entry: >>>>> >>>>> ???https://bugs.openjdk.java.net/browse/JDK-8214259 >>>>> >>>>> Webrevs: >>>>> ??? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>>> >>>>> For those who want to see the full change, have a look at the >>>>> shenandoah-complete >>>>> >>>>> >>>>> >>>>> directory, >>>>> it contains the full combined webrev. Alternatively, there is the file >>>>> shenandoah-master.patch >>>>> , >>>>> >>>>> >>>>> which is what I intend to commit (and which should be equivalent to >>>>> the >>>>> 'shenandoah-complete' webrev). >>>>> >>>>> Sections to review (at this point) are the following: >>>>> ???*) shenandoah-gc >>>>> >>>>> >>>>> >>>>> ????? - Actual Shenandoah implementation, almost completely >>>>> residing in >>>>> gc/shenandoah >>>>> >>>>> ???*) shared-gc >>>>> >>>>> >>>>> ????? - This is mostly boilerplate that is common to any GC >>>>> ????? - referenceProcessor.cpp has a little change to make one >>>>> assert not >>>>> fail (next to CMS and G1) >>>>> ????? - taskqueue.hpp has some small adjustments to enable subclassing >>>>> >>>>> ???*) shared-serviceability >>>>> >>>>> >>>>> >>>>> ????? - The usual code to support another GC >>>>> >>>>> ???*) shared-runtime >>>>> >>>>> >>>>> >>>>> ????? - A number of friends declarations to allow Shenandoah >>>>> iterators to >>>>> hook up with, >>>>> ??????? e.g. ClassLoaderData, CodeCache, etc >>>>> ????? - Warning and disabling JFR LeakProfiler >>>>> ????? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>>> Shenandoah C2 optimizations >>>>> ????? - Locks initialization in mutexLocker.cpp as usual >>>>> ????? - VM operations defines for Shenandoah's VM ops >>>>> ????? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>>> Shenandoah's logging >>>>> ????? - The usual macros in macro.hpp >>>>> >>>>> ???*) shared-build >>>>> >>>>> >>>>> >>>>> ????? - Add shenandoah feature, enabled by default, as agreed with >>>>> Vladimir K. beforehand >>>>> ????? - Some flags for shenandoah-enabled compilation to get >>>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>>> ??????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>>> Shenandoah's barriers >>>>> ????? - --param inline-unit-growth=1000 settings for 2 shenandoah >>>>> source >>>>> files, which is >>>>> ??????? useful to get the whole marking loop inlined (observed >>>>> significant >>>>> regression if we >>>>> ??????? don't) >>>>> >>>>> ???*) shared-tests >>>>> >>>>> >>>>> >>>>> ????? - Test infrastructure to support Shenandoah >>>>> ????? - Shenandoah test groups >>>>> ????? - Exclude Shenandoah in various tests that can be run with >>>>> selected GC >>>>> ????? - Enable/add configure for Shenandoah for tests that make >>>>> sense to >>>>> run with it >>>>> >>>>> ???*) shenandoah-tests >>>>> >>>>> >>>>> >>>>> ????? - Shenandoah specific tests, most reside in gc/shenandoah >>>>> subdirectory >>>>> ????? - A couple of tests configurations have been added, e.g. >>>>> TestGCBasherWithShenandoah.java >>>>> >>>>> I intentionally left out shared-compiler for now, because we have some >>>>> work left to do >>>>> there, but if you click around you'll find the patch anyway, in >>>>> case you >>>>> want to take >>>>> a peek at it. >>>>> >>>>> We have regular builds on: >>>>> ??? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>>> ??? - {Windows} x {x86_64}, >>>>> ??? - {MacOS X} x {x86_64} >>>>> >>>>> This also routinely passes: >>>>> ??? - the new Shenandoah tests >>>>> ??? - jcstress with/without aggressive Shenandoah verification >>>>> ??? - specjvm2008 with/without aggressive Shenandoah verification >>>>> >>>>> >>>>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>>>> deserves >>>>> the credit for being the original inventor of Shenandoah, Aleksey >>>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>>> teams for tirelessly helping with and reviewing all the GC >>>>> interface and >>>>> related changes, and of course the many early adopters for reporting >>>>> bugs and success stories and feature requests: we wouldn't be here >>>>> without any of you! >>>>> >>>>> Best regards, >>>>> Roman >>>>> >> From rkennke at redhat.com Tue Nov 27 17:54:05 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 27 Nov 2018 18:54:05 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> <4650c622-c937-4a76-3b9f-10f93c677dcc@redhat.com> Message-ID: <7ac7b365-5383-a28b-2e76-c6548177e840@redhat.com> >>> You need to check if shenandoahgc is disabled first - check >>> DISABLED_JVM_FEATURES list (see code for jvmci). >> >> Why? If Shenandoah is disabled, it will be on this list, and therefore >> not be built (see JvmFeatures.gmk). > > Did you test with --with-jvm-variants=-shenandoahgc to make sure it is > disabled? Test: Built the Shenandoah-patched tree with --with-jvm-features=-shenandoahgc. Then: java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC --version Error occurred during initialization of VM Option -XX:+UseShenandoahGC not supported Also, we regularily build minimal and zero, and those disable Shenandoah too, that's how we know this works reliably. Thanks, Roman From kim.barrett at oracle.com Tue Nov 27 18:03:00 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 27 Nov 2018 13:03:00 -0500 Subject: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> Message-ID: <4F249D17-A348-4BB2-A19C-ECFFC6AB80B8@oracle.com> > On Nov 27, 2018, at 6:51 AM, Lindenmaier, Goetz wrote: > > Hi, > > I built aix slowdebug, fastdebug and product builds successfully. > The other builds run tonight. > > Best regards, > Goetz. > > >> -----Original Message----- >> From: Lindenmaier, Goetz >> Sent: Tuesday, November 27, 2018 9:10 AM >> To: 'Kim Barrett' ; hotspot-dev developers >> >> Subject: RE: 8213481: [REDO] Fix incorrect copy constructors in hotspot >> >> Hi Kim, >> >> I put this into our build system to assure our compilers grok this code. >> AIX is building already, all other platforms will build tonight >> (European night??). I think xlc is the most likely to cause issues, >> On linux we are on gcc 7.3 anyways, which should be fine. >> >> Best regards, >> Goetz. Thanks for doing this testing. I wasn?t really expecting problems, since the code is substantially similar to what was there before, but it?s good to have that confirmed. From dean.long at oracle.com Tue Nov 27 18:07:26 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 27 Nov 2018 10:07:26 -0800 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: References: Message-ID: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> Hi Erik.? Can you explain why you removed the pre-allocated "next stub"?? I guess it was no longer necessary?? If so, then you should update the "always has sentinel" comment in is_empty. In your guarantee messages, some use "should not fail" and some use "should never fail".? It's not a big deal but maybe they should be the same. You introduced a couple of infinite loops.? Are we guaranteed to exit these loops eventually?? Is there an upper bound on how many iterations are needed? The rest looks good. dl On 11/27/18 5:00 AM, Erik ?sterlund wrote: > Hi, > > IC stub refilling currently requires a safepoint operation. When done > right at the point where an CompiledIC is about to get patched to a > transitional state using an IC stub, locks may already be held, > causing a bunch of locking issues - especially for concurrent class > unloading. Therefore, the IC stub refilling ought to be moved out so > that IC cache cleaning and transitioning may be done without any > safepoints, and the locks in the path ought to not perform safepoint > checking. > > This is implemented by allowing IC transitions to fail when they > require IC stubs, and we run out of them. This propages back to a > higher level where IC stubs are refilled after having released the IC > locker. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8214338 > > Thanks, > /Erik From kim.barrett at oracle.com Tue Nov 27 18:12:11 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 27 Nov 2018 13:12:11 -0500 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> Message-ID: > On Nov 26, 2018, at 10:21 PM, David Holmes wrote: > > On 27/11/2018 1:15 pm, Kim Barrett wrote: >>> On Nov 26, 2018, at 8:39 PM, David Holmes wrote: >>>> The ResourceObj assignment operator also should not be assuming the >>>> target of the assignment was stack/embeded allocated; that's just not >>>> a valid assumption. The corresponding assertion has been removed. >>> >>> Isn't that enforcing a usage requirement of ResourceObj? C-heap/resource-area/arena allocated ResourceObj should only be used via pointers - assigning one to another makes no sense to me. >> I don?t see any reason for such an artificial restriction, and think it might prevent reasonable uses. >> I don?t have a specific use-case in mind, but the natural result is observed from doing the natural thing. > > What is the "natural result" here? I don't see (and Vladimir seems to share a similar view) how it makes any sense to assign non stack/embedded resource objects to each other? And mixing assignment across different types of ResourceObj makes no sense to me at all. I'm not even sure there's a usecase for stack/embedded but that at least seem a consistent usage. > > David The "natural result" is that the derived class copy-assign code does what it does. There are things that can go badly wrong here regarding lifetimes of embedded object references. But the asserted restriction on the allocated location of the object being copy-assigned to does nothing that I can see to prevent those potential problems. Can you provide a use-case where the assertion actually does something useful? I removed it because I don?t think such a thing exists. From alexey.menkov at oracle.com Tue Nov 27 21:01:14 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 27 Nov 2018 13:01:14 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> Message-ID: Hi Dean, Thank you for the feedback and for the comments in Jira. >> How does this solve the problem of >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? It doesn't. It fixes another issue. The test suspend test thread and ensures that top frame is a "fibonachi" method. Then it turns SingleStep on, does PopFrame and resumes the thread. The problem is the thread continues execution of compiled code (ignoring SingleStep) and we get SindleStep event only when adjustCompilationLevel method is called (it's interpreted code). There are several other bugs which are (most likely) caused by suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) --alex On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: > Hi Dean, > > Thank you a lot for looking at this! > Just a couple of points from me (it is up to Alex to provide a full answer). > > > I think, Alex in this RFR missed to tell that we knew about this issue > that an incorrect frame will be popped. > But after some discussion we decided to file a separate issue on this. > Alex wanted to create a good stand-alone test case demonstrating this > problem before filing it. > > Now, as I can see, the JDK-8195635 > looks very close to a > bug that we wanted to file. > The only difference is that our scenario includes the > SharedRuntime::resolve_static_call_C instead of the > JavaCallWrapper::JavaCallWrapper as a helper for Java method invocation. > If I understand corrctly (Alex will fix me if needed), the jtreg test we > used to chase this issue did not catch a problem with the > HotSpotJVMCIRuntime.adjustCompilationLevel. > > > The suggested fix was to fix the mismatch in the > TieredThresholdPolicy::even with the comment in the interpreter code: > nmethod* InterpreterRuntime::frequency_counter_overflow(JavaThread* > thread, address branch_bcp) { > ????. . . > ??if (nm != NULL && thread->is_interp_only_mode()) { > ????// Normally we never get an nm if is_interp_only_mode() is true, > because > ????// policy()->event has a check for this and won't compile the > method when > ????// true. However, it's possible for is_interp_only_mode() to become > true > ????// during the compilation. We don't want to return the nm in that case > ????// because we want to continue to execute interpreted. > ????nm = NULL; > ??} > > > So I think the fix needs to be in code like > SharedRuntime::resolve_static_call_C(), > > where it returns get_c2i_entry() if is_interp_only_mode() is true. > > I'm not sure, the adding this condition and returning the > get_c2i_entry() is always correct. > We need some help from the Compiler team to make it right. > > BTW, the interp_only_mode has been enabled only when some interp_only > events are enabled. > It is not set by either PopFrame or ForceEarlyReturn. > But the popframe009 test enables single stepping, so we wanted to make > this part right. > > Thanks, > Serguei > > > On 11/26/18 16:05, dean.long at oracle.com wrote: >> How does this solve the problem of >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >> >> I don't think this fix is the right approach.? Doesn't >> is_interp_only_mode() only apply to the current thread?? I don't think >> it's safe to assume no compiles will happen in other threads, or that >> a method call target is not already compiled, because as far as I can >> tell, JVMTI only deoptimizes the active frames.? The bug report >> describes a case where the caller has been deoptimized while resolving >> a call.? I don't see how this fix prevents the target from being a >> previously compiled method.? But we do need to make sure not to call >> into compiled code, so I think the fix needs to be in code like >> SharedRuntime::resolve_static_call_C(), where it returns >> get_c2i_entry() if is_interp_only_mode() is true.? However, there is >> still another problem.? By allowing JVMTI to suspend the thread during >> call setup, but reporting the frame as still in the caller instead of >> the callee, we confuse JVMTI into thinking that execution will resume >> in the caller instead of the callee.? We may want to restrict where we >> offer JVMTI suspend points, and not offer a JVMTI suspend point in >> SharedRuntime::resolve_static_call_C and friends at all. >> >> dl >> >> >> On 11/26/18 11:14 AM, Alex Menkov wrote: >>> Hi all, >>> >>> Please review the fix for >>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>> webrev: >>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>> >>> Description: >>> The test suspends a thread, turns on single stepping and then calls >>> PopFrame. SingleStep event is expected as soon as the thread is >>> resumed and PopFrame is processed (so we have call stack with the >>> depth 1 less than it was before PopFrame). Instead SingleStep event >>> is received with much deeper call stack (and PopFrame processes wrong >>> frame). >>> Research shown that this is caused by missed deoptimization of the >>> current frame. >>> As far as I understand CompilationPolicy::event should handle the >>> case when the thread has is_interp_only_mode() set, but >>> TieredThresholdPolicy::event checks this only then CompLevel is >>> CompLevel_none. >>> CompilerRuntime always calls policy()->event with CompLevel == >>> CompLevel_aot. >>> >>> The fix looks quite simple, but I'd appreciate feedback from runtime >>> and compiler teams as I'm not sure I completely understand all the >>> details of the "PopFrame dance". >>> >>> --alex >> > From david.holmes at oracle.com Tue Nov 27 21:23:35 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Nov 2018 07:23:35 +1000 Subject: Java AWT, Swing, threads, anon classes, and garbage collection In-Reply-To: References: Message-ID: <36cb18ff-76a0-a454-6a24-7ef9bca9d0df@oracle.com> Hi, On 28/11/2018 2:24 am, Ludwig, Mark wrote: > Greetings, > > Abstract: > > I'd appreciate pointers to any papers or discussions about how anonymous classes are garbage-collected in current JVMs (Java 9+). > Hopefully someone will respond that the historical problem with garbage collection of anonymous classes has been solved. Do you mean the instances of the anonymous classes? As Michael Rasmussen points out these are just regular objects and GC'd like any other object. That said it is possible there may been issues with memory leaks in the past with respect to AWTSwing event handlers - you'd need to ask the GUI folk on awt-dev or swing-dev about that. The perm gen was removed a number of releases ago so we no longer have a pre-defined upper bound on what can be contained there. But as the name suggests that was for things which had a lifetime that matches that of the VM itself. If you are having issues with class unloading (or lack thereof) then you need to load short-lived classes into a separate classloader so that they can be reclaimed when the classloader is unreachable. No classes loaded by the system loaders will ever be unloaded. David > TL;DR > > My information is over 10 years old, when almost all of the JVMs were 32-bit; we ran into a problem with a coding pattern that constructed anonymous classes more quickly than the garbage collection could handle, and the permanent generation ran out of space, bringing down the VM. At the time, it was running as an Applet, in a browser started by one of our thousands of users, owned by the user's employer (our customer), on a machine tightly configured by our customer's IT department to run multiple applications, where we do not have any ability to inject JVM launch options to, for example, control the size of the permanent generation. > > It seemed at the time that the garbage collectors didn't even look in the permanent generation very frequently, so there as an architectural mis-match with high-speed construction of anonymous classes. > > My understanding is that 64-bit JVMs don't have the concept of a permanent generation. Is there a replacement in current JVMs where anonymous classes might still overflow? > > While the application was usually launched as an Applet, it also has a main() and can run as an application. That's the way it will run, going forward, obviously. Now that we can influence the JVM startup options, I want to know if I should be considering this. The reason is that I want to use this pattern to fix another problem with GUI activity (Java AWT). This fix will spike the construction of anonymous classes significantly.** If there is any likelihood of crashing the JVM (as it would, historically), I want to understand what the risk is. > > ** The new pattern would use: > > SwingUtilities.invokeLater(new Runnable() {public void run() {doGUI();}}); > > Where doGUI() might be as trivial as painting a dot. This thing could resemble a denial-of-service attack on the thread constructor and GUI event thread scheduler, if such a DoS concept existed. As long as garbage collection works, I expect it all should work...? > > Thanks! > > Mark Ludwig > From erik.osterlund at oracle.com Tue Nov 27 21:29:34 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 27 Nov 2018 22:29:34 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> Message-ID: <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> Hi Dean, Thank you for reviewing this. Full webrev: http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ Incremental webrev: http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ On 2018-11-27 19:07, dean.long at oracle.com wrote: > Hi Erik.? Can you explain why you removed the pre-allocated "next > stub"?? I guess it was no longer necessary?? If so, then you should > update the "always has sentinel" comment in is_empty. Updated. Indeed, the sentinel no longer serves a purpose so I removed it. Found some more traces of the sentinel that I removed in the last webrev. > In your guarantee messages, some use "should not fail" and some use > "should never fail".? It's not a big deal but maybe they should be the > same. Sure. Updated. > You introduced a couple of infinite loops.? Are we guaranteed to exit > these loops eventually?? Is there an upper bound on how many iterations > are needed? In my latest webrev I removed some unnecessary set_to_clean() where IC caches are already is_clean(). With that in place, there should be global progress guarantees and a single IC stub in the buffer should be all you really "need". Although, you might want more. In fact, I'm questioning if the 10K size buffer is big enough today, but I'll leave that for another day. Thanks, /Erik > The rest looks good. > > dl > > On 11/27/18 5:00 AM, Erik ?sterlund wrote: >> Hi, >> >> IC stub refilling currently requires a safepoint operation. When done >> right at the point where an CompiledIC is about to get patched to a >> transitional state using an IC stub, locks may already be held, >> causing a bunch of locking issues - especially for concurrent class >> unloading. Therefore, the IC stub refilling ought to be moved out so >> that IC cache cleaning and transitioning may be done without any >> safepoints, and the locks in the path ought to not perform safepoint >> checking. >> >> This is implemented by allowing IC transitions to fail when they >> require IC stubs, and we run out of them. This propages back to a >> higher level where IC stubs are refilled after having released the IC >> locker. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8214338 >> >> Thanks, >> /Erik > From david.holmes at oracle.com Tue Nov 27 21:34:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Nov 2018 07:34:23 +1000 Subject: RFR: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <377b9b49-ae76-5da3-63f0-beabddf8abf9@oracle.com> <3da6b5b4-861a-a89c-f82b-eda3f681fdca@oracle.com> Message-ID: On 28/11/2018 4:12 am, Kim Barrett wrote: >> On Nov 26, 2018, at 10:21 PM, David Holmes wrote: >> >> On 27/11/2018 1:15 pm, Kim Barrett wrote: >>>> On Nov 26, 2018, at 8:39 PM, David Holmes wrote: >>>>> The ResourceObj assignment operator also should not be assuming the >>>>> target of the assignment was stack/embeded allocated; that's just not >>>>> a valid assumption. The corresponding assertion has been removed. >>>> >>>> Isn't that enforcing a usage requirement of ResourceObj? C-heap/resource-area/arena allocated ResourceObj should only be used via pointers - assigning one to another makes no sense to me. >>> I don?t see any reason for such an artificial restriction, and think it might prevent reasonable uses. >>> I don?t have a specific use-case in mind, but the natural result is observed from doing the natural thing. >> >> What is the "natural result" here? I don't see (and Vladimir seems to share a similar view) how it makes any sense to assign non stack/embedded resource objects to each other? And mixing assignment across different types of ResourceObj makes no sense to me at all. I'm not even sure there's a usecase for stack/embedded but that at least seem a consistent usage. >> >> David > > The "natural result" is that the derived class copy-assign code does > what it does. There are things that can go badly wrong here regarding > lifetimes of embedded object references. But the asserted restriction > on the allocated location of the object being copy-assigned to does > nothing that I can see to prevent those potential problems. > > Can you provide a use-case where the assertion actually does something > useful? I removed it because I don?t think such a thing exists. AFAICS the only time it makes any sense to use assignment is with stack/embedded ResourceObj. The assert verifies that is the case. This seems a very clear cut use of an assert to ensure an API is not being misused. Does it prevent all possible misuse - no. But that's not a reason to remove what it does do. Do we ever need the assignment semantics? Maybe we should just prohibit it altogether? David From vladimir.kozlov at oracle.com Tue Nov 27 21:36:34 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 27 Nov 2018 13:36:34 -0800 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <7ac7b365-5383-a28b-2e76-c6548177e840@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <7fca3c07-ee1e-6b33-eef6-37c09dd6f65c@oracle.com> <4650c622-c937-4a76-3b9f-10f93c677dcc@redhat.com> <7ac7b365-5383-a28b-2e76-c6548177e840@redhat.com> Message-ID: <2de836d4-df71-f7bf-65f6-2e028d6cc309@oracle.com> Good. Thank you for testing. I don't have any more question about build changes. I will wait C2 changes. About C2 nodes names: >> Is it possible to reuse LoadBarrier by adding GC specific flag to it? >> I am not against adding new nodes if really needed. My concern is about >> using GC name in node's names. > > That would be very weird. Shenandoah's barrier is *not* a load barrier. > GC's names in node name makes sense (to me) because the generated code > is GC specific, and it's used in .ad files to match. I guess we could > give it a more generic names (ReadBarrier, WriteBarrier, > GCCompareAndSwap, ..) and then match them in .ad and call via > BarrierSetAssembler to generate GC specific code. But it seems odd and > hard to read+understand to me. I got that you can't reuse exisitng nodes. Sorry, but I would prefer generic names for new nodes. You don't need BarrierSetAssembler if these new nodes are used only for Shenandoah. Side note, don't use #ifdef in .ad files. The check for nodes generation should be done in C2 code during ideal graph generation. Thanks, Vladimir On 11/27/18 9:54 AM, Roman Kennke wrote: >>>> You need to check if shenandoahgc is disabled first - check >>>> DISABLED_JVM_FEATURES list (see code for jvmci). >>> >>> Why? If Shenandoah is disabled, it will be on this list, and therefore >>> not be built (see JvmFeatures.gmk). >> >> Did you test with --with-jvm-variants=-shenandoahgc to make sure it is >> disabled? > > Test: Built the Shenandoah-patched tree with > --with-jvm-features=-shenandoahgc. Then: > > java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC --version > Error occurred during initialization of VM > Option -XX:+UseShenandoahGC not supported > > Also, we regularily build minimal and zero, and those disable Shenandoah > too, that's how we know this works reliably. > > Thanks, > Roman > From serguei.spitsyn at oracle.com Tue Nov 27 22:19:04 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 27 Nov 2018 14:19:04 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> Message-ID: <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> Hi Dean, You also asked the questions: > Doesn't is_interp_only_mode() only apply to the current thread? Yes it does. > I don't think it's safe to assume no compiles will happen in other threads, > or that a method call target is not already compiled, because as far > as I can tell, JVMTI only deoptimizes the active frames. I agree with it. However, compilations on the thread with the interp_only_mode enabled is the most important case. Disabling compilations such thread improves testing stability. > The bug report describes a case where the caller has been deoptimized > while resolving a call.? I don't see how this fix prevents the target > from being a previously compiled method. > But we do need to make sure not to call into compiled code, so I think > the fix needs to be in code like SharedRuntime::resolve_static_call_C(), > where it returns get_c2i_entry() if is_interp_only_mode() is true. Thank you for making this point. It would be nice to attack this as well. We can try to investigate this approach further. One problem is that there are more cases like resolve_static_call_C, for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. We may need to fix the same in other places, like JavaCallWrapper::JavaCallWrapper. We can start from fixing it in the resolve_static_call_C. If it is successful then continue this effort for other cases as well. What do you think? Thanks, Serguei On 11/27/18 1:01 PM, Alex Menkov wrote: > Hi Dean, > > Thank you for the feedback and for the comments in Jira. > > >> How does this solve the problem of > >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? > > It doesn't. It fixes another issue. > The test suspend test thread and ensures that top frame is a > "fibonachi" method. Then it turns SingleStep on, does PopFrame and > resumes the thread. > The problem is the thread continues execution of compiled code > (ignoring SingleStep) and we get SindleStep event only when > adjustCompilationLevel method is called (it's interpreted code). > > There are several other bugs which are (most likely) caused by suspend > during call setup (JDK-8195635, JDK-8214093, JDK-8207013) > > --alex > > On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >> Hi Dean, >> >> Thank you a lot for looking at this! >> Just a couple of points from me (it is up to Alex to provide a full >> answer). >> >> >> I think, Alex in this RFR missed to tell that we knew about this >> issue that an incorrect frame will be popped. >> But after some discussion we decided to file a separate issue on this. >> Alex wanted to create a good stand-alone test case demonstrating this >> problem before filing it. >> >> Now, as I can see, the JDK-8195635 >> looks very close >> to a bug that we wanted to file. >> The only difference is that our scenario includes the >> SharedRuntime::resolve_static_call_C instead of the >> JavaCallWrapper::JavaCallWrapper as a helper for Java method invocation. >> If I understand corrctly (Alex will fix me if needed), the jtreg test >> we used to chase this issue did not catch a problem with the >> HotSpotJVMCIRuntime.adjustCompilationLevel. >> >> >> The suggested fix was to fix the mismatch in the >> TieredThresholdPolicy::even with the comment in the interpreter code: >> nmethod* InterpreterRuntime::frequency_counter_overflow(JavaThread* >> thread, address branch_bcp) { >> ?????. . . >> ???if (nm != NULL && thread->is_interp_only_mode()) { >> ?????// Normally we never get an nm if is_interp_only_mode() is true, >> because >> ?????// policy()->event has a check for this and won't compile the >> method when >> ?????// true. However, it's possible for is_interp_only_mode() to >> become true >> ?????// during the compilation. We don't want to return the nm in >> that case >> ?????// because we want to continue to execute interpreted. >> ?????nm = NULL; >> ???} >> >> ?> So I think the fix needs to be in code like >> SharedRuntime::resolve_static_call_C(), >> ?> where it returns get_c2i_entry() if is_interp_only_mode() is true. >> >> I'm not sure, the adding this condition and returning the >> get_c2i_entry() is always correct. >> We need some help from the Compiler team to make it right. >> >> BTW, the interp_only_mode has been enabled only when some interp_only >> events are enabled. >> It is not set by either PopFrame or ForceEarlyReturn. >> But the popframe009 test enables single stepping, so we wanted to >> make this part right. >> >> Thanks, >> Serguei >> >> >> On 11/26/18 16:05, dean.long at oracle.com wrote: >>> How does this solve the problem of >>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>> >>> I don't think this fix is the right approach.? Doesn't >>> is_interp_only_mode() only apply to the current thread?? I don't >>> think it's safe to assume no compiles will happen in other threads, >>> or that a method call target is not already compiled, because as far >>> as I can tell, JVMTI only deoptimizes the active frames.? The bug >>> report describes a case where the caller has been deoptimized while >>> resolving a call.? I don't see how this fix prevents the target from >>> being a previously compiled method.? But we do need to make sure not >>> to call into compiled code, so I think the fix needs to be in code >>> like SharedRuntime::resolve_static_call_C(), where it returns >>> get_c2i_entry() if is_interp_only_mode() is true.? However, there is >>> still another problem.? By allowing JVMTI to suspend the thread >>> during call setup, but reporting the frame as still in the caller >>> instead of the callee, we confuse JVMTI into thinking that execution >>> will resume in the caller instead of the callee.? We may want to >>> restrict where we offer JVMTI suspend points, and not offer a JVMTI >>> suspend point in SharedRuntime::resolve_static_call_C and friends at >>> all. >>> >>> dl >>> >>> >>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>> Hi all, >>>> >>>> Please review the fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>> webrev: >>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>> >>>> Description: >>>> The test suspends a thread, turns on single stepping and then calls >>>> PopFrame. SingleStep event is expected as soon as the thread is >>>> resumed and PopFrame is processed (so we have call stack with the >>>> depth 1 less than it was before PopFrame). Instead SingleStep event >>>> is received with much deeper call stack (and PopFrame processes >>>> wrong frame). >>>> Research shown that this is caused by missed deoptimization of the >>>> current frame. >>>> As far as I understand CompilationPolicy::event should handle the >>>> case when the thread has is_interp_only_mode() set, but >>>> TieredThresholdPolicy::event checks this only then CompLevel is >>>> CompLevel_none. >>>> CompilerRuntime always calls policy()->event with CompLevel == >>>> CompLevel_aot. >>>> >>>> The fix looks quite simple, but I'd appreciate feedback from >>>> runtime and compiler teams as I'm not sure I completely understand >>>> all the details of the "PopFrame dance". >>>> >>>> --alex >>> >> From dean.long at oracle.com Tue Nov 27 22:50:05 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 27 Nov 2018 14:50:05 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> Message-ID: <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> Let me list the proposed solutions: 1. short-circuit compiles in CompilationPolicy::event in is_interp_only_mode 2. detect is_interp_only_mode in resolve stubs and force a c2i call 3. don't offer a JVMTI suspend point in resolve stubs (or JavaCallWrapper) Solution 1 penalizes other threads, and don't prevent other threads from compiling the method, so while it may help a little, it's incomplete. Solutions 1 and 2 allow the thread to resume in a different method (the callee method), causing other problems. With Solution 3, the frame we suspend is the same as the frame we end up in after the resume, so I believe it solves all the problems. IMHO this is the correct solution to pursue. dl On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: > Hi Dean, > > You also asked the questions: > > > Doesn't is_interp_only_mode() only apply to the current thread? > > Yes it does. > > > > I don't think it's safe to assume no compiles will happen in other > threads, > > or that a method call target is not already compiled, because as far > > as I can tell, JVMTI only deoptimizes the active frames. > > I agree with it. > However, compilations on the thread with the interp_only_mode enabled > is the most important case. > Disabling compilations such thread improves testing stability. > > > The bug report describes a case where the caller has been deoptimized > > while resolving a call.? I don't see how this fix prevents the target > > from being a previously compiled method. > > But we do need to make sure not to call into compiled code, so I think > > the fix needs to be in code like > SharedRuntime::resolve_static_call_C(), > > where it returns get_c2i_entry() if is_interp_only_mode() is true. > > Thank you for making this point. > > It would be nice to attack this as well. > We can try to investigate this approach further. > One problem is that there are more cases like resolve_static_call_C, > for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. > We may need to fix the same in other places, like > JavaCallWrapper::JavaCallWrapper. > > We can start from fixing it in the resolve_static_call_C. > If it is successful then continue this effort for other cases as well. > > What do you think? > > Thanks, > Serguei > > > On 11/27/18 1:01 PM, Alex Menkov wrote: >> Hi Dean, >> >> Thank you for the feedback and for the comments in Jira. >> >> >> How does this solve the problem of >> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >> >> It doesn't. It fixes another issue. >> The test suspend test thread and ensures that top frame is a >> "fibonachi" method. Then it turns SingleStep on, does PopFrame and >> resumes the thread. >> The problem is the thread continues execution of compiled code >> (ignoring SingleStep) and we get SindleStep event only when >> adjustCompilationLevel method is called (it's interpreted code). >> >> There are several other bugs which are (most likely) caused by >> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >> >> --alex >> >> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>> Hi Dean, >>> >>> Thank you a lot for looking at this! >>> Just a couple of points from me (it is up to Alex to provide a full >>> answer). >>> >>> >>> I think, Alex in this RFR missed to tell that we knew about this >>> issue that an incorrect frame will be popped. >>> But after some discussion we decided to file a separate issue on this. >>> Alex wanted to create a good stand-alone test case demonstrating >>> this problem before filing it. >>> >>> Now, as I can see, the JDK-8195635 >>> looks very close >>> to a bug that we wanted to file. >>> The only difference is that our scenario includes the >>> SharedRuntime::resolve_static_call_C instead of the >>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>> invocation. >>> If I understand corrctly (Alex will fix me if needed), the jtreg >>> test we used to chase this issue did not catch a problem with the >>> HotSpotJVMCIRuntime.adjustCompilationLevel. >>> >>> >>> The suggested fix was to fix the mismatch in the >>> TieredThresholdPolicy::even with the comment in the interpreter code: >>> nmethod* InterpreterRuntime::frequency_counter_overflow(JavaThread* >>> thread, address branch_bcp) { >>> ?????. . . >>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>> ?????// Normally we never get an nm if is_interp_only_mode() is >>> true, because >>> ?????// policy()->event has a check for this and won't compile the >>> method when >>> ?????// true. However, it's possible for is_interp_only_mode() to >>> become true >>> ?????// during the compilation. We don't want to return the nm in >>> that case >>> ?????// because we want to continue to execute interpreted. >>> ?????nm = NULL; >>> ???} >>> >>> ?> So I think the fix needs to be in code like >>> SharedRuntime::resolve_static_call_C(), >>> ?> where it returns get_c2i_entry() if is_interp_only_mode() is true. >>> >>> I'm not sure, the adding this condition and returning the >>> get_c2i_entry() is always correct. >>> We need some help from the Compiler team to make it right. >>> >>> BTW, the interp_only_mode has been enabled only when some >>> interp_only events are enabled. >>> It is not set by either PopFrame or ForceEarlyReturn. >>> But the popframe009 test enables single stepping, so we wanted to >>> make this part right. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>> How does this solve the problem of >>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>> >>>> I don't think this fix is the right approach.? Doesn't >>>> is_interp_only_mode() only apply to the current thread?? I don't >>>> think it's safe to assume no compiles will happen in other threads, >>>> or that a method call target is not already compiled, because as >>>> far as I can tell, JVMTI only deoptimizes the active frames.? The >>>> bug report describes a case where the caller has been deoptimized >>>> while resolving a call.? I don't see how this fix prevents the >>>> target from being a previously compiled method.? But we do need to >>>> make sure not to call into compiled code, so I think the fix needs >>>> to be in code like SharedRuntime::resolve_static_call_C(), where it >>>> returns get_c2i_entry() if is_interp_only_mode() is true.? However, >>>> there is still another problem.? By allowing JVMTI to suspend the >>>> thread during call setup, but reporting the frame as still in the >>>> caller instead of the callee, we confuse JVMTI into thinking that >>>> execution will resume in the caller instead of the callee.? We may >>>> want to restrict where we offer JVMTI suspend points, and not offer >>>> a JVMTI suspend point in SharedRuntime::resolve_static_call_C and >>>> friends at all. >>>> >>>> dl >>>> >>>> >>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>> Hi all, >>>>> >>>>> Please review the fix for >>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>> webrev: >>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>> >>>>> Description: >>>>> The test suspends a thread, turns on single stepping and then >>>>> calls PopFrame. SingleStep event is expected as soon as the thread >>>>> is resumed and PopFrame is processed (so we have call stack with >>>>> the depth 1 less than it was before PopFrame). Instead SingleStep >>>>> event is received with much deeper call stack (and PopFrame >>>>> processes wrong frame). >>>>> Research shown that this is caused by missed deoptimization of the >>>>> current frame. >>>>> As far as I understand CompilationPolicy::event should handle the >>>>> case when the thread has is_interp_only_mode() set, but >>>>> TieredThresholdPolicy::event checks this only then CompLevel is >>>>> CompLevel_none. >>>>> CompilerRuntime always calls policy()->event with CompLevel == >>>>> CompLevel_aot. >>>>> >>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>> runtime and compiler teams as I'm not sure I completely understand >>>>> all the details of the "PopFrame dance". >>>>> >>>>> --alex >>>> >>> > From dean.long at oracle.com Tue Nov 27 22:55:26 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Tue, 27 Nov 2018 14:55:26 -0800 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> Message-ID: <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> There's a typo in the new guarantee message: "tranisition". Regarding the infinite loops, can we detect if progress isn't being made and assert/panic, rather than hanging? dl On 11/27/18 1:29 PM, Erik ?sterlund wrote: > Hi Dean, > > Thank you for reviewing this. > > Full webrev: > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ > > Incremental webrev: > http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ > > On 2018-11-27 19:07, dean.long at oracle.com wrote: >> Hi Erik.? Can you explain why you removed the pre-allocated "next >> stub"?? I guess it was no longer necessary?? If so, then you should >> update the "always has sentinel" comment in is_empty. > > Updated. Indeed, the sentinel no longer serves a purpose so I removed > it. Found some more traces of the sentinel that I removed in the last > webrev. > >> In your guarantee messages, some use "should not fail" and some use >> "should never fail".? It's not a big deal but maybe they should be >> the same. > > Sure. Updated. > >> You introduced a couple of infinite loops.? Are we guaranteed to exit >> these loops eventually?? Is there an upper bound on how many >> iterations are needed? > > In my latest webrev I removed some unnecessary set_to_clean() where IC > caches are already is_clean(). With that in place, there should be > global progress guarantees and a single IC stub in the buffer should > be all you really "need". Although, you might want more. In fact, I'm > questioning if the 10K size buffer is big enough today, but I'll leave > that for another day. > > Thanks, > /Erik > >> The rest looks good. >> >> dl >> >> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> IC stub refilling currently requires a safepoint operation. When >>> done right at the point where an CompiledIC is about to get patched >>> to a transitional state using an IC stub, locks may already be held, >>> causing a bunch of locking issues - especially for concurrent class >>> unloading. Therefore, the IC stub refilling ought to be moved out so >>> that IC cache cleaning and transitioning may be done without any >>> safepoints, and the locks in the path ought to not perform safepoint >>> checking. >>> >>> This is implemented by allowing IC transitions to fail when they >>> require IC stubs, and we run out of them. This propages back to a >>> higher level where IC stubs are refilled after having released the >>> IC locker. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>> >>> Thanks, >>> /Erik >> From kim.barrett at oracle.com Tue Nov 27 23:27:48 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 27 Nov 2018 18:27:48 -0500 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <46CBF9C2-7684-4EED-80F8-7103D48FA3A0@oracle.com> Message-ID: <4A6598C6-7A6E-4CD0-AF47-1F4E9CFE9B89@oracle.com> > On Nov 27, 2018, at 4:46 AM, Roman Kennke wrote: > > Hi Kim, > >>> Sections to review (at this point) are the following: >>> >>> *) shared-gc >>> - This is mostly boilerplate that is common to any GC >>> - referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1) >> >> Ick. But I don?t have a better suggestion that doesn?t involve adding a new API >> to CollectedHeap for use by this assertion, which seems a bit excessive if there >> are no other uses. > > Yeah. > I guess we could add a config _discovery_is_concurrent or similar in RP, > and check that. Or maybe one of _discovery_is_mt or _discovery_is_atomic > already covers that? I couldn't immediately tell/100% understand their > semantics. Seems worthy to look at after this? It might be equivalent to _discovery_is_atomic; I don?t remember the exact semantics right now. I think it?s unrelated to _discovery_is_mt. Yes, looking at this later is fine. Please file an RFE. >>> - taskqueue.hpp has some small adjustments to enable subclassing >> >> Why this change instead of JDK-8204947? As the description from that RFE says: >> "The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.? > > Yeah, see Zhengyu's comment. Let's ignore those changes for this review > (for now), expect our impl ported to taskqueue.hpp/cpp soon. I?m okay with that plan. Maybe add a comment in JDK-8204947 about this? From leonid.mesnik at oracle.com Tue Nov 27 23:36:24 2018 From: leonid.mesnik at oracle.com (leonid.mesnik at oracle.com) Date: Tue, 27 Nov 2018 15:36:24 -0800 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: Hi I looked at the tests changes only. It seems that? a some tests might start failing if JDK built without Shenandoah GC. Also some test are not going to be selected as expected. Unfortunately logic of '@requires' and @run in jtreg tests doesn't always work well for specific cases of different GC selection. The '@requires' tags are combined with '&' and whole test is selected or not. Test always execute ALL @run actions so it fails if option -XX:+UseShenandoahGC is not supported (not valid). The only way to split 'run' actions is to add more @test with same sources. They could be in the same file. See detailed info about jtreg tags here:http://openjdk.java.net/jtreg/tag-spec.html Could you please run your tests with with JDK which built without Shenandoah GC . It helps to verify that Shenandoah-specific tests/runs are not selected when this GC is not supported. ? Also it would be nice to verify that there are no any valid tests which became filtered out with your patch. The running of all test suite with available but not selected Shenandoah GC and enabled Graal also might help to verify @requires settings. (It doesn't help when Shenandoah GCis not supported though.) I haven't looked at the tests in directory gc/shenandoah in details. But all of them should be guarded with @requires. Placing them in separate directory is not enough. See G1 tests as example: http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestEagerReclaimHumongousRegions.java See more detailed comments about shared tests: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/TEST.groups.sdiff.html 219: tier2_gc_shenandoah = \ Usually tierN doesn't include tests from tierN-1. TierN is executed after TierN-1 completed so no need to re-run the same tests.? The typical groups might looks like: tier1_gc_shenandoah = \ ?gc/shenandoah/ \ ?other-tests tier2_gc_shenandoah = \ ?? gc/shenandoah/\ ? -:tier1_gc_shenandoah tier3_gc_shenandoah = \ ?? gc/shenandoah/ \? //all-other-tests ? -:tier2_gc_shenandoah http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/CriticalNativeArgs.java.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/stress/CriticalNativeStress.java.html So your test will be skipped if any of -XX:+UseEpsilonGC or -XX:+UseShenandoahGC is set. Also test might run only all run actions or none of them. It would be better to split this test into 2 tests. So epsilon tests might be executed if Shenandoah is absent. I think that (vm.bits == "64") is redundant in tag (you set x64 or arm64). Please use 4-space indentation? in java code. http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestFullGCCount.java.sdiff.html Even original requires seems confusing to me (but it works for CMS/G1 pair) 28? * @requires !(vm.gc.ConcMarkSweep & vm.opt.ExplicitGCInvokesConcurrent == true) So currently test is executed if GC is CMS or default GC and ExplicitGCInvokesConcurrent is not set to true. With your additional requirements 'vm.gc == "Shenandoah"' test is not selected if ANY GC is set. Test doesn't set any GC itself so only default GC might be tested.? See [1]. http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.sdiff.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestSystemGC.java.sdiff.html Tests will always just fail if -XX:+UseShenandoahGC? is not supported. (invalid option) You might want to split test is it done for CMS GC. http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestAlignmentToUseLargePages.java.sdiff.html I think 56? * @requires vm.gc=="null" & !vm.graal.enabled should be something like @requires vm.gc.Shenandoah & !vm.graal.enabled http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestUseCompressedOopsErgo.java.sdiff.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/class_unloading/TestClassUnloadingDisabled.java.sdiff.html The same for? 62? * @requires vm.gc=="null" & !vm.graal.enabled and 72? * @requires vm.gc=="null" & !vm.graal.enabled http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/ergonomics/TestDynamicNumberOfGCThreads.java.sdiff.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/logging/TestGCId.java.sdiff.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/CompressedOops/UseCompressedOops.java.sdiff.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/MemberName/MemberNameLeak.java.sdiff.html http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html Also these tests are going to be run with all GC and fails if Shenandoah is not supported. Leonid [1] http://openjdk.java.net/jtreg/tag-spec.html On 11/26/18 1:39 PM, Roman Kennke wrote: > > Hi, > > This is the first round of changes for including Shenandoah GC into > mainline. > I divided the review into parts that roughly correspond to the mailing > lists > that would normally review it, and I divided it into 'shared' code > changes and > 'shenandoah' code changes (actually, mostly additions). The intend is > to eventually > push them as single 'combined' changeset, once reviewed. > > JEP: > https://openjdk.java.net/jeps/189 > Bug entry: > > https://bugs.openjdk.java.net/browse/JDK-8214259 > > Webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ > > For those who want to see the full change, have a look at the > shenandoah-complete > > directory, > it contains the full combined webrev. Alternatively, there is the file > shenandoah-master.patch > , > which is what I intend to commit (and which should be equivalent to > the 'shenandoah-complete' webrev). > > Sections to review (at this point) are the following: > ?*) shenandoah-gc > > ??? - Actual Shenandoah implementation, almost completely residing in > gc/shenandoah > > ?*) shared-gc > > ??? - This is mostly boilerplate that is common to any GC > ??? - referenceProcessor.cpp has a little change to make one assert > not fail (next to CMS and G1) > ??? - taskqueue.hpp has some small adjustments to enable subclassing > > ?*) shared-serviceability > > ??? - The usual code to support another GC > > ?*) shared-runtime > > ??? - A number of friends declarations to allow Shenandoah iterators > to hook up with, > ????? e.g. ClassLoaderData, CodeCache, etc > ??? - Warning and disabling JFR LeakProfiler > ??? - fieldDescriptor.hpp added is_stable() accessor, for use in > Shenandoah C2 optimizations > ??? - Locks initialization in mutexLocker.cpp as usual > ??? - VM operations defines for Shenandoah's VM ops > ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in > Shenandoah's logging > ??? - The usual macros in macro.hpp > > ?*) shared-build > > ??? - Add shenandoah feature, enabled by default, as agreed with > Vladimir K. beforehand > ??? - Some flags for shenandoah-enabled compilation to get > SUPPORT_BARRIER_ON_PRIMITIVES > ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for > Shenandoah's barriers > ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source > files, which is > ????? useful to get the whole marking loop inlined (observed > significant regression if we > ????? don't) > > ?*) shared-tests > > ??? - Test infrastructure to support Shenandoah > ??? - Shenandoah test groups > ??? - Exclude Shenandoah in various tests that can be run with selected GC > ??? - Enable/add configure for Shenandoah for tests that make sense to > run with it > > ?*) shenandoah-tests > > ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory > ??? - A couple of tests configurations have been added, e.g. > TestGCBasherWithShenandoah.java > > I intentionally left out shared-compiler for now, because we have some > work left to do > there, but if you click around you'll find the patch anyway, in case > you want to take > a peek at it. > > We have regular builds on: > ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} > ? - {Windows} x {x86_64}, > ? - {MacOS X} x {x86_64} > > This also routinely passes: > ? - the new Shenandoah tests > ? - jcstress with/without aggressive Shenandoah verification > ? - specjvm2008 with/without aggressive Shenandoah verification > > > I'd like to thank my collegues at Red Hat: Christine Flood, she > deserves the credit for being the original inventor of Shenandoah, > Aleksey Shipl?v, Roland Westrelin & Zhengyu Gu for their countless > contributions, everybody else in Red Hat's OpenJDK team for testing, > advice and support, my collegues in Oracle's GC, runtime and compiler > teams for tirelessly helping with and reviewing all the GC interface > and related changes, and of course the many early adopters for > reporting bugs and success stories and feature requests: we wouldn't > be here without any of you! > > Best regards, > Roman > From serguei.spitsyn at oracle.com Tue Nov 27 23:38:20 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 27 Nov 2018 15:38:20 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> Message-ID: <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> Good summary with a list of solutions below! On 11/27/18 2:50 PM, dean.long at oracle.com wrote: > Let me list the proposed solutions: > > 1. short-circuit compiles in CompilationPolicy::event in > is_interp_only_mode > 2. detect is_interp_only_mode in resolve stubs and force a c2i call > 3. don't offer a JVMTI suspend point in resolve stubs (or > JavaCallWrapper) > > Solution 1 penalizes other threads, and don't prevent other threads > from compiling the method, so while it may help a little, it's incomplete. I doubt, the Solution 1 penalizes other threads. First, it is an important pretty frequent/common case that allows to save on compilations. Second, there is no point to compile methods on a thread executed in interp_only_mode. Also, there is no big advantage to compile methods even for other threads when an active debugging (e.g. single stepping) is in progress. Third, this would make it consistent with what the the interpreter is doing. So, I think, the Solution 1 is needed independently of other solutions we take. My suggestion is to apply it in the JDK-8195639 bug fix. > Solutions 1 and 2 allow the thread to resume in a different method > (the callee method), causing other problems. Agreed. > With Solution 3, the frame we suspend is the same as the frame we end > up in after the resume, so I believe it solves all the problems. > IMHO this is the correct solution to pursue. I agree in general. This solution has some risks involved as it impacts a lot of code including platform-independent. It is also going to fix the other bugs: JDK-8195635, JDK-8214093, JDK-8207013 (not sure about all of them yet). My suggestion is to pursue it as the JDK-8195635 fix. Could you, please, confirm if works for you? Also, we have no consensus yet on the Solution 1. Thanks, Serguei > dl > > On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: >> Hi Dean, >> >> You also asked the questions: >> >> > Doesn't is_interp_only_mode() only apply to the current thread? >> >> Yes it does. >> >> >> > I don't think it's safe to assume no compiles will happen in other >> threads, >> > or that a method call target is not already compiled, because as far >> > as I can tell, JVMTI only deoptimizes the active frames. >> >> I agree with it. >> However, compilations on the thread with the interp_only_mode enabled >> is the most important case. >> Disabling compilations such thread improves testing stability. >> >> > The bug report describes a case where the caller has been deoptimized >> > while resolving a call.? I don't see how this fix prevents the target >> > from being a previously compiled method. >> > But we do need to make sure not to call into compiled code, so I think >> > the fix needs to be in code like >> SharedRuntime::resolve_static_call_C(), >> > where it returns get_c2i_entry() if is_interp_only_mode() is true. >> >> Thank you for making this point. >> >> It would be nice to attack this as well. >> We can try to investigate this approach further. >> One problem is that there are more cases like resolve_static_call_C, >> for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. >> We may need to fix the same in other places, like >> JavaCallWrapper::JavaCallWrapper. >> >> We can start from fixing it in the resolve_static_call_C. >> If it is successful then continue this effort for other cases as well. >> >> What do you think? >> >> Thanks, >> Serguei >> >> >> On 11/27/18 1:01 PM, Alex Menkov wrote: >>> Hi Dean, >>> >>> Thank you for the feedback and for the comments in Jira. >>> >>> >> How does this solve the problem of >>> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>> >>> It doesn't. It fixes another issue. >>> The test suspend test thread and ensures that top frame is a >>> "fibonachi" method. Then it turns SingleStep on, does PopFrame and >>> resumes the thread. >>> The problem is the thread continues execution of compiled code >>> (ignoring SingleStep) and we get SindleStep event only when >>> adjustCompilationLevel method is called (it's interpreted code). >>> >>> There are several other bugs which are (most likely) caused by >>> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >>> >>> --alex >>> >>> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>>> Hi Dean, >>>> >>>> Thank you a lot for looking at this! >>>> Just a couple of points from me (it is up to Alex to provide a full >>>> answer). >>>> >>>> >>>> I think, Alex in this RFR missed to tell that we knew about this >>>> issue that an incorrect frame will be popped. >>>> But after some discussion we decided to file a separate issue on this. >>>> Alex wanted to create a good stand-alone test case demonstrating >>>> this problem before filing it. >>>> >>>> Now, as I can see, the JDK-8195635 >>>> looks very close >>>> to a bug that we wanted to file. >>>> The only difference is that our scenario includes the >>>> SharedRuntime::resolve_static_call_C instead of the >>>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>>> invocation. >>>> If I understand corrctly (Alex will fix me if needed), the jtreg >>>> test we used to chase this issue did not catch a problem with the >>>> HotSpotJVMCIRuntime.adjustCompilationLevel. >>>> >>>> >>>> The suggested fix was to fix the mismatch in the >>>> TieredThresholdPolicy::even with the comment in the interpreter code: >>>> nmethod* InterpreterRuntime::frequency_counter_overflow(JavaThread* >>>> thread, address branch_bcp) { >>>> ?????. . . >>>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>>> ?????// Normally we never get an nm if is_interp_only_mode() is >>>> true, because >>>> ?????// policy()->event has a check for this and won't compile the >>>> method when >>>> ?????// true. However, it's possible for is_interp_only_mode() to >>>> become true >>>> ?????// during the compilation. We don't want to return the nm in >>>> that case >>>> ?????// because we want to continue to execute interpreted. >>>> ?????nm = NULL; >>>> ???} >>>> >>>> ?> So I think the fix needs to be in code like >>>> SharedRuntime::resolve_static_call_C(), >>>> ?> where it returns get_c2i_entry() if is_interp_only_mode() is true. >>>> >>>> I'm not sure, the adding this condition and returning the >>>> get_c2i_entry() is always correct. >>>> We need some help from the Compiler team to make it right. >>>> >>>> BTW, the interp_only_mode has been enabled only when some >>>> interp_only events are enabled. >>>> It is not set by either PopFrame or ForceEarlyReturn. >>>> But the popframe009 test enables single stepping, so we wanted to >>>> make this part right. >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>>> How does this solve the problem of >>>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>> >>>>> I don't think this fix is the right approach.? Doesn't >>>>> is_interp_only_mode() only apply to the current thread?? I don't >>>>> think it's safe to assume no compiles will happen in other >>>>> threads, or that a method call target is not already compiled, >>>>> because as far as I can tell, JVMTI only deoptimizes the active >>>>> frames.? The bug report describes a case where the caller has been >>>>> deoptimized while resolving a call.? I don't see how this fix >>>>> prevents the target from being a previously compiled method.? But >>>>> we do need to make sure not to call into compiled code, so I think >>>>> the fix needs to be in code like >>>>> SharedRuntime::resolve_static_call_C(), where it returns >>>>> get_c2i_entry() if is_interp_only_mode() is true. However, there >>>>> is still another problem.? By allowing JVMTI to suspend the thread >>>>> during call setup, but reporting the frame as still in the caller >>>>> instead of the callee, we confuse JVMTI into thinking that >>>>> execution will resume in the caller instead of the callee.? We may >>>>> want to restrict where we offer JVMTI suspend points, and not >>>>> offer a JVMTI suspend point in >>>>> SharedRuntime::resolve_static_call_C and friends at all. >>>>> >>>>> dl >>>>> >>>>> >>>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review the fix for >>>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>>> webrev: >>>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>>> >>>>>> Description: >>>>>> The test suspends a thread, turns on single stepping and then >>>>>> calls PopFrame. SingleStep event is expected as soon as the >>>>>> thread is resumed and PopFrame is processed (so we have call >>>>>> stack with the depth 1 less than it was before PopFrame). Instead >>>>>> SingleStep event is received with much deeper call stack (and >>>>>> PopFrame processes wrong frame). >>>>>> Research shown that this is caused by missed deoptimization of >>>>>> the current frame. >>>>>> As far as I understand CompilationPolicy::event should handle the >>>>>> case when the thread has is_interp_only_mode() set, but >>>>>> TieredThresholdPolicy::event checks this only then CompLevel is >>>>>> CompLevel_none. >>>>>> CompilerRuntime always calls policy()->event with CompLevel == >>>>>> CompLevel_aot. >>>>>> >>>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>>> runtime and compiler teams as I'm not sure I completely >>>>>> understand all the details of the "PopFrame dance". >>>>>> >>>>>> --alex >>>>> >>>> >> > From serguei.spitsyn at oracle.com Tue Nov 27 23:47:22 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 27 Nov 2018 15:47:22 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> Message-ID: On 11/27/18 3:38 PM, serguei.spitsyn at oracle.com wrote: > Good summary with a list of solutions below! > > On 11/27/18 2:50 PM, dean.long at oracle.com wrote: >> Let me list the proposed solutions: >> >> 1. short-circuit compiles in CompilationPolicy::event in >> is_interp_only_mode >> 2. detect is_interp_only_mode in resolve stubs and force a c2i call >> 3. don't offer a JVMTI suspend point in resolve stubs (or >> JavaCallWrapper) >> >> Solution 1 penalizes other threads, and don't prevent other threads >> from compiling the method, so while it may help a little, it's >> incomplete. > > I doubt, the Solution 1 penalizes other threads. > First, it is an important pretty frequent/common case that allows to > save on compilations. > Second, there is no point to compile methods on a thread executed in > interp_only_mode. > Also, there is no big advantage to compile methods even for other threads > when an active debugging (e.g. single stepping) is in progress. > Third, this would make it consistent with what the the interpreter is > doing. > > So, I think, the Solution 1 is needed independently of other solutions > we take. > My suggestion is to apply it in the JDK-8195639 bug fix. > >> Solutions 1 and 2 allow the thread to resume in a different method >> (the callee method), causing other problems. > > Agreed. > >> With Solution 3, the frame we suspend is the same as the frame we end >> up in after the resume, so I believe it solves all the problems. >> IMHO this is the correct solution to pursue. > > I agree in general. > This solution has some risks involved as it impacts a lot of code > including platform-independent. A correction: I wanted to say "platform-dependent". Thanks, Serguei > It is also going to fix the other bugs: JDK-8195635, JDK-8214093, > JDK-8207013 (not sure about all of them yet). > > My suggestion is to pursue it as the JDK-8195635 fix. > Could you, please, confirm if works for you? > > Also, we have no consensus yet on the Solution 1. > > Thanks, > Serguei > >> dl >> >> On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Dean, >>> >>> You also asked the questions: >>> >>> > Doesn't is_interp_only_mode() only apply to the current thread? >>> >>> Yes it does. >>> >>> >>> > I don't think it's safe to assume no compiles will happen in other >>> threads, >>> > or that a method call target is not already compiled, because as far >>> > as I can tell, JVMTI only deoptimizes the active frames. >>> >>> I agree with it. >>> However, compilations on the thread with the interp_only_mode >>> enabled is the most important case. >>> Disabling compilations such thread improves testing stability. >>> >>> > The bug report describes a case where the caller has been deoptimized >>> > while resolving a call.? I don't see how this fix prevents the target >>> > from being a previously compiled method. >>> > But we do need to make sure not to call into compiled code, so I >>> think >>> > the fix needs to be in code like >>> SharedRuntime::resolve_static_call_C(), >>> > where it returns get_c2i_entry() if is_interp_only_mode() is true. >>> >>> Thank you for making this point. >>> >>> It would be nice to attack this as well. >>> We can try to investigate this approach further. >>> One problem is that there are more cases like resolve_static_call_C, >>> for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. >>> We may need to fix the same in other places, like >>> JavaCallWrapper::JavaCallWrapper. >>> >>> We can start from fixing it in the resolve_static_call_C. >>> If it is successful then continue this effort for other cases as well. >>> >>> What do you think? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/27/18 1:01 PM, Alex Menkov wrote: >>>> Hi Dean, >>>> >>>> Thank you for the feedback and for the comments in Jira. >>>> >>>> >> How does this solve the problem of >>>> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>> >>>> It doesn't. It fixes another issue. >>>> The test suspend test thread and ensures that top frame is a >>>> "fibonachi" method. Then it turns SingleStep on, does PopFrame and >>>> resumes the thread. >>>> The problem is the thread continues execution of compiled code >>>> (ignoring SingleStep) and we get SindleStep event only when >>>> adjustCompilationLevel method is called (it's interpreted code). >>>> >>>> There are several other bugs which are (most likely) caused by >>>> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >>>> >>>> --alex >>>> >>>> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>>>> Hi Dean, >>>>> >>>>> Thank you a lot for looking at this! >>>>> Just a couple of points from me (it is up to Alex to provide a >>>>> full answer). >>>>> >>>>> >>>>> I think, Alex in this RFR missed to tell that we knew about this >>>>> issue that an incorrect frame will be popped. >>>>> But after some discussion we decided to file a separate issue on >>>>> this. >>>>> Alex wanted to create a good stand-alone test case demonstrating >>>>> this problem before filing it. >>>>> >>>>> Now, as I can see, the JDK-8195635 >>>>> looks very >>>>> close to a bug that we wanted to file. >>>>> The only difference is that our scenario includes the >>>>> SharedRuntime::resolve_static_call_C instead of the >>>>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>>>> invocation. >>>>> If I understand corrctly (Alex will fix me if needed), the jtreg >>>>> test we used to chase this issue did not catch a problem with the >>>>> HotSpotJVMCIRuntime.adjustCompilationLevel. >>>>> >>>>> >>>>> The suggested fix was to fix the mismatch in the >>>>> TieredThresholdPolicy::even with the comment in the interpreter code: >>>>> nmethod* >>>>> InterpreterRuntime::frequency_counter_overflow(JavaThread* thread, >>>>> address branch_bcp) { >>>>> ?????. . . >>>>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>>>> ?????// Normally we never get an nm if is_interp_only_mode() is >>>>> true, because >>>>> ?????// policy()->event has a check for this and won't compile the >>>>> method when >>>>> ?????// true. However, it's possible for is_interp_only_mode() to >>>>> become true >>>>> ?????// during the compilation. We don't want to return the nm in >>>>> that case >>>>> ?????// because we want to continue to execute interpreted. >>>>> ?????nm = NULL; >>>>> ???} >>>>> >>>>> ?> So I think the fix needs to be in code like >>>>> SharedRuntime::resolve_static_call_C(), >>>>> ?> where it returns get_c2i_entry() if is_interp_only_mode() is true. >>>>> >>>>> I'm not sure, the adding this condition and returning the >>>>> get_c2i_entry() is always correct. >>>>> We need some help from the Compiler team to make it right. >>>>> >>>>> BTW, the interp_only_mode has been enabled only when some >>>>> interp_only events are enabled. >>>>> It is not set by either PopFrame or ForceEarlyReturn. >>>>> But the popframe009 test enables single stepping, so we wanted to >>>>> make this part right. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>>>> How does this solve the problem of >>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>> >>>>>> I don't think this fix is the right approach.? Doesn't >>>>>> is_interp_only_mode() only apply to the current thread? I don't >>>>>> think it's safe to assume no compiles will happen in other >>>>>> threads, or that a method call target is not already compiled, >>>>>> because as far as I can tell, JVMTI only deoptimizes the active >>>>>> frames.? The bug report describes a case where the caller has >>>>>> been deoptimized while resolving a call.? I don't see how this >>>>>> fix prevents the target from being a previously compiled method.? >>>>>> But we do need to make sure not to call into compiled code, so I >>>>>> think the fix needs to be in code like >>>>>> SharedRuntime::resolve_static_call_C(), where it returns >>>>>> get_c2i_entry() if is_interp_only_mode() is true. However, there >>>>>> is still another problem.? By allowing JVMTI to suspend the >>>>>> thread during call setup, but reporting the frame as still in the >>>>>> caller instead of the callee, we confuse JVMTI into thinking that >>>>>> execution will resume in the caller instead of the callee.? We >>>>>> may want to restrict where we offer JVMTI suspend points, and not >>>>>> offer a JVMTI suspend point in >>>>>> SharedRuntime::resolve_static_call_C and friends at all. >>>>>> >>>>>> dl >>>>>> >>>>>> >>>>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review the fix for >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>>>> >>>>>>> Description: >>>>>>> The test suspends a thread, turns on single stepping and then >>>>>>> calls PopFrame. SingleStep event is expected as soon as the >>>>>>> thread is resumed and PopFrame is processed (so we have call >>>>>>> stack with the depth 1 less than it was before PopFrame). >>>>>>> Instead SingleStep event is received with much deeper call stack >>>>>>> (and PopFrame processes wrong frame). >>>>>>> Research shown that this is caused by missed deoptimization of >>>>>>> the current frame. >>>>>>> As far as I understand CompilationPolicy::event should handle >>>>>>> the case when the thread has is_interp_only_mode() set, but >>>>>>> TieredThresholdPolicy::event checks this only then CompLevel is >>>>>>> CompLevel_none. >>>>>>> CompilerRuntime always calls policy()->event with CompLevel == >>>>>>> CompLevel_aot. >>>>>>> >>>>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>>>> runtime and compiler teams as I'm not sure I completely >>>>>>> understand all the details of the "PopFrame dance". >>>>>>> >>>>>>> --alex >>>>>> >>>>> >>> >> > From coleen.phillimore at oracle.com Tue Nov 27 23:57:18 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 27 Nov 2018 18:57:18 -0500 Subject: Extend NMT to JDK native libraries? In-Reply-To: References: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> <22785794-bee7-98e9-2151-c8487f65744a@redhat.com> Message-ID: On 11/26/18 3:43 PM, Thomas St?fe wrote: > Hi Zhengyu, > > > On Wed, Nov 21, 2018 at 5:26 PM Zhengyu Gu wrote: >>> But note the unbalanced-malloc issue. The more we expose NMT to the >>> JDK wild, the more probable are such bugs. Within the hotspot all is >>> nice and tidy. >> Hotspot also has a few unbalanced-malloc instances. > Oh, where? Really, raw malloc with os::free, or the other way around? > > Would they not create the problem I described, corrupted C-heap etc? > >> And yes, we will >> find more in library, I would consider it as initial investment to fix >> them, especially, if we can do them module-by-module, no? > Yes, they should be fixed if possible. My concern is that it is > realistic to assume that we miss some, or even if not that new ones > will creep in if the code changes. And that the penalties for these > unbalanced mallocs is severe - crashes and heap corruptions - if we > use malloc headers. > > Even worse, they would only show up, in release builds, if NMT is > switched on, since only NMT adds malloc headers in release builds. > That would be annoying, telling a customer to switch on NMT just for > him to hit such a code path and crash the VM. > > I don't know, I seem to miss something here. Do you not see this as > serious problem preventing NMT from covering code outside hotspot? It seems like it would be a lot more effort than it is worth and probably very hard to get bug-free.? I don't think we currently have any mismatched os::malloc -> ::free or vice versa calls in the vm now.? There are a couple intentional ::malloc/::free pairs though. For tracking memory outside of the JVM, it seems like tools like valgrind or other OS specific tools are probably a lot better than NMT, and somebody has already written and debugged them. Coleen > >> Mismatched statistics is quite annoying ... Did see people actually >> counting bytes and expecting to match :-) JDK-8191369 actually was >> driven by customers, who tried to match smap. >> > I remember :) and yes, we have many curious customers too. But that is > fine. We want NMT to be as exact as possible. > > Cheers, Thomas > >> Thanks, >> >> -Zhengyu >> >>> If we wanted to avoid these bugs, we would have to remove malloc >>> headers from both os::malloc() and NMT MallocTracker and add a malloc >>> pointer hashtable of some sorts to the latter. This is not very >>> difficult, but would still need an initial investment. >> >> >>> Thanks, Thomas >>> >>>> Thanks, >>>> >>>> -Zhengyu >>>> >>>>> (I think even if we were to instrument parts of the JDK - e.g. just >>>>> NIO - this would already be very helpful. In parts we do this already >>>>> for mtOther.). >>>>> >>>>> On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu wrote: >>>>>> FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) >>>>>> https://bugs.openjdk.java.net/browse/JDK-6995789 >>>>>> >>>>>> -Zhengyu >>>>>> >>>>>> On 11/21/18 9:28 AM, Thomas St?fe wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> (yet again not sure if this is serviceablity-dev or not - I start at >>>>>>> hs-dev, feel free to move this mail around.) >>>>>>> >>>>>>> Do we have any plans to extend NMT to cover native JDK libaries too? >>>>>>> That would be a really cool feature. >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> We at SAP have done a similar thing in the past: >>>>>>> >>>>>>> We have a monitoring facility in our port which tracks C-heap >>>>>>> allocations, non-imaginatively called "malloc statistic". This feature >>>>>>> predates NMT somewhat - had we had NMT at that time, we would not have >>>>>>> bothered. Our Malloc statistic is less powerful than NMT and >>>>>>> implementation-wise completely at odds with it, so I never felt the >>>>>>> urge to bring it upstream. However, one thing we did do is we extended >>>>>>> its coverage to the JDK native code. >>>>>>> >>>>>>> This has been quite helpful in the past to find leaks in JDK, see >>>>>>> e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 >>>>>>> >>>>>>> We did this by exposing os::malloc, os::free etc from libjvm.so >>>>>>> ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we >>>>>>> then either manually replaced calls to raw ::malloc(), ::free() etc >>>>>>> with JVM_malloc(), JVM_free(). Or, in places where this was possible, >>>>>>> we did this replacement stuff wholesale by employing a header which >>>>>>> re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, >>>>>>> we also had to add a number of linkage dependencies to the libjvm.so. >>>>>>> >>>>>>> All this is pretty standard stuff. >>>>>>> >>>>>>> One detail stood out: malloc headers are evil. In our experience, JDK >>>>>>> native code was more difficult to control and "unbalanced >>>>>>> malloc/frees" kept creeping in - especially with the >>>>>>> wholesale-redefinition technique. Unbalanced mallocs/frees means cases >>>>>>> where malloc() is instrumented but ::free() stays raw, or the other >>>>>>> way around. Both combinations are catastrophic since os::malloc uses >>>>>>> malloc headers. We typically corrupted the C-Heap and crashed, often >>>>>>> much later in completely unrelated places. >>>>>>> >>>>>>> These types of bugs were very hard to spot and hence very expensive. >>>>>>> And they can creep in in many ways. One example, there exist a >>>>>>> surprising number of system APIs which return results in C-heap and >>>>>>> require the user to free that, which of course must happen with raw >>>>>>> ::free(), not os::free(). >>>>>>> >>>>>>> We fixed this by not using malloc headers. That means a pointer >>>>>>> returned by os::malloc() is compatible with raw ::free() and vice >>>>>>> versa. The only bad thing happening would be our statistic numbers >>>>>>> being slightly off. >>>>>>> >>>>>>> Instead of malloc headers we use a hand-groomed hash table to track >>>>>>> the malloced memory. It is actually quite fast, fast enough that this >>>>>>> malloc statistic feature is on-by-default in our port. >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Of course, if we extend NMT to JDK native code we also would want to >>>>>>> extend it to mmap() etc - we never did this with our statistic, since >>>>>>> it only tracked malloc. >>>>>>> >>>>>>> What do you think? Did anyone else play with similar ideas? Would it >>>>>>> be worth the effort? >>>>>>> >>>>>>> Cheers, Thomas >>>>>>> From kim.barrett at oracle.com Wed Nov 28 00:07:20 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 27 Nov 2018 19:07:20 -0500 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: <39D4E0BC-9669-43FE-A28D-34E46F8CB268@oracle.com> > On Nov 26, 2018, at 4:39 PM, Roman Kennke wrote: > *) shared-gc > - This is mostly boilerplate that is common to any GC > - referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1) > - taskqueue.hpp has some small adjustments to enable subclassing I've reviewed the shared-gc webrev. I only found a few trivial nits. ------------------------------------------------------------------------------ src/hotspot/share/gc/shared/gcName.hpp 42 NA, 43 Shenandoah, Putting Shenandoah after NA seems odd. ------------------------------------------------------------------------------ src/hotspot/share/gc/shared/gcConfig.cpp 63 CMSGC_ONLY(static CMSArguments cmsArguments;) ... 69 SHENANDOAHGC_ONLY(static ShenandoahArguments shenandoahArguments;) Code alignment should probably be updated. Similarly here: 73 static const SupportedGC SupportedGCs[] = { ... 79 SHENANDOAHGC_ONLY_ARG(SupportedGC(UseShenandoahGC, CollectedHeap::Shenandoah, shenandoahArguments, "shenandoah gc")) and here: 97 void GCConfig::fail_if_unsupported_gc_is_selected() { ... 105 NOT_SHENANDOAHGC(FAIL_IF_SELECTED(UseShenandoahGC, true)); ------------------------------------------------------------------------------ src/hotspot/share/gc/shared/collectedHeap.hpp 92 // ShenandoahHeap Moving it after ParallelScavengeHeap would give a better order. ------------------------------------------------------------------------------ src/hotspot/share/gc/shared/barrierSetConfig.hpp 36 SHENANDOAHGC_ONLY(f(Shenandoah)) Why is this "Shenandoah" while all the others are "BarrierSet"? ------------------------------------------------------------------------------ From jcbeyler at google.com Wed Nov 28 00:36:47 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 27 Nov 2018 16:36:47 -0800 Subject: RFR (L) 8213501 : Deploy ExceptionJniWrapper for a few tests In-Reply-To: <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> References: <94fa815b-b34c-1025-8f52-cda3b44996cb@oracle.com> <43547532-1435-d86b-ba8e-55d19aac40d2@oracle.com> <7c816cdd-6fbf-d258-7a93-62b22a812d14@oracle.com> <4eb234a8-ef05-f143-57b8-0e0258492f2d@oracle.com> <349fac38-cfde-f35a-8b48-1d03a057b4bf@oracle.com> Message-ID: Hi Chris, Yes I was waiting for another review since you had explicitly asked :) And sounds good that when someone from GC or runtime gives a review, I'll wait for your full review on the webrev.02! Thanks again for your help, Jc On Tue, Nov 27, 2018 at 12:48 PM Chris Plummer wrote: > Hi JC, > > I think it would be good to get a review from the gc or runtime teams, > since this also affects their tests. > > Also, once we are settled on this FatalError approach, I still need to > give your webrev-02 a full review. I only skimmed over parts of it (I did > look at all the changes in webrevo-01). > > thanks, > > Chris > > On 11/27/18 8:58 AM, serguei.spitsyn at oracle.com wrote: > > Hi Jc, > > I've already reviewed this too. > > Thanks, > Serguei > > > On 11/27/18 06:56, JC Beyler wrote: > > Thanks Chris, > > Anybody else motivated to look at this and review it? :) > Jc > > On Mon, Nov 26, 2018 at 1:26 PM Chris Plummer > wrote: > >> Hi JC, >> >> I'm ok with the FatalError approach, but would like to hear opinions from >> others also. >> >> thanks, >> >> Chris >> >> On 11/21/18 8:19 AM, JC Beyler wrote: >> >> Hi Chris, >> >> Thanks for taking the time to look at it and yes you have raised exactly >> why the webrev is between two worlds: in cases where a fatal error on >> failure is wanted, should we simplify the code to remove the return tests >> since we do them internally? Now that I've looked around for non-fatal >> cases, I think the answer is yes, it simplifies the code while maintaining >> the checks. >> >> I looked a bit and it seems that I can't find easily a case where the >> test accepts a JNI failure to then move on. Therefore, perhaps, for now, >> the fail with a Fatal is enough and we can work on the tests to clean them >> up? >> >> That means that this is the new webrev with only Fatal and cleans up the >> tests so that it is no longer in between two worlds: >> >> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >> >> (This passes testing on my dev machine for all the modified tests) >> >> with the example you provided, it now looks like: >> >> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.02/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >> >> Where it does, to me at least, seem cleaner and less "noisy". >> >> Let me know what you think, >> Jc >> >> >> On Tue, Nov 20, 2018 at 9:33 PM Chris Plummer >> wrote: >> >>> Hi JC, >>> >>> Sorry about the delay. I had to go back an look at the initial 8210842 >>> webrev and RFR thread to see what this was initially all about. >>> >>> In general the changes look good. >>> >>> I don't have a good answer to your FatalError/NonFatalError question. It >>> makes the code a lot cleaner to use FatalError, but then it is a behavior >>> change, and you also need to deal with tests that intentionally induce >>> errors (do you have an example of that). >>> >>> In any case, right now your webrev seems to be between two worlds. You >>> are producing FatalError, but still checking results. Here's a good example: >>> >>> >>> http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01/test/hotspot/jtreg/vmTestbase/nsk/jvmti/scenarios/allocation/AP04/ap04t003/ap04t003.cpp.frames.html >>> >>> I'm not sure if this is just a temporary state until it was decided >>> which approach to take. >>> >>> thanks, >>> >>> Chris >>> >>> >>> On 11/20/18 2:14 PM, JC Beyler wrote: >>> >>> Hi all, >>> >>> Chris thought it made sense to have more eyes on this change than just >>> serviceability as it will modify to tests that are not only serviceability >>> tests so I've moved this to conversation here :) >>> >>> For convenience, I've copy-pasted the initial RFR: >>> >>> Could I have a review for the extension and usage of the >>> ExceptionJniWrapper. This adds lines and filenames to the end of the >>> wrapper JNI methods, adds tracing, and throws an error if need be. I've >>> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >>> ported a few of the tests that were already changed for the assignment >>> webrev for JDK-8212884. >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.01 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>> >>> For illustration, if I force an error to the AP04/ap04t03 test and set >>> the verbosity on, I get something like: >>> >>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>> >> Calling with these parameter(s): >>> java/lang/Threadd >>> Wait for thread to finish >>> << Called JNI method FindClass from ap04t003.cpp:343 >>> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >>> java/lang/Threadd >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> Method) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >>> at >>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>> at >>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>> at >>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>> ... 3 more >>> FATAL ERROR in native method: JNI method FindClass : internal error from >>> ap04t003.cpp:343 >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>> Method) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>> at >>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>> >>> Questions/comments I have about this are: >>> - Do we want to force fatal errors when a JNI call fails in general? >>> Most of these tests do the right thing and test the return of the JNI >>> calls, for example: >>> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >>> if (thrClass == NULL) { >>> >>> but now the wrapper actually would do a fatal if the FindClass call >>> would return a nullptr, so we could remove that test altogether. What do >>> you think? >>> - I prefer to leave them as the tests then become closer to what >>> real users would have in their code and is the "recommended" way of doing it >>> >>> >>> - The alternative is to use the NonFatalError I added which then just >>> prints out that something went wrong, letting the test continue. Question >>> will be what should be the default? The fatal or the non-fatal error >>> handling? >>> >>> On a different subject: >>> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >>> wrapper handles the tracing and the verify in almost the same way; only >>> difference I can really tell is that the complain method from NSK has a max >>> complain before stopping to "complain"; I have not added that part of the >>> code in this webrev >>> >>> Once we decide on these, I can continue on the files from JDK-8212884 >>> and then do both the assignment in an if extraction followed-by this type >>> of webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can >>> be deprecated as well as we go forward. >>> >>> Thanks! >>> Jc >>> >>> On Mon, Nov 19, 2018 at 11:34 AM Chris Plummer >>> wrote: >>> >>>> On 11/19/18 10:07 AM, JC Beyler wrote: >>>> >>>> Hi all, >>>> >>>> @David/Chris: should I then push this RFR to the hotspot mailing or the >>>> runtime one? For what it's worth, a lot of the tests under the vmTestbase >>>> are jvmti so the review also affects serviceability; it just turns out I >>>> started with the GC originally and then hit some other tests I had touched >>>> via the assignment extraction. >>>> >>>> I think hotspot would be best. >>>> >>>> Chris >>>> >>>> >>>> @Serguei: Done for the method renaming, for the indent, are you talking >>>> about going from the 8-indent to 4-indent? If so, would it not just be >>>> better to do a new JBS bug and do the whole files in one go? I ask because >>>> otherwise, it will look a bit weird to have parts of the file as 8-indent >>>> and others 4-indent? >>>> >>>> Thanks for looking at it! >>>> Jc >>>> >>>> On Mon, Nov 19, 2018 at 1:25 AM serguei.spitsyn at oracle.com < >>>> serguei.spitsyn at oracle.com> wrote: >>>> >>>>> Hi Jc, >>>>> >>>>> We have to start this review anyway. :) >>>>> It looks good to me in general. >>>>> Thank you for your consistency in this refactoring! >>>>> >>>>> Some minor comments. >>>>> >>>>> http://cr.openjdk.java.net/%7Ejcbeyler/8213501/webrev.00/test/hotspot/jtreg/vmTestbase/nsk/share/jni/ExceptionCheckingJniEnv.cpp.udiff.html >>>>> >>>>> +static const char* remove_folders(const char* fullname) { >>>>> >>>>> I'd suggest to rename the function name to something traditional like get_basename. >>>>> Otherwise, it sounds like this function has to really remove folders. :) >>>>> >>>>> >>>>> Also, all *Locker.cpp have wrong indent in the bodies of if and while statements. >>>>> Could this be fixed with the refactoring? >>>>> >>>>> I did not look on how this impacts the tests other than serviceability. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 11/16/18 19:43, JC Beyler wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Anybody motivated to review this? :) >>>>> Jc >>>>> >>>>> On Wed, Nov 7, 2018 at 9:53 PM JC Beyler wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> Could I have a review for the extension and usage of the >>>>>> ExceptionJniWrapper. This adds lines and filenames to the end of the >>>>>> wrapper JNI methods, adds tracing, and throws an error if need be. I've >>>>>> ported the gc/lock files to use the new TRACE_JNI_CALL add-on and I've >>>>>> ported a few of the tests that were already changed for the assignment >>>>>> webrev for JDK-8212884. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8213501/webrev.00/ >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8213501 >>>>>> >>>>>> For illustration, if I force an error to the AP04/ap04t03 test and >>>>>> set the verbosity on, I get something like: >>>>>> >>>>>> >> Calling JNI method FindClass from ap04t003.cpp:343 >>>>>> >> Calling with these parameter(s): >>>>>> java/lang/Threadd >>>>>> Wait for thread to finish >>>>>> << Called JNI method FindClass from ap04t003.cpp:343 >>>>>> Exception in thread "Thread-0" java.lang.NoClassDefFoundError: >>>>>> java/lang/Threadd >>>>>> at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> Method) >>>>>> at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> Caused by: java.lang.ClassNotFoundException: java.lang.Threadd >>>>>> at >>>>>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) >>>>>> at >>>>>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) >>>>>> at >>>>>> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) >>>>>> ... 3 more >>>>>> FATAL ERROR in native method: JNI method FindClass : internal error >>>>>> from ap04t003.cpp:343 >>>>>> at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003.runIterateOverHeap(Native >>>>>> Method) >>>>>> at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003HeapIterator.runIteration(ap04t003.java:140) >>>>>> at >>>>>> nsk.jvmti.scenarios.allocation.AP04.ap04t003Thread.run(ap04t003.java:201) >>>>>> >>>>>> Questions/comments I have about this are: >>>>>> - Do we want to force fatal errors when a JNI call fails in >>>>>> general? Most of these tests do the right thing and test the return of the >>>>>> JNI calls, for example: >>>>>> thrClass = jni->FindClass("java/lang/Threadd", TRACE_JNI_CALL); >>>>>> if (thrClass == NULL) { >>>>>> >>>>>> but now the wrapper actually would do a fatal if the FindClass call >>>>>> would return a nullptr, so we could remove that test altogether. What do >>>>>> you think? >>>>>> - I prefer to leave them as the tests then become closer to what >>>>>> real users would have in their code and is the "recommended" way of doing it >>>>>> >>>>>> - The alternative is to use the NonFatalError I added which then >>>>>> just prints out that something went wrong, letting the test continue. >>>>>> Question will be what should be the default? The fatal or the non-fatal >>>>>> error handling? >>>>>> >>>>>> On a different subject: >>>>>> - On the new tests, I've removed the NSK_JNI_VERIFY since the JNI >>>>>> wrapper handles the tracing and the verify in almost the same way; only >>>>>> difference I can really tell is that the complain method from NSK has a max >>>>>> complain before stopping to "complain"; I have not added that part of the >>>>>> code in this webrev >>>>>> >>>>>> Once we decide on these, I can continue on the files from JDK-8212884 >>>>>> and then do both the assignment in an if extraction followed-by this type >>>>>> of webrev in an easier fashion. Depending on decisions here, NSK*VERIFY can >>>>>> be deprecated as well as we go forward. >>>>>> >>>>>> Thank you for the reviews/comments :) >>>>>> Jc >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> Jc >>>>> >>>>> >>>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> Jc >>>> >>>> >>>> >>> >>> -- >>> >>> Thanks, >>> Jc >>> >>> >>> >> >> -- >> >> Thanks, >> Jc >> >> >> > > -- > > Thanks, > Jc > > > > -- Thanks, Jc From david.holmes at oracle.com Wed Nov 28 04:17:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Nov 2018 14:17:15 +1000 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> Message-ID: <5291cfa5-65ab-8dcf-44e4-e2253141343a@oracle.com> Hi Robbin, Part 2: I've looked at the generic waitBarrier implementation and it seems okay. A lot of the usage constraints not clear in the API are more evident in the asserts in the code. I still have some doubts about the benefit of helping with the wakeups rather than it all being done by one thread, due to potential contention on the semaphore - but your performance numbers show it can help (I just hope it doesn't hurt on other platforms/systems - time will tell). Some minor nits with comments: - src/hotspot/share/utilities/waitBarrier_generic.hpp 31 // Except for the barrier tag it self, it uses two counter to keep the semaphore 32 // count correct and not leave any late thread hanging. s/it self/itself/ s/counter/counters/ suggest s/hanging/waiting/ 38 // Which means it can become a waiter. suggest -> // These threads can become waiters. --- - src/hotspot/share/utilities/waitBarrier_generic.cpp 55 // We need an exact count and never go below 0. 56 // Otherwise the semaphore might contain to many posts. suggest: // We need an exact count which never goes below zero, // otherwise the semaphore may be signalled too many times. 74 // We must loop here until there is no waiters or potential waiters. s/is/are/ More to follow. Thanks, David On 24/11/2018 2:55 am, Robbin Ehn wrote: > Forgot RFR in subject. > > /Robbin > > On 2018-11-23 17:51, Robbin Ehn wrote: >> Hi all, please review. >> >> When a safepoint is ended we need a way to get back to 100% >> utilization as fast >> as possible. 100% utilization means no idle cpu in the system if there >> is a >> JavaThread that could be executed. The traditional ways to wake many, >> e.g. >> semaphore, pthread_cond, is not implemented with a single syscall >> instead they >> typical do one syscall per thread to wake. >> >> This change-set contains that primitive, the WaitBarrier, and a gtest >> for it. >> No actual users, which is in coming patches. >> >> The WaitBarrier solves by doing a cooperative semaphore posting, >> threads woken >> will also post. On Linux we can instead directly use a futex and with one >> syscall wake all. Depending on how many threads and cpus the >> performance vary, >> but a good utilization of the machine, just on the edge of saturated, >> the time to reach 100% utilization is around 3 times faster with the >> WaitBarrier (where futex is faster than semaphore). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8214271 >> >> Passes 100 iterations of gtest on our platforms, both fastdebug and >> release. >> And have been stable when used in safepoints (t1-8) (coming patches). >> >> Thanks, Robbin From david.holmes at oracle.com Wed Nov 28 05:41:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 28 Nov 2018 15:41:15 +1000 Subject: RFR(m): 8214271: Fast primitive to wake many threads In-Reply-To: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> References: <010211e3-93a6-80b9-678c-c84b08812e43@oracle.com> Message-ID: Hi Robbin, Part 3 (and final part) This is looking at the futex implementation and the test. Overall looks good! Shouldn't you be using the FUTEX_*_PRIVATE variants as this is a process-local FUTEX? Your main code would be simpler to read (and thus understand) if your futex helper function handled the unused arguments e.g. instead of: static int futex(volatile int *uaddr, int futex_op, int val, const struct timespec *timeout, int *uaddr2, int val3) { return syscall(SYS_futex, uaddr, futex_op, val, timeout, uaddr2, val3); } use (with name changes): static int futex(volatile int *addr, int futex_op, int op_arg) { return syscall(SYS_futex, addr, futex_op, op_arg, NULL, NULL, 0); } --- Minor suggestions on this comment block: 88 // Return value 0, re-check in case of spurious wake-up. 89 // EINTR and re-check and go back to waiting. 90 // EAGAIN we already are disarmed, we should pass the check, 91 // if not re-armed with same tag. -> // Return value 0: woken up, but re-check in case of spurious wakeup // Error EINTR: woken by signal, so re-check and re-wait if necessary. // Error EAGAIN: we are already disarmed and so will pass the check As per initial review comments I think re-arming with same tag should be an error. I think the tag semantics should be that the tag is a strictly increasing value, and we can then store the previous tag and check that. --- The test could do with at least an introductory comment block explaining the basic operation of the test. Further inline comments wouldn't hurt either. Thanks, David On 24/11/2018 2:55 am, Robbin Ehn wrote: > Forgot RFR in subject. > > /Robbin > > On 2018-11-23 17:51, Robbin Ehn wrote: >> Hi all, please review. >> >> When a safepoint is ended we need a way to get back to 100% >> utilization as fast >> as possible. 100% utilization means no idle cpu in the system if there >> is a >> JavaThread that could be executed. The traditional ways to wake many, >> e.g. >> semaphore, pthread_cond, is not implemented with a single syscall >> instead they >> typical do one syscall per thread to wake. >> >> This change-set contains that primitive, the WaitBarrier, and a gtest >> for it. >> No actual users, which is in coming patches. >> >> The WaitBarrier solves by doing a cooperative semaphore posting, >> threads woken >> will also post. On Linux we can instead directly use a futex and with one >> syscall wake all. Depending on how many threads and cpus the >> performance vary, >> but a good utilization of the machine, just on the edge of saturated, >> the time to reach 100% utilization is around 3 times faster with the >> WaitBarrier (where futex is faster than semaphore). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8214271/webrev/ >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8214271 >> >> Passes 100 iterations of gtest on our platforms, both fastdebug and >> release. >> And have been stable when used in safepoints (t1-8) (coming patches). >> >> Thanks, Robbin From goetz.lindenmaier at sap.com Wed Nov 28 07:38:20 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 28 Nov 2018 07:38:20 +0000 Subject: 8213481: [REDO] Fix incorrect copy constructors in hotspot In-Reply-To: <4F249D17-A348-4BB2-A19C-ECFFC6AB80B8@oracle.com> References: <9A8D73B6-FA6D-414E-B22D-C14666D6B618@oracle.com> <4F249D17-A348-4BB2-A19C-ECFFC6AB80B8@oracle.com> Message-ID: Hi Kim, we have seen various issues with xlc and acc wrt. to constructors/destructors. xlc still issues a row of warnings, which we disabled so that the builds passes. Therefore I thought I better make sure it works :) All the other builds are fine, too. Best regards, Goetz. > -----Original Message----- > From: Kim Barrett > Sent: Dienstag, 27. November 2018 19:03 > To: Lindenmaier, Goetz > Cc: hotspot-dev developers > Subject: Re: 8213481: [REDO] Fix incorrect copy constructors in hotspot > > > On Nov 27, 2018, at 6:51 AM, Lindenmaier, Goetz > wrote: > > > > Hi, > > > > I built aix slowdebug, fastdebug and product builds successfully. > > The other builds run tonight. > > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: Lindenmaier, Goetz > >> Sent: Tuesday, November 27, 2018 9:10 AM > >> To: 'Kim Barrett' ; hotspot-dev developers > >> > >> Subject: RE: 8213481: [REDO] Fix incorrect copy constructors in hotspot > >> > >> Hi Kim, > >> > >> I put this into our build system to assure our compilers grok this code. > >> AIX is building already, all other platforms will build tonight > >> (European night??). I think xlc is the most likely to cause issues, > >> On linux we are on gcc 7.3 anyways, which should be fine. > >> > >> Best regards, > >> Goetz. > > Thanks for doing this testing. I wasn?t really expecting problems, since the > code is substantially similar to what was there before, but it?s good to have > that confirmed. From matthias.baesken at sap.com Wed Nov 28 08:18:19 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 28 Nov 2018 08:18:19 +0000 Subject: RFR : [XS] 8214373: adjust usage of ReleaseLongArrayElements in MacosxDebuggerLocal In-Reply-To: References: Message-ID: Hi Thomas and JC , thanks for the reviews ! Best regards, Matthias From: JC Beyler Sent: Dienstag, 27. November 2018 16:48 To: Baesken, Matthias Cc: hotspot-dev Source Developers ; serviceability-dev at openjdk.java.net Subject: Re: RFR : [XS] 8214373: adjust usage of ReleaseLongArrayElements in MacosxDebuggerLocal Hi Matthias, Not a reviewer but looks good to me :). I checked if other releases were not in the right place and they all seemed good as well. Jc On Tue, Nov 27, 2018 at 7:09 AM Baesken, Matthias > wrote: Hello, please review this small fix . In MacosxDebuggerLocal.m , at a few places the functions GetLongArrayElements and ReleaseLongArrayElements are used. However there is one place where in an early return the ReleaseLongArrayElements call has been forgotten . Additionally at another place ReleaseLongArrayElements is at the wrong place (although this is probably not a big issue currently). Bug/webrev: https://bugs.openjdk.java.net/browse/JDK-8214373 http://cr.openjdk.java.net/~mbaesken/webrevs/8214373.0/ Thanks, Matthias -- Thanks, Jc From Yang.Zhang at arm.com Wed Nov 28 08:32:15 2018 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Wed, 28 Nov 2018 08:32:15 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> Message-ID: Hi Andrew, Vladimir I update the reviewers and remove the new assert in Matcher::vector_shift_count_ideal_reg(). Could you please help review again? http://cr.openjdk.java.net/~yzhang/8213134/webrev.02/ Regards Yang -----Original Message----- From: Vladimir Kozlov Sent: Thursday, November 22, 2018 1:56 AM To: Yang Zhang (Arm Technology China) ; hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 You are right. I think I was confused that you are not limiting MaxVectorSize but simple use min(16, MaxVectorSize). The assert is fine. Thanks, Vladimir On 11/20/18 10:28 PM, Yang Zhang (Arm Technology China) wrote: > >> I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. >> In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). >> aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. > > Matcher::vector_width_in_bytes() limits vector size to min(16, MaxVectorSize). > Matcher::vector_shift_count_ideal_reg() assert(MaxVectorSize >= size, > "Length isn't supported"); When MaxVectorSize is 8, size is 8. > When MaxVectorSize is greater than or equal 16, size is 16. > So (MaxVectorSize >= size) is always true. > > Test cases of cr6340864 are passed with or without this new assert. > > Do I need to remove this new assert? > > Regards > Yang > > > On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: >> Hi, >> >> When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? >> >> Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ >> JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 >> >> This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. >> Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. >> compiler/c2/cr6340864/TestByteVect.java >> compiler/c2/cr6340864/TestIntVect.java >> compiler/c2/cr6340864/TestShortVect.java >> compiler/codegen/TestCharVect2.java >> >> Regards >> Yang >> >> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. From goetz.lindenmaier at sap.com Wed Nov 28 09:00:22 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 28 Nov 2018 09:00:22 +0000 Subject: RFR(S) [downport 11]: 8213410: UseCompressedOops requirement check fails fails on 32-bit system Message-ID: <0a52708b571f4c729664900b4a7e5bd1@sap.com> Hi, I need a review to downport 8213410. 8213410 fixes disabling tests for UseCompressedOops on 32-bit platforms. In jdk12, "@requires vm.bits == 64" was introduced to CompressedClassPointers.java by "8208655: use JTreg skipped status in hotspot tests". Because of this change, 8213410 does not apply properly. To fix this, I introduce "@requires vm.bits == 64" to this test in jdk11, too. http://cr.openjdk.java.net/~goetz/wr18/8213410-32bit_cOops-jdk11/01/ Best regards, Goetz. From dean.long at oracle.com Wed Nov 28 09:07:10 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 28 Nov 2018 01:07:10 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> Message-ID: OK, reconsidering solution 1, I have a couple more questions: The bug report says the problem is reproducible with -Xcomp, but I don't see the policy()->event() path taken with -Xcomp.? Instead, -Xcomp uses CompilationPolicy::compile_if_required, so I don't see how Solution 1 fixes the problem with -Xcomp. > CompilerRuntime always calls policy()->event with CompLevel == CompLevel_aot. As far as I can tell, CompLevel_aot is only used when there is active AOT code.? Is this an AOT-specific problem?? I would expect there to be a problem with c1 as well. dl On 11/27/18 3:38 PM, serguei.spitsyn at oracle.com wrote: > Good summary with a list of solutions below! > > On 11/27/18 2:50 PM, dean.long at oracle.com wrote: >> Let me list the proposed solutions: >> >> 1. short-circuit compiles in CompilationPolicy::event in >> is_interp_only_mode >> 2. detect is_interp_only_mode in resolve stubs and force a c2i call >> 3. don't offer a JVMTI suspend point in resolve stubs (or >> JavaCallWrapper) >> >> Solution 1 penalizes other threads, and don't prevent other threads >> from compiling the method, so while it may help a little, it's >> incomplete. > > I doubt, the Solution 1 penalizes other threads. > First, it is an important pretty frequent/common case that allows to > save on compilations. > Second, there is no point to compile methods on a thread executed in > interp_only_mode. > Also, there is no big advantage to compile methods even for other threads > when an active debugging (e.g. single stepping) is in progress. > Third, this would make it consistent with what the the interpreter is > doing. > > So, I think, the Solution 1 is needed independently of other solutions > we take. > My suggestion is to apply it in the JDK-8195639 bug fix. > >> Solutions 1 and 2 allow the thread to resume in a different method >> (the callee method), causing other problems. > > Agreed. > >> With Solution 3, the frame we suspend is the same as the frame we end >> up in after the resume, so I believe it solves all the problems. >> IMHO this is the correct solution to pursue. > > I agree in general. > This solution has some risks involved as it impacts a lot of code > including platform-independent. > It is also going to fix the other bugs: JDK-8195635, JDK-8214093, > JDK-8207013 (not sure about all of them yet). > > My suggestion is to pursue it as the JDK-8195635 fix. > Could you, please, confirm if works for you? > > Also, we have no consensus yet on the Solution 1. > > Thanks, > Serguei > >> dl >> >> On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: >>> Hi Dean, >>> >>> You also asked the questions: >>> >>> > Doesn't is_interp_only_mode() only apply to the current thread? >>> >>> Yes it does. >>> >>> >>> > I don't think it's safe to assume no compiles will happen in other >>> threads, >>> > or that a method call target is not already compiled, because as far >>> > as I can tell, JVMTI only deoptimizes the active frames. >>> >>> I agree with it. >>> However, compilations on the thread with the interp_only_mode >>> enabled is the most important case. >>> Disabling compilations such thread improves testing stability. >>> >>> > The bug report describes a case where the caller has been deoptimized >>> > while resolving a call.? I don't see how this fix prevents the target >>> > from being a previously compiled method. >>> > But we do need to make sure not to call into compiled code, so I >>> think >>> > the fix needs to be in code like >>> SharedRuntime::resolve_static_call_C(), >>> > where it returns get_c2i_entry() if is_interp_only_mode() is true. >>> >>> Thank you for making this point. >>> >>> It would be nice to attack this as well. >>> We can try to investigate this approach further. >>> One problem is that there are more cases like resolve_static_call_C, >>> for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. >>> We may need to fix the same in other places, like >>> JavaCallWrapper::JavaCallWrapper. >>> >>> We can start from fixing it in the resolve_static_call_C. >>> If it is successful then continue this effort for other cases as well. >>> >>> What do you think? >>> >>> Thanks, >>> Serguei >>> >>> >>> On 11/27/18 1:01 PM, Alex Menkov wrote: >>>> Hi Dean, >>>> >>>> Thank you for the feedback and for the comments in Jira. >>>> >>>> >> How does this solve the problem of >>>> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>> >>>> It doesn't. It fixes another issue. >>>> The test suspend test thread and ensures that top frame is a >>>> "fibonachi" method. Then it turns SingleStep on, does PopFrame and >>>> resumes the thread. >>>> The problem is the thread continues execution of compiled code >>>> (ignoring SingleStep) and we get SindleStep event only when >>>> adjustCompilationLevel method is called (it's interpreted code). >>>> >>>> There are several other bugs which are (most likely) caused by >>>> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >>>> >>>> --alex >>>> >>>> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>>>> Hi Dean, >>>>> >>>>> Thank you a lot for looking at this! >>>>> Just a couple of points from me (it is up to Alex to provide a >>>>> full answer). >>>>> >>>>> >>>>> I think, Alex in this RFR missed to tell that we knew about this >>>>> issue that an incorrect frame will be popped. >>>>> But after some discussion we decided to file a separate issue on >>>>> this. >>>>> Alex wanted to create a good stand-alone test case demonstrating >>>>> this problem before filing it. >>>>> >>>>> Now, as I can see, the JDK-8195635 >>>>> looks very >>>>> close to a bug that we wanted to file. >>>>> The only difference is that our scenario includes the >>>>> SharedRuntime::resolve_static_call_C instead of the >>>>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>>>> invocation. >>>>> If I understand corrctly (Alex will fix me if needed), the jtreg >>>>> test we used to chase this issue did not catch a problem with the >>>>> HotSpotJVMCIRuntime.adjustCompilationLevel. >>>>> >>>>> >>>>> The suggested fix was to fix the mismatch in the >>>>> TieredThresholdPolicy::even with the comment in the interpreter code: >>>>> nmethod* >>>>> InterpreterRuntime::frequency_counter_overflow(JavaThread* thread, >>>>> address branch_bcp) { >>>>> ?????. . . >>>>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>>>> ?????// Normally we never get an nm if is_interp_only_mode() is >>>>> true, because >>>>> ?????// policy()->event has a check for this and won't compile the >>>>> method when >>>>> ?????// true. However, it's possible for is_interp_only_mode() to >>>>> become true >>>>> ?????// during the compilation. We don't want to return the nm in >>>>> that case >>>>> ?????// because we want to continue to execute interpreted. >>>>> ?????nm = NULL; >>>>> ???} >>>>> >>>>> ?> So I think the fix needs to be in code like >>>>> SharedRuntime::resolve_static_call_C(), >>>>> ?> where it returns get_c2i_entry() if is_interp_only_mode() is true. >>>>> >>>>> I'm not sure, the adding this condition and returning the >>>>> get_c2i_entry() is always correct. >>>>> We need some help from the Compiler team to make it right. >>>>> >>>>> BTW, the interp_only_mode has been enabled only when some >>>>> interp_only events are enabled. >>>>> It is not set by either PopFrame or ForceEarlyReturn. >>>>> But the popframe009 test enables single stepping, so we wanted to >>>>> make this part right. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>>>> How does this solve the problem of >>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>> >>>>>> I don't think this fix is the right approach.? Doesn't >>>>>> is_interp_only_mode() only apply to the current thread? I don't >>>>>> think it's safe to assume no compiles will happen in other >>>>>> threads, or that a method call target is not already compiled, >>>>>> because as far as I can tell, JVMTI only deoptimizes the active >>>>>> frames.? The bug report describes a case where the caller has >>>>>> been deoptimized while resolving a call.? I don't see how this >>>>>> fix prevents the target from being a previously compiled method.? >>>>>> But we do need to make sure not to call into compiled code, so I >>>>>> think the fix needs to be in code like >>>>>> SharedRuntime::resolve_static_call_C(), where it returns >>>>>> get_c2i_entry() if is_interp_only_mode() is true. However, there >>>>>> is still another problem.? By allowing JVMTI to suspend the >>>>>> thread during call setup, but reporting the frame as still in the >>>>>> caller instead of the callee, we confuse JVMTI into thinking that >>>>>> execution will resume in the caller instead of the callee.? We >>>>>> may want to restrict where we offer JVMTI suspend points, and not >>>>>> offer a JVMTI suspend point in >>>>>> SharedRuntime::resolve_static_call_C and friends at all. >>>>>> >>>>>> dl >>>>>> >>>>>> >>>>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review the fix for >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>>>> webrev: >>>>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>>>> >>>>>>> Description: >>>>>>> The test suspends a thread, turns on single stepping and then >>>>>>> calls PopFrame. SingleStep event is expected as soon as the >>>>>>> thread is resumed and PopFrame is processed (so we have call >>>>>>> stack with the depth 1 less than it was before PopFrame). >>>>>>> Instead SingleStep event is received with much deeper call stack >>>>>>> (and PopFrame processes wrong frame). >>>>>>> Research shown that this is caused by missed deoptimization of >>>>>>> the current frame. >>>>>>> As far as I understand CompilationPolicy::event should handle >>>>>>> the case when the thread has is_interp_only_mode() set, but >>>>>>> TieredThresholdPolicy::event checks this only then CompLevel is >>>>>>> CompLevel_none. >>>>>>> CompilerRuntime always calls policy()->event with CompLevel == >>>>>>> CompLevel_aot. >>>>>>> >>>>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>>>> runtime and compiler teams as I'm not sure I completely >>>>>>> understand all the details of the "PopFrame dance". >>>>>>> >>>>>>> --alex >>>>>> >>>>> >>> >> > From serguei.spitsyn at oracle.com Wed Nov 28 09:27:03 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 28 Nov 2018 01:27:03 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> Message-ID: On 11/28/18 01:07, dean.long at oracle.com wrote: > OK, reconsidering solution 1, I have a couple more questions: > > The bug report says the problem is reproducible with -Xcomp, but I > don't see the policy()->event() path taken with -Xcomp. Instead, > -Xcomp uses CompilationPolicy::compile_if_required, so I don't see how > Solution 1 fixes the problem with -Xcomp. > > > CompilerRuntime always calls policy()->event with CompLevel == > CompLevel_aot. > > As far as I can tell, CompLevel_aot is only used when there is active > AOT code.? Is this an AOT-specific problem? Tracing showed that the CompLevel_aot is always used for JVMCI which is pretty confusing. Most likely, it is because now the AOT is enabled by default (is it correct?). I'm not sure it actually has anything to do with the AOT mode. > ? I would expect there to be a problem with c1 as well. Not sure about the c1. I hope, Alex could comment on this tomorrow. Thanks, Serguei > dl > > On 11/27/18 3:38 PM, serguei.spitsyn at oracle.com wrote: >> Good summary with a list of solutions below! >> >> On 11/27/18 2:50 PM, dean.long at oracle.com wrote: >>> Let me list the proposed solutions: >>> >>> 1. short-circuit compiles in CompilationPolicy::event in >>> is_interp_only_mode >>> 2. detect is_interp_only_mode in resolve stubs and force a c2i call >>> 3. don't offer a JVMTI suspend point in resolve stubs (or >>> JavaCallWrapper) >>> >>> Solution 1 penalizes other threads, and don't prevent other threads >>> from compiling the method, so while it may help a little, it's >>> incomplete. >> >> I doubt, the Solution 1 penalizes other threads. >> First, it is an important pretty frequent/common case that allows to >> save on compilations. >> Second, there is no point to compile methods on a thread executed in >> interp_only_mode. >> Also, there is no big advantage to compile methods even for other >> threads >> when an active debugging (e.g. single stepping) is in progress. >> Third, this would make it consistent with what the the interpreter is >> doing. >> >> So, I think, the Solution 1 is needed independently of other >> solutions we take. >> My suggestion is to apply it in the JDK-8195639 bug fix. >> >>> Solutions 1 and 2 allow the thread to resume in a different method >>> (the callee method), causing other problems. >> >> Agreed. >> >>> With Solution 3, the frame we suspend is the same as the frame we >>> end up in after the resume, so I believe it solves all the problems. >>> IMHO this is the correct solution to pursue. >> >> I agree in general. >> This solution has some risks involved as it impacts a lot of code >> including platform-independent. >> It is also going to fix the other bugs: JDK-8195635, JDK-8214093, >> JDK-8207013 (not sure about all of them yet). >> >> My suggestion is to pursue it as the JDK-8195635 fix. >> Could you, please, confirm if works for you? >> >> Also, we have no consensus yet on the Solution 1. >> >> Thanks, >> Serguei >> >>> dl >>> >>> On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: >>>> Hi Dean, >>>> >>>> You also asked the questions: >>>> >>>> > Doesn't is_interp_only_mode() only apply to the current thread? >>>> >>>> Yes it does. >>>> >>>> >>>> > I don't think it's safe to assume no compiles will happen in >>>> other threads, >>>> > or that a method call target is not already compiled, because as far >>>> > as I can tell, JVMTI only deoptimizes the active frames. >>>> >>>> I agree with it. >>>> However, compilations on the thread with the interp_only_mode >>>> enabled is the most important case. >>>> Disabling compilations such thread improves testing stability. >>>> >>>> > The bug report describes a case where the caller has been >>>> deoptimized >>>> > while resolving a call.? I don't see how this fix prevents the >>>> target >>>> > from being a previously compiled method. >>>> > But we do need to make sure not to call into compiled code, so I >>>> think >>>> > the fix needs to be in code like >>>> SharedRuntime::resolve_static_call_C(), >>>> > where it returns get_c2i_entry() if is_interp_only_mode() is true. >>>> >>>> Thank you for making this point. >>>> >>>> It would be nice to attack this as well. >>>> We can try to investigate this approach further. >>>> One problem is that there are more cases like resolve_static_call_C, >>>> for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. >>>> We may need to fix the same in other places, like >>>> JavaCallWrapper::JavaCallWrapper. >>>> >>>> We can start from fixing it in the resolve_static_call_C. >>>> If it is successful then continue this effort for other cases as well. >>>> >>>> What do you think? >>>> >>>> Thanks, >>>> Serguei >>>> >>>> >>>> On 11/27/18 1:01 PM, Alex Menkov wrote: >>>>> Hi Dean, >>>>> >>>>> Thank you for the feedback and for the comments in Jira. >>>>> >>>>> >> How does this solve the problem of >>>>> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>> >>>>> It doesn't. It fixes another issue. >>>>> The test suspend test thread and ensures that top frame is a >>>>> "fibonachi" method. Then it turns SingleStep on, does PopFrame and >>>>> resumes the thread. >>>>> The problem is the thread continues execution of compiled code >>>>> (ignoring SingleStep) and we get SindleStep event only when >>>>> adjustCompilationLevel method is called (it's interpreted code). >>>>> >>>>> There are several other bugs which are (most likely) caused by >>>>> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >>>>> >>>>> --alex >>>>> >>>>> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Thank you a lot for looking at this! >>>>>> Just a couple of points from me (it is up to Alex to provide a >>>>>> full answer). >>>>>> >>>>>> >>>>>> I think, Alex in this RFR missed to tell that we knew about this >>>>>> issue that an incorrect frame will be popped. >>>>>> But after some discussion we decided to file a separate issue on >>>>>> this. >>>>>> Alex wanted to create a good stand-alone test case demonstrating >>>>>> this problem before filing it. >>>>>> >>>>>> Now, as I can see, the JDK-8195635 >>>>>> looks very >>>>>> close to a bug that we wanted to file. >>>>>> The only difference is that our scenario includes the >>>>>> SharedRuntime::resolve_static_call_C instead of the >>>>>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>>>>> invocation. >>>>>> If I understand corrctly (Alex will fix me if needed), the jtreg >>>>>> test we used to chase this issue did not catch a problem with the >>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel. >>>>>> >>>>>> >>>>>> The suggested fix was to fix the mismatch in the >>>>>> TieredThresholdPolicy::even with the comment in the interpreter >>>>>> code: >>>>>> nmethod* >>>>>> InterpreterRuntime::frequency_counter_overflow(JavaThread* >>>>>> thread, address branch_bcp) { >>>>>> ?????. . . >>>>>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>>>>> ?????// Normally we never get an nm if is_interp_only_mode() is >>>>>> true, because >>>>>> ?????// policy()->event has a check for this and won't compile >>>>>> the method when >>>>>> ?????// true. However, it's possible for is_interp_only_mode() to >>>>>> become true >>>>>> ?????// during the compilation. We don't want to return the nm in >>>>>> that case >>>>>> ?????// because we want to continue to execute interpreted. >>>>>> ?????nm = NULL; >>>>>> ???} >>>>>> >>>>>> ?> So I think the fix needs to be in code like >>>>>> SharedRuntime::resolve_static_call_C(), >>>>>> ?> where it returns get_c2i_entry() if is_interp_only_mode() is >>>>>> true. >>>>>> >>>>>> I'm not sure, the adding this condition and returning the >>>>>> get_c2i_entry() is always correct. >>>>>> We need some help from the Compiler team to make it right. >>>>>> >>>>>> BTW, the interp_only_mode has been enabled only when some >>>>>> interp_only events are enabled. >>>>>> It is not set by either PopFrame or ForceEarlyReturn. >>>>>> But the popframe009 test enables single stepping, so we wanted to >>>>>> make this part right. >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>>>>> How does this solve the problem of >>>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>>> >>>>>>> I don't think this fix is the right approach.? Doesn't >>>>>>> is_interp_only_mode() only apply to the current thread? I don't >>>>>>> think it's safe to assume no compiles will happen in other >>>>>>> threads, or that a method call target is not already compiled, >>>>>>> because as far as I can tell, JVMTI only deoptimizes the active >>>>>>> frames. The bug report describes a case where the caller has >>>>>>> been deoptimized while resolving a call.? I don't see how this >>>>>>> fix prevents the target from being a previously compiled >>>>>>> method.? But we do need to make sure not to call into compiled >>>>>>> code, so I think the fix needs to be in code like >>>>>>> SharedRuntime::resolve_static_call_C(), where it returns >>>>>>> get_c2i_entry() if is_interp_only_mode() is true. However, there >>>>>>> is still another problem.? By allowing JVMTI to suspend the >>>>>>> thread during call setup, but reporting the frame as still in >>>>>>> the caller instead of the callee, we confuse JVMTI into thinking >>>>>>> that execution will resume in the caller instead of the callee.? >>>>>>> We may want to restrict where we offer JVMTI suspend points, and >>>>>>> not offer a JVMTI suspend point in >>>>>>> SharedRuntime::resolve_static_call_C and friends at all. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>> >>>>>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review the fix for >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>>>>> webrev: >>>>>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>>>>> >>>>>>>> Description: >>>>>>>> The test suspends a thread, turns on single stepping and then >>>>>>>> calls PopFrame. SingleStep event is expected as soon as the >>>>>>>> thread is resumed and PopFrame is processed (so we have call >>>>>>>> stack with the depth 1 less than it was before PopFrame). >>>>>>>> Instead SingleStep event is received with much deeper call >>>>>>>> stack (and PopFrame processes wrong frame). >>>>>>>> Research shown that this is caused by missed deoptimization of >>>>>>>> the current frame. >>>>>>>> As far as I understand CompilationPolicy::event should handle >>>>>>>> the case when the thread has is_interp_only_mode() set, but >>>>>>>> TieredThresholdPolicy::event checks this only then CompLevel is >>>>>>>> CompLevel_none. >>>>>>>> CompilerRuntime always calls policy()->event with CompLevel == >>>>>>>> CompLevel_aot. >>>>>>>> >>>>>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>>>>> runtime and compiler teams as I'm not sure I completely >>>>>>>> understand all the details of the "PopFrame dance". >>>>>>>> >>>>>>>> --alex >>>>>>> >>>>>> >>>> >>> >> > From per.liden at oracle.com Wed Nov 28 10:12:38 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 Nov 2018 11:12:38 +0100 Subject: Call for reviews: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: Message-ID: Hi Roman, Under "Non-Goals" the JEP says: "The GC pause time is bounded by the time required to scan all of the thread stacks[...]" This is not quite true, is it? I'd say the theoretical worst cases pause in Shenandoah is still bound by the live-set and heap size. I can think of a few scenarios: 1) Reference processing is still done in a pause, but the theoretical worst case scenario for reference processing is bound by the live-set size. For example, marking of FinalReference objects during reference processing can in the worst case require marking the whole live-set (and in non-worst cases a substantial part of the live-set). 2) Weak root cleaning (e.g. StringTable/JNIGlobalRefs) is still done in a pause, but this has a theoretical worst case scenario of being bound by the heap size. For example, every object on the heap is a String which has been interned, so the time to scan/clean the StringTable will be bound by the heap size. cheers, Per On 11/27/18 11:52 AM, Roman Kennke wrote: > Hi there, > > I just noticed that we have only one 'Reviewed-by:' entry in the > Shenandoah JEP. I know that several people have reviewed it in the past, > as I've incorporated several suggestions into the JEP. I believe the JEP > cannot move forward to 'proposed to target' without at least one more > added reviewer. > > https://bugs.openjdk.java.net/browse/JDK-8046179 > > If you have reviewed the JEP in the past, and/or feel like reviewing it > now, please don't forget to add your 'Reviewed-by:' entry. I believe any > Reviewer familiar with the relevant field of expertise (GC and/or > Hotspot in this case) can review it and add his name to it. > > Thanks! > Roman > From rkennke at redhat.com Wed Nov 28 10:48:09 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 11:48:09 +0100 Subject: Call for reviews: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: Message-ID: <0dd6e61b-744c-1db4-67db-d7394f86daf8@redhat.com> Hi Per, > Under "Non-Goals" the JEP says: > > "The GC pause time is bounded by the time required to scan all of the > thread stacks[...]" > > This is not quite true, is it? I'd say the theoretical worst cases pause > in Shenandoah is still bound by the live-set and heap size. > > I can think of a few scenarios: > > 1) Reference processing is still done in a pause, but the theoretical > worst case scenario for reference processing is bound by the live-set > size. For example, marking of FinalReference objects during reference > processing can in the worst case require marking the whole live-set (and > in non-worst cases a substantial part of the live-set). > > 2) Weak root cleaning (e.g. StringTable/JNIGlobalRefs) is still done in > a pause, but this has a theoretical worst case scenario of being bound > by the heap size. For example, every object on the heap is a String > which has been interned, so the time to scan/clean the StringTable will > be bound by the heap size. Ok, right. What do you suggest then? Mention those scenarios in the JEP? We don't intend to implement concurrent stringtable/weakrefs/etc processing at this point. We might do so in the future, but that is not certain either. Roman > cheers, > Per > > On 11/27/18 11:52 AM, Roman Kennke wrote: >> Hi there, >> >> I just noticed that we have only one 'Reviewed-by:' entry in the >> Shenandoah JEP. I know that several people have reviewed it in the past, >> as I've incorporated several suggestions into the JEP. I believe the JEP >> cannot move forward to 'proposed to target' without at least one more >> added reviewer. >> >> https://bugs.openjdk.java.net/browse/JDK-8046179 >> >> If you have reviewed the JEP in the past, and/or feel like reviewing it >> now, please don't forget to add your 'Reviewed-by:' entry. I believe any >> Reviewer familiar with the relevant field of expertise (GC and/or >> Hotspot in this case) can review it and add his name to it. >> >> Thanks! >> Roman >> From robbin.ehn at oracle.com Wed Nov 28 10:48:30 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 28 Nov 2018 11:48:30 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <86d894d1-6a32-3a1a-01e9-1919f1efceae@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> <5BF423D2.1070800@oracle.com> <5BF819B1.7040908@oracle.com> <43818dbd-5008-b3a0-7045-2d314e47f5fa@oracle.com> <5BF81BBC.2060606@oracle.com> <86d894d1-6a32-3a1a-01e9-1919f1efceae@oracle.com> Message-ID: <73bef199-d1f1-23e7-372b-5bb152431443@oracle.com> > Incremental: > http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01_02/ Still good! > > Full: > http://cr.openjdk.java.net/~eosterlund/8213565/webrev.02/ > > Still need one more reviewer for this change. Also bump :) /Robbin > > /Erik > > On 2018-11-23 16:24, Erik ?sterlund wrote: >> Hi Robbin, >> >> Thanks for the review. >> >> /Erik >> >> On 2018-11-23 16:26, Robbin Ehn wrote: >>> Looks good, thanks! >>> >>> /Robbin >>> >>> On 2018-11-23 16:16, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> Here is an updated version with some minor adjustments based on feedback from Robbin Ehn. >>>> >>>> Incremental: >>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00_01/ >>>> >>>> Full: >>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01/ >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-11-20 16:10, Erik ?sterlund wrote: >>>>> Ping. >>>>> >>>>> /Erik >>>>> >>>>> On 2018-11-12 23:02, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> In my change 8209189, I moved a lot of code around relating to code cache unloading. Unfortunately, I failed to >>>>>> see that when making nmethods unloaded, and their dependents are flushed... they are not immediately removed. >>>>>> Instead, they are tagged to have stale entries, that are later cleaned during weak metadata cleaning of >>>>>> InstanceKlass. As for method handles, they leak instead, with some hacks to reduce the leaking by expunging >>>>>> entries while adding and removing. >>>>>> >>>>>> Obviously, with InstanceKlass cleanup and code cache unloading now >>>>>> possibly running in parallel after 8209189, things can go wrong. These two phases used >>>>>> to be separated by a "wait barrier" for the worker threads preventing that parallelism. >>>>>> >>>>>> The reason dependency contexts were not cleaned when they were found >>>>>> during code cache cleaning, is because it was not thread safe when code >>>>>> cache unloading was made parallel instead of serial. But now that we are >>>>>> implementing concurrent class unloading, we are going to need concurrent >>>>>> cleanup of dependency contexts anyway. Therefore I will propose a bug fix that fixes the problem in a way that >>>>>> works for both serial, parallel and concurrent class unloading. >>>>>> >>>>>> The solution is to during code cache unloading claim cleaning of >>>>>> encountered stale dependency contexts, and clean them straight away. >>>>>> Entries are unlinked in a lock-free fashion, and placed on a purge list that is walked and deleted during >>>>>> ClassLoaderDataGraph::purge(). This follows the thinking of first unlinking, then syncing all threads, and then >>>>>> purging. >>>>>> >>>>>> New accessors for the head and next links, hide is_unloading entries and helps unlinking them and putting them >>>>>> into the purge list. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8213565 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>> >>>> >> From per.liden at oracle.com Wed Nov 28 11:01:39 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 Nov 2018 12:01:39 +0100 Subject: Call for reviews: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <0dd6e61b-744c-1db4-67db-d7394f86daf8@redhat.com> References: <0dd6e61b-744c-1db4-67db-d7394f86daf8@redhat.com> Message-ID: Hi Roman, On 11/28/18 11:48 AM, Roman Kennke wrote: > Hi Per, > > >> Under "Non-Goals" the JEP says: >> >> "The GC pause time is bounded by the time required to scan all of the >> thread stacks[...]" >> >> This is not quite true, is it? I'd say the theoretical worst cases pause >> in Shenandoah is still bound by the live-set and heap size. >> >> I can think of a few scenarios: >> >> 1) Reference processing is still done in a pause, but the theoretical >> worst case scenario for reference processing is bound by the live-set >> size. For example, marking of FinalReference objects during reference >> processing can in the worst case require marking the whole live-set (and >> in non-worst cases a substantial part of the live-set). >> >> 2) Weak root cleaning (e.g. StringTable/JNIGlobalRefs) is still done in >> a pause, but this has a theoretical worst case scenario of being bound >> by the heap size. For example, every object on the heap is a String >> which has been interned, so the time to scan/clean the StringTable will >> be bound by the heap size. > > Ok, right. What do you suggest then? Mention those scenarios in the JEP? > We don't intend to implement concurrent stringtable/weakrefs/etc > processing at this point. We might do so in the future, but that is not > certain either. I'm not sure I have a really a good answer. In the "Non-Goals" section I guess you could just leave that sentence out completely. The "Success Metrics" now says: "This project will be a success if we can keep consistent short gc pause times regardless of heap size." And I guess that would kind of need to be adjusted too. Maybe something like: "This project will be a success if average GC pause times are keep short." cheers, Per > > Roman > >> cheers, >> Per >> >> On 11/27/18 11:52 AM, Roman Kennke wrote: >>> Hi there, >>> >>> I just noticed that we have only one 'Reviewed-by:' entry in the >>> Shenandoah JEP. I know that several people have reviewed it in the past, >>> as I've incorporated several suggestions into the JEP. I believe the JEP >>> cannot move forward to 'proposed to target' without at least one more >>> added reviewer. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8046179 >>> >>> If you have reviewed the JEP in the past, and/or feel like reviewing it >>> now, please don't forget to add your 'Reviewed-by:' entry. I believe any >>> Reviewer familiar with the relevant field of expertise (GC and/or >>> Hotspot in this case) can review it and add his name to it. >>> >>> Thanks! >>> Roman >>> > From rkennke at redhat.com Wed Nov 28 11:44:23 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 12:44:23 +0100 Subject: Call for reviews: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <0dd6e61b-744c-1db4-67db-d7394f86daf8@redhat.com> Message-ID: <08dd5d08-c81c-ede5-485b-44762e8672f8@redhat.com> Hi Per, I removed the sentence under Non-Goals and changed the sentence under success metrics to remove the heap-size part: "This project will be a success if we can keep consistent short gc pause times." Does that sound good enough? Roman > Hi Roman, > > On 11/28/18 11:48 AM, Roman Kennke wrote: >> Hi Per, >> >> >>> Under "Non-Goals" the JEP says: >>> >>> "The GC pause time is bounded by the time required to scan all of the >>> thread stacks[...]" >>> >>> This is not quite true, is it? I'd say the theoretical worst cases pause >>> in Shenandoah is still bound by the live-set and heap size. >>> >>> I can think of a few scenarios: >>> >>> 1) Reference processing is still done in a pause, but the theoretical >>> worst case scenario for reference processing is bound by the live-set >>> size. For example, marking of FinalReference objects during reference >>> processing can in the worst case require marking the whole live-set (and >>> in non-worst cases a substantial part of the live-set). >>> >>> 2) Weak root cleaning (e.g. StringTable/JNIGlobalRefs) is still done in >>> a pause, but this has a theoretical worst case scenario of being bound >>> by the heap size. For example, every object on the heap is a String >>> which has been interned, so the time to scan/clean the StringTable will >>> be bound by the heap size. >> >> Ok, right. What do you suggest then? Mention those scenarios in the JEP? >> We don't intend to implement concurrent stringtable/weakrefs/etc >> processing at this point. We might do so in the future, but that is not >> certain either. > > I'm not sure I have a really a good answer. In the "Non-Goals" section I > guess you could just leave that sentence out completely. The "Success > Metrics" now says: > > "This project will be a success if we can keep consistent short gc pause > times regardless of heap size." > > And I guess that would kind of need to be adjusted too. Maybe something > like: > > "This project will be a success if average GC pause times are keep short." > > cheers, > Per > >> >> Roman >> >>> cheers, >>> Per >>> >>> On 11/27/18 11:52 AM, Roman Kennke wrote: >>>> Hi there, >>>> >>>> I just noticed that we have only one 'Reviewed-by:' entry in the >>>> Shenandoah JEP. I know that several people have reviewed it in the >>>> past, >>>> as I've incorporated several suggestions into the JEP. I believe the >>>> JEP >>>> cannot move forward to 'proposed to target' without at least one more >>>> added reviewer. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8046179 >>>> >>>> If you have reviewed the JEP in the past, and/or feel like reviewing it >>>> now, please don't forget to add your 'Reviewed-by:' entry. I believe >>>> any >>>> Reviewer familiar with the relevant field of expertise (GC and/or >>>> Hotspot in this case) can review it and add his name to it. >>>> >>>> Thanks! >>>> Roman >>>> >> From per.liden at oracle.com Wed Nov 28 12:15:48 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 Nov 2018 13:15:48 +0100 Subject: Call for reviews: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <08dd5d08-c81c-ede5-485b-44762e8672f8@redhat.com> References: <0dd6e61b-744c-1db4-67db-d7394f86daf8@redhat.com> <08dd5d08-c81c-ede5-485b-44762e8672f8@redhat.com> Message-ID: Hi Roman, On 11/28/18 12:44 PM, Roman Kennke wrote: > Hi Per, > > I removed the sentence under Non-Goals and changed the sentence under > success metrics to remove the heap-size part: > > "This project will be a success if we can keep consistent short gc pause > times." > > Does that sound good enough? Sure, good enough for me. In general, I'd recommend you to under promise and over deliver when in comes to success metrics in the JEP. Otherwise you're just exposing yourself to the risk that someone claims that you haven't fulfilled the success metrics and that the project isn't ready to to in yet. cheers, Per > > Roman > >> Hi Roman, >> >> On 11/28/18 11:48 AM, Roman Kennke wrote: >>> Hi Per, >>> >>> >>>> Under "Non-Goals" the JEP says: >>>> >>>> "The GC pause time is bounded by the time required to scan all of the >>>> thread stacks[...]" >>>> >>>> This is not quite true, is it? I'd say the theoretical worst cases pause >>>> in Shenandoah is still bound by the live-set and heap size. >>>> >>>> I can think of a few scenarios: >>>> >>>> 1) Reference processing is still done in a pause, but the theoretical >>>> worst case scenario for reference processing is bound by the live-set >>>> size. For example, marking of FinalReference objects during reference >>>> processing can in the worst case require marking the whole live-set (and >>>> in non-worst cases a substantial part of the live-set). >>>> >>>> 2) Weak root cleaning (e.g. StringTable/JNIGlobalRefs) is still done in >>>> a pause, but this has a theoretical worst case scenario of being bound >>>> by the heap size. For example, every object on the heap is a String >>>> which has been interned, so the time to scan/clean the StringTable will >>>> be bound by the heap size. >>> >>> Ok, right. What do you suggest then? Mention those scenarios in the JEP? >>> We don't intend to implement concurrent stringtable/weakrefs/etc >>> processing at this point. We might do so in the future, but that is not >>> certain either. >> >> I'm not sure I have a really a good answer. In the "Non-Goals" section I >> guess you could just leave that sentence out completely. The "Success >> Metrics" now says: >> >> "This project will be a success if we can keep consistent short gc pause >> times regardless of heap size." >> >> And I guess that would kind of need to be adjusted too. Maybe something >> like: >> >> "This project will be a success if average GC pause times are keep short." >> >> cheers, >> Per >> >>> >>> Roman >>> >>>> cheers, >>>> Per >>>> >>>> On 11/27/18 11:52 AM, Roman Kennke wrote: >>>>> Hi there, >>>>> >>>>> I just noticed that we have only one 'Reviewed-by:' entry in the >>>>> Shenandoah JEP. I know that several people have reviewed it in the >>>>> past, >>>>> as I've incorporated several suggestions into the JEP. I believe the >>>>> JEP >>>>> cannot move forward to 'proposed to target' without at least one more >>>>> added reviewer. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8046179 >>>>> >>>>> If you have reviewed the JEP in the past, and/or feel like reviewing it >>>>> now, please don't forget to add your 'Reviewed-by:' entry. I believe >>>>> any >>>>> Reviewer familiar with the relevant field of expertise (GC and/or >>>>> Hotspot in this case) can review it and add his name to it. >>>>> >>>>> Thanks! >>>>> Roman >>>>> >>> > From rkennke at redhat.com Wed Nov 28 12:41:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 13:41:24 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <4A6598C6-7A6E-4CD0-AF47-1F4E9CFE9B89@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <46CBF9C2-7684-4EED-80F8-7103D48FA3A0@oracle.com> <4A6598C6-7A6E-4CD0-AF47-1F4E9CFE9B89@oracle.com> Message-ID: Hi Kim, >> On Nov 27, 2018, at 4:46 AM, Roman Kennke wrote: >> >> Hi Kim, >> >>>> Sections to review (at this point) are the following: >>>> >>>> *) shared-gc >>>> - This is mostly boilerplate that is common to any GC >>>> - referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1) >>> >>> Ick. But I don?t have a better suggestion that doesn?t involve adding a new API >>> to CollectedHeap for use by this assertion, which seems a bit excessive if there >>> are no other uses. >> >> Yeah. >> I guess we could add a config _discovery_is_concurrent or similar in RP, >> and check that. Or maybe one of _discovery_is_mt or _discovery_is_atomic >> already covers that? I couldn't immediately tell/100% understand their >> semantics. Seems worthy to look at after this? > > It might be equivalent to _discovery_is_atomic; I don?t remember the exact > semantics right now. I think it?s unrelated to _discovery_is_mt. > > Yes, looking at this later is fine. Please file an RFE. Ok, done: https://bugs.openjdk.java.net/browse/JDK-8214441 >>>> - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> Why this change instead of JDK-8204947? As the description from that RFE says: >>> "The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.? >> >> Yeah, see Zhengyu's comment. Let's ignore those changes for this review >> (for now), expect our impl ported to taskqueue.hpp/cpp soon. > > I?m okay with that plan. Maybe add a comment in JDK-8204947 about this? https://bugs.openjdk.java.net/browse/JDK-8204947?focusedCommentId=14226307&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14226307 Thanks for reviewing, Roman From rkennke at redhat.com Wed Nov 28 12:42:48 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 13:42:48 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <39D4E0BC-9669-43FE-A28D-34E46F8CB268@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <39D4E0BC-9669-43FE-A28D-34E46F8CB268@oracle.com> Message-ID: <69867881-48fd-3817-54b7-e904f0c1fa4e@redhat.com> Hi Kim, >> On Nov 26, 2018, at 4:39 PM, Roman Kennke wrote: >> *) shared-gc >> - This is mostly boilerplate that is common to any GC >> - referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1) >> - taskqueue.hpp has some small adjustments to enable subclassing > > I've reviewed the shared-gc webrev. I only found a few trivial nits. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/shared/gcName.hpp > 42 NA, > 43 Shenandoah, > > Putting Shenandoah after NA seems odd. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/shared/gcConfig.cpp > 63 CMSGC_ONLY(static CMSArguments cmsArguments;) > ... > 69 SHENANDOAHGC_ONLY(static ShenandoahArguments shenandoahArguments;) > > Code alignment should probably be updated. > > Similarly here: > 73 static const SupportedGC SupportedGCs[] = { > ... > 79 SHENANDOAHGC_ONLY_ARG(SupportedGC(UseShenandoahGC, CollectedHeap::Shenandoah, shenandoahArguments, "shenandoah gc")) > > and here: > 97 void GCConfig::fail_if_unsupported_gc_is_selected() { > ... > 105 NOT_SHENANDOAHGC(FAIL_IF_SELECTED(UseShenandoahGC, true)); > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/shared/collectedHeap.hpp > 92 // ShenandoahHeap > > Moving it after ParallelScavengeHeap would give a better order. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/shared/barrierSetConfig.hpp > 36 SHENANDOAHGC_ONLY(f(Shenandoah)) > > Why is this "Shenandoah" while all the others are "BarrierSet"? > Don't know. I'll change it to be consistent. I have proposed a patch with all changes to shenandoah-dev: http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008420.html Will post an updated upstreaming webrev soon. Thanks for reviewing, Roman From per.liden at oracle.com Wed Nov 28 13:53:22 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 Nov 2018 14:53:22 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: <566619f4-87f5-a605-ca49-8b9083fc17eb@oracle.com> Hi Roman, On 11/26/18 10:39 PM, Roman Kennke wrote: [...] > ?*) shared-serviceability > > ??? - The usual code to support another GC Just had a quick look at the SA part. I was thinking you'd have the same problem as ZGC here, with regards to parsing the heap and potentially reading garbage when you step on a Klass* which had been unloaded? cheers, Per From rkennke at redhat.com Wed Nov 28 14:00:21 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 15:00:21 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <566619f4-87f5-a605-ca49-8b9083fc17eb@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <566619f4-87f5-a605-ca49-8b9083fc17eb@oracle.com> Message-ID: <20c6ccb0-4161-90e9-a6b7-f1aff4c05bef@redhat.com> Hi Per, > Hi Roman, > > On 11/26/18 10:39 PM, Roman Kennke wrote: > [...] >> ??*) shared-serviceability >> >> >> ???? - The usual code to support another GC > > Just had a quick look at the SA part. I was thinking you'd have the same > problem as ZGC here, with regards to parsing the heap and potentially > reading garbage when you step on a Klass* which had been unloaded? Possible. I am myself not very familiar with SA. I guess it depends on how SA does it: if it iterates objects via CH::object_iterate() (e.g. same entry point as, e.g., heap-dumping code), then we should be fine. We're kicking off a traversal rather than straight scan there. If however SA somehow makes a raw scan itself, then we'd have the problem you describe. Roman From thomas.stuefe at gmail.com Wed Nov 28 14:19:08 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 28 Nov 2018 15:19:08 +0100 Subject: Extend NMT to JDK native libraries? In-Reply-To: References: <94b83916-8f4f-f5f3-53ec-6fc0288ac678@redhat.com> <22785794-bee7-98e9-2151-c8487f65744a@redhat.com> Message-ID: Hi Coleen, On Wed, Nov 28, 2018 at 12:59 AM wrote: > > > > On 11/26/18 3:43 PM, Thomas St?fe wrote: > > Hi Zhengyu, > > > > > > On Wed, Nov 21, 2018 at 5:26 PM Zhengyu Gu wrote: > >>> But note the unbalanced-malloc issue. The more we expose NMT to the > >>> JDK wild, the more probable are such bugs. Within the hotspot all is > >>> nice and tidy. > >> Hotspot also has a few unbalanced-malloc instances. > > Oh, where? Really, raw malloc with os::free, or the other way around? > > > > Would they not create the problem I described, corrupted C-heap etc? > > > >> And yes, we will > >> find more in library, I would consider it as initial investment to fix > >> them, especially, if we can do them module-by-module, no? > > Yes, they should be fixed if possible. My concern is that it is > > realistic to assume that we miss some, or even if not that new ones > > will creep in if the code changes. And that the penalties for these > > unbalanced mallocs is severe - crashes and heap corruptions - if we > > use malloc headers. > > > > Even worse, they would only show up, in release builds, if NMT is > > switched on, since only NMT adds malloc headers in release builds. > > That would be annoying, telling a customer to switch on NMT just for > > him to hit such a code path and crash the VM. > > > > I don't know, I seem to miss something here. Do you not see this as > > serious problem preventing NMT from covering code outside hotspot? > > It seems like it would be a lot more effort than it is worth and > probably very hard to get bug-free. I don't think we currently have any > mismatched os::malloc -> ::free or vice versa calls in the vm now. > There are a couple intentional ::malloc/::free pairs though. > > For tracking memory outside of the JVM, it seems like tools like > valgrind or other OS specific tools are probably a lot better than NMT, > and somebody has already written and debugged them. > Well, we did it and got it bug-free 99% of all cases - but you are right, 100% are almost impossible. That is why we avoided malloc headers and instead opted for a hash map to track mallocs. Funny enough, that was not even bad in terms of performance. We leave it on by default. But I understand that you are skeptic about the usefulness of full coverage. We did find a few memory leaks, but that could be found with valgrind too - on nice platforms, on terrible ones like AIX you are on your own :( Which leaves, as a purpose, memory tracking. But for memory tracking, I think we can instrument the JDK piecemeal, as Zhengyu mentioned. Instrumenting big users like mmaps from NIO would already be very useful. Thanks, Thomas > Coleen > > > > > >> Mismatched statistics is quite annoying ... Did see people actually > >> counting bytes and expecting to match :-) JDK-8191369 actually was > >> driven by customers, who tried to match smap. > >> > > I remember :) and yes, we have many curious customers too. But that is > > fine. We want NMT to be as exact as possible. > > > > Cheers, Thomas > > > >> Thanks, > >> > >> -Zhengyu > >> > >>> If we wanted to avoid these bugs, we would have to remove malloc > >>> headers from both os::malloc() and NMT MallocTracker and add a malloc > >>> pointer hashtable of some sorts to the latter. This is not very > >>> difficult, but would still need an initial investment. > >> > >> > >>> Thanks, Thomas > >>> > >>>> Thanks, > >>>> > >>>> -Zhengyu > >>>> > >>>>> (I think even if we were to instrument parts of the JDK - e.g. just > >>>>> NIO - this would already be very helpful. In parts we do this already > >>>>> for mtOther.). > >>>>> > >>>>> On Wed, Nov 21, 2018 at 3:54 PM Zhengyu Gu wrote: > >>>>>> FYI: There was a phase 2 RFE: Native Memory Tracking (Phase 2) > >>>>>> https://bugs.openjdk.java.net/browse/JDK-6995789 > >>>>>> > >>>>>> -Zhengyu > >>>>>> > >>>>>> On 11/21/18 9:28 AM, Thomas St?fe wrote: > >>>>>>> Hi all, > >>>>>>> > >>>>>>> (yet again not sure if this is serviceablity-dev or not - I start at > >>>>>>> hs-dev, feel free to move this mail around.) > >>>>>>> > >>>>>>> Do we have any plans to extend NMT to cover native JDK libaries too? > >>>>>>> That would be a really cool feature. > >>>>>>> > >>>>>>> -- > >>>>>>> > >>>>>>> We at SAP have done a similar thing in the past: > >>>>>>> > >>>>>>> We have a monitoring facility in our port which tracks C-heap > >>>>>>> allocations, non-imaginatively called "malloc statistic". This feature > >>>>>>> predates NMT somewhat - had we had NMT at that time, we would not have > >>>>>>> bothered. Our Malloc statistic is less powerful than NMT and > >>>>>>> implementation-wise completely at odds with it, so I never felt the > >>>>>>> urge to bring it upstream. However, one thing we did do is we extended > >>>>>>> its coverage to the JDK native code. > >>>>>>> > >>>>>>> This has been quite helpful in the past to find leaks in JDK, see > >>>>>>> e.g.: https://bugs.openjdk.java.net/browse/JDK-8155211 > >>>>>>> > >>>>>>> We did this by exposing os::malloc, os::free etc from libjvm.so > >>>>>>> ("JVM_malloc", "JVM_realloc", "JVM_free"). In the JDK native code, we > >>>>>>> then either manually replaced calls to raw ::malloc(), ::free() etc > >>>>>>> with JVM_malloc(), JVM_free(). Or, in places where this was possible, > >>>>>>> we did this replacement stuff wholesale by employing a header which > >>>>>>> re-defined malloc(), free() etc JVM_malloc, JVM_free etc. Of course, > >>>>>>> we also had to add a number of linkage dependencies to the libjvm.so. > >>>>>>> > >>>>>>> All this is pretty standard stuff. > >>>>>>> > >>>>>>> One detail stood out: malloc headers are evil. In our experience, JDK > >>>>>>> native code was more difficult to control and "unbalanced > >>>>>>> malloc/frees" kept creeping in - especially with the > >>>>>>> wholesale-redefinition technique. Unbalanced mallocs/frees means cases > >>>>>>> where malloc() is instrumented but ::free() stays raw, or the other > >>>>>>> way around. Both combinations are catastrophic since os::malloc uses > >>>>>>> malloc headers. We typically corrupted the C-Heap and crashed, often > >>>>>>> much later in completely unrelated places. > >>>>>>> > >>>>>>> These types of bugs were very hard to spot and hence very expensive. > >>>>>>> And they can creep in in many ways. One example, there exist a > >>>>>>> surprising number of system APIs which return results in C-heap and > >>>>>>> require the user to free that, which of course must happen with raw > >>>>>>> ::free(), not os::free(). > >>>>>>> > >>>>>>> We fixed this by not using malloc headers. That means a pointer > >>>>>>> returned by os::malloc() is compatible with raw ::free() and vice > >>>>>>> versa. The only bad thing happening would be our statistic numbers > >>>>>>> being slightly off. > >>>>>>> > >>>>>>> Instead of malloc headers we use a hand-groomed hash table to track > >>>>>>> the malloced memory. It is actually quite fast, fast enough that this > >>>>>>> malloc statistic feature is on-by-default in our port. > >>>>>>> > >>>>>>> -- > >>>>>>> > >>>>>>> Of course, if we extend NMT to JDK native code we also would want to > >>>>>>> extend it to mmap() etc - we never did this with our statistic, since > >>>>>>> it only tracked malloc. > >>>>>>> > >>>>>>> What do you think? Did anyone else play with similar ideas? Would it > >>>>>>> be worth the effort? > >>>>>>> > >>>>>>> Cheers, Thomas > >>>>>>> > From per.liden at oracle.com Wed Nov 28 14:21:51 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 28 Nov 2018 15:21:51 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <20c6ccb0-4161-90e9-a6b7-f1aff4c05bef@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <566619f4-87f5-a605-ca49-8b9083fc17eb@oracle.com> <20c6ccb0-4161-90e9-a6b7-f1aff4c05bef@redhat.com> Message-ID: <169dbd79-2ae1-8c9a-67b2-bc3a9f0f01a7@oracle.com> On 11/28/18 3:00 PM, Roman Kennke wrote: > Hi Per, > >> Hi Roman, >> >> On 11/26/18 10:39 PM, Roman Kennke wrote: >> [...] >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >> >> Just had a quick look at the SA part. I was thinking you'd have the same >> problem as ZGC here, with regards to parsing the heap and potentially >> reading garbage when you step on a Klass* which had been unloaded? > > Possible. I am myself not very familiar with SA. I guess it depends on > how SA does it: if it iterates objects via CH::object_iterate() (e.g. > same entry point as, e.g., heap-dumping code), then we should be fine. > We're kicking off a traversal rather than straight scan there. If > however SA somehow makes a raw scan itself, then we'd have the problem > you describe. The SA does a raw scan itself, which is the root of the problem. ObejctHeap.iterateLiveRegions() will locate the first object in a region by doing OopHandle handle = bottom.addOffsetToAsOopHandle(0); and to get the next object it does handle.addOffsetToAsOopHandle(obj.getObjectSize()); and you'll crash. So I'm afraid this will not work for Shenandoah either. cheers, Per > > Roman > > From rkennke at redhat.com Wed Nov 28 14:24:00 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 15:24:00 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <169dbd79-2ae1-8c9a-67b2-bc3a9f0f01a7@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <566619f4-87f5-a605-ca49-8b9083fc17eb@oracle.com> <20c6ccb0-4161-90e9-a6b7-f1aff4c05bef@redhat.com> <169dbd79-2ae1-8c9a-67b2-bc3a9f0f01a7@oracle.com> Message-ID: <8d5b0cd0-d791-3c77-7302-eea97012897e@redhat.com> >> Hi Per, >> >>> Hi Roman, >>> >>> On 11/26/18 10:39 PM, Roman Kennke wrote: >>> [...] >>>> ???*) shared-serviceability >>>> >>>> >>>> >>>> ????? - The usual code to support another GC >>> >>> Just had a quick look at the SA part. I was thinking you'd have the same >>> problem as ZGC here, with regards to parsing the heap and potentially >>> reading garbage when you step on a Klass* which had been unloaded? >> >> Possible. I am myself not very familiar with SA. I guess it depends on >> how SA does it: if it iterates objects via CH::object_iterate() (e.g. >> same entry point as, e.g., heap-dumping code), then we should be fine. >> We're kicking off a traversal rather than straight scan there. If >> however SA somehow makes a raw scan itself, then we'd have the problem >> you describe. > > The SA does a raw scan itself, which is the root of the problem. > ObejctHeap.iterateLiveRegions() will locate the first object in a region > by doing > > ? OopHandle handle = bottom.addOffsetToAsOopHandle(0); > > and to get the next object it does > > ? handle.addOffsetToAsOopHandle(obj.getObjectSize()); > > and you'll crash. So I'm afraid this will not work for Shenandoah either. Alright. I'll 'disable' it like you did with ZGC then. Thanks for pointing it out. I'm wondering: this would crash with G1 and +ClassUnloadingWithConcurrentMark too, then? Roman From vladimir.kozlov at oracle.com Wed Nov 28 18:03:57 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 28 Nov 2018 10:03:57 -0800 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <73bef199-d1f1-23e7-372b-5bb152431443@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> <5BF423D2.1070800@oracle.com> <5BF819B1.7040908@oracle.com> <43818dbd-5008-b3a0-7045-2d314e47f5fa@oracle.com> <5BF81BBC.2060606@oracle.com> <86d894d1-6a32-3a1a-01e9-1919f1efceae@oracle.com> <73bef199-d1f1-23e7-372b-5bb152431443@oracle.com> Message-ID: <1da906f1-f361-b159-2698-c53e8f9e044e@oracle.com> Looks good to me too. Thanks, Vladimir On 11/28/18 2:48 AM, Robbin Ehn wrote: >> Incremental: >> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01_02/ > > Still good! > >> >> Full: >> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.02/ >> >> Still need one more reviewer for this change. > > Also bump :) > > /Robbin > >> >> /Erik >> >> On 2018-11-23 16:24, Erik ?sterlund wrote: >>> Hi Robbin, >>> >>> Thanks for the review. >>> >>> /Erik >>> >>> On 2018-11-23 16:26, Robbin Ehn wrote: >>>> Looks good, thanks! >>>> >>>> /Robbin >>>> >>>> On 2018-11-23 16:16, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> Here is an updated version with some minor adjustments based on feedback from Robbin Ehn. >>>>> >>>>> Incremental: >>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00_01/ >>>>> >>>>> Full: >>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01/ >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-11-20 16:10, Erik ?sterlund wrote: >>>>>> Ping. >>>>>> >>>>>> /Erik >>>>>> >>>>>> On 2018-11-12 23:02, Erik ?sterlund wrote: >>>>>>> Hi, >>>>>>> >>>>>>> In my change 8209189, I moved a lot of code around relating to code cache unloading. Unfortunately, I failed to >>>>>>> see that when making nmethods unloaded, and their dependents are flushed... they are not immediately removed. >>>>>>> Instead, they are tagged to have stale entries, that are later cleaned during weak metadata cleaning of >>>>>>> InstanceKlass. As for method handles, they leak instead, with some hacks to reduce the leaking by expunging >>>>>>> entries while adding and removing. >>>>>>> >>>>>>> Obviously, with InstanceKlass cleanup and code cache unloading now >>>>>>> possibly running in parallel after 8209189, things can go wrong. These two phases used >>>>>>> to be separated by a "wait barrier" for the worker threads preventing that parallelism. >>>>>>> >>>>>>> The reason dependency contexts were not cleaned when they were found >>>>>>> during code cache cleaning, is because it was not thread safe when code >>>>>>> cache unloading was made parallel instead of serial. But now that we are >>>>>>> implementing concurrent class unloading, we are going to need concurrent >>>>>>> cleanup of dependency contexts anyway. Therefore I will propose a bug fix that fixes the problem in a way that >>>>>>> works for both serial, parallel and concurrent class unloading. >>>>>>> >>>>>>> The solution is to during code cache unloading claim cleaning of >>>>>>> encountered stale dependency contexts, and clean them straight away. >>>>>>> Entries are unlinked in a lock-free fashion, and placed on a purge list that is walked and deleted during >>>>>>> ClassLoaderDataGraph::purge(). This follows the thinking of first unlinking, then syncing all threads, and then >>>>>>> purging. >>>>>>> >>>>>>> New accessors for the head and next links, hide is_unloading entries and helps unlinking them and putting them >>>>>>> into the purge list. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213565 >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>> >>>>> >>> From erik.osterlund at oracle.com Wed Nov 28 18:14:22 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 28 Nov 2018 19:14:22 +0100 Subject: RFR: 8213565: Crash in DependencyContext::remove_dependent_nmethod In-Reply-To: <1da906f1-f361-b159-2698-c53e8f9e044e@oracle.com> References: <9afbc1a9-17dd-94ce-cb5e-fa8e24198ac6@oracle.com> <5BF423D2.1070800@oracle.com> <5BF819B1.7040908@oracle.com> <43818dbd-5008-b3a0-7045-2d314e47f5fa@oracle.com> <5BF81BBC.2060606@oracle.com> <86d894d1-6a32-3a1a-01e9-1919f1efceae@oracle.com> <73bef199-d1f1-23e7-372b-5bb152431443@oracle.com> <1da906f1-f361-b159-2698-c53e8f9e044e@oracle.com> Message-ID: Hi Vladimir, Thanks for the review! /Erik > On 28 Nov 2018, at 19:03, Vladimir Kozlov wrote: > > Looks good to me too. > > Thanks, > Vladimir > > On 11/28/18 2:48 AM, Robbin Ehn wrote: >>> Incremental: >>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01_02/ >> Still good! >>> >>> Full: >>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.02/ >>> >>> Still need one more reviewer for this change. >> Also bump :) >> /Robbin >>> >>> /Erik >>> >>>> On 2018-11-23 16:24, Erik ?sterlund wrote: >>>> Hi Robbin, >>>> >>>> Thanks for the review. >>>> >>>> /Erik >>>> >>>>> On 2018-11-23 16:26, Robbin Ehn wrote: >>>>> Looks good, thanks! >>>>> >>>>> /Robbin >>>>> >>>>>> On 2018-11-23 16:16, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> Here is an updated version with some minor adjustments based on feedback from Robbin Ehn. >>>>>> >>>>>> Incremental: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00_01/ >>>>>> >>>>>> Full: >>>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.01/ >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> On 2018-11-20 16:10, Erik ?sterlund wrote: >>>>>>> Ping. >>>>>>> >>>>>>> /Erik >>>>>>> >>>>>>>> On 2018-11-12 23:02, Erik ?sterlund wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> In my change 8209189, I moved a lot of code around relating to code cache unloading. Unfortunately, I failed to see that when making nmethods unloaded, and their dependents are flushed... they are not immediately removed. Instead, they are tagged to have stale entries, that are later cleaned during weak metadata cleaning of InstanceKlass. As for method handles, they leak instead, with some hacks to reduce the leaking by expunging entries while adding and removing. >>>>>>>> >>>>>>>> Obviously, with InstanceKlass cleanup and code cache unloading now >>>>>>>> possibly running in parallel after 8209189, things can go wrong. These two phases used >>>>>>>> to be separated by a "wait barrier" for the worker threads preventing that parallelism. >>>>>>>> >>>>>>>> The reason dependency contexts were not cleaned when they were found >>>>>>>> during code cache cleaning, is because it was not thread safe when code >>>>>>>> cache unloading was made parallel instead of serial. But now that we are >>>>>>>> implementing concurrent class unloading, we are going to need concurrent >>>>>>>> cleanup of dependency contexts anyway. Therefore I will propose a bug fix that fixes the problem in a way that works for both serial, parallel and concurrent class unloading. >>>>>>>> >>>>>>>> The solution is to during code cache unloading claim cleaning of >>>>>>>> encountered stale dependency contexts, and clean them straight away. >>>>>>>> Entries are unlinked in a lock-free fashion, and placed on a purge list that is walked and deleted during ClassLoaderDataGraph::purge(). This follows the thinking of first unlinking, then syncing all threads, and then purging. >>>>>>>> >>>>>>>> New accessors for the head and next links, hide is_unloading entries and helps unlinking them and putting them into the purge list. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8213565/webrev.00/ >>>>>>>> >>>>>>>> Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8213565 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>>> >>>>>> >>>> From vladimir.kozlov at oracle.com Wed Nov 28 18:12:42 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 28 Nov 2018 10:12:42 -0800 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> Message-ID: <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> Looks good. What testing you did? Unfortunately compiler/c2/cr634086 test is not part of tier1 testing and not executed by 'submit' repo testing. You need to run it. Thanks, Vladimir On 11/28/18 12:32 AM, Yang Zhang (Arm Technology China) wrote: > Hi Andrew, Vladimir > > I update the reviewers and remove the new assert in Matcher::vector_shift_count_ideal_reg(). Could you please help review again? > > http://cr.openjdk.java.net/~yzhang/8213134/webrev.02/ > > Regards > Yang > > -----Original Message----- > From: Vladimir Kozlov > Sent: Thursday, November 22, 2018 1:56 AM > To: Yang Zhang (Arm Technology China) ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 > > You are right. I think I was confused that you are not limiting MaxVectorSize but simple use min(16, MaxVectorSize). > > The assert is fine. > > Thanks, > Vladimir > > On 11/20/18 10:28 PM, Yang Zhang (Arm Technology China) wrote: >> >>> I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. >>> In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). >>> aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. >> >> Matcher::vector_width_in_bytes() limits vector size to min(16, MaxVectorSize). >> Matcher::vector_shift_count_ideal_reg() assert(MaxVectorSize >= size, >> "Length isn't supported"); When MaxVectorSize is 8, size is 8. >> When MaxVectorSize is greater than or equal 16, size is 16. >> So (MaxVectorSize >= size) is always true. >> >> Test cases of cr6340864 are passed with or without this new assert. >> >> Do I need to remove this new assert? >> >> Regards >> Yang >> >> >> On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: >>> Hi, >>> >>> When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? >>> >>> Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 >>> >>> This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. >>> Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. >>> compiler/c2/cr6340864/TestByteVect.java >>> compiler/c2/cr6340864/TestIntVect.java >>> compiler/c2/cr6340864/TestShortVect.java >>> compiler/codegen/TestCharVect2.java >>> >>> Regards >>> Yang >>> >>> >> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. >> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > From vladimir.kozlov at oracle.com Wed Nov 28 18:25:55 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 28 Nov 2018 10:25:55 -0800 Subject: RFR: 8214257: IC cache not clean after cleaning assertion failure In-Reply-To: References: Message-ID: <81c911f1-81a1-2187-1dc9-7990e4c4d0e7@oracle.com> Looks good to me. Thanks, Vladimir On 11/27/18 5:00 AM, Erik ?sterlund wrote: > Hi, > > Back in 8212681, an abstract CompiledICLocker was introduce to protect inline caches from concurrent patching. > > Unfortunately, not all patching of ICs started using the CompiledICLocker. There were still some callsites using the > Patching_lock. Therefore, the assert may trigger sometimes saying the IC cache is not clean right after cleaning, due to > other concurrent modifications under the Patching_lock. > > Also, in CompiledIC::set_to_clean(), the condition for safe_transition was altered incorrectly from is_at_safepoint() to > CompiledICLocker::is_safe(), causing cleaning to not used transition, so I reinstated that in this patch. > > This patch depends on 8214338 by assuming that the CompiledICLocker does not use safepoint checks, as it is now used > from leaf calls where safepointing is not allowed. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8214257 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8214257/webrev.00/ > > Thanks, > /Erik From rkennke at redhat.com Wed Nov 28 18:36:17 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 19:36:17 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: Hi Leonid, > I looked at the tests changes only. It seems that? a some tests might > start failing if JDK built without Shenandoah GC. Also some test are not > going to be selected as expected. > > Unfortunately logic of '@requires' and @run in jtreg tests doesn't > always work well for specific cases of different GC selection. The > '@requires' tags are combined with '&' and whole test is selected or > not. Test always execute ALL @run actions so it fails if option > -XX:+UseShenandoahGC is not supported (not valid). The only way to split > 'run' actions is to add more @test with same sources. They could be in > the same file. > > See detailed info about jtreg tags > here:http://openjdk.java.net/jtreg/tag-spec.html > Thanks for pointing this out. I fixed all of what you pointed out (I think) and some more: http://cr.openjdk.java.net/~rkennke/fix-shared-tests/webrev.02/ It will show up in round2 of this review. Some more comments inline: > Could you please run your tests with with JDK which built without > Shenandoah GC. It helps to verify that Shenandoah-specific tests/runs > are not selected when this GC is not supported. I did now, and they are good (with above changes). > ? Also it would be nice > to verify that there are no any valid tests which became filtered out > with your patch. The running of all test suite with available but not > selected Shenandoah GC and enabled Graal also might help to verify > @requires settings. (It doesn't help when Shenandoah GCis not supported > though.)? I'll see into running more combinations. So far I did hotspot_gc and a few others with and without Shenandoah. > I haven't looked at the tests in directory gc/shenandoah in details. But > all of them should be guarded with @requires. Placing them in separate > directory is not enough. See G1 tests as example: > > http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestEagerReclaimHumongousRegions.java > Right. We've done that now. > See more detailed comments about shared tests: > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/TEST.groups.sdiff.html > > 219: tier2_gc_shenandoah = \ > > Usually tierN doesn't include tests from tierN-1. TierN is executed > after TierN-1 completed so no need to re-run the same tests.? The > typical groups might looks like: > > tier1_gc_shenandoah = \ > ?gc/shenandoah/ \ > ?other-tests > > tier2_gc_shenandoah = \ > ?? gc/shenandoah/\ > ? -:tier1_gc_shenandoah > > tier3_gc_shenandoah = \ > ?? gc/shenandoah/ \? //all-other-tests > ? -:tier2_gc_shenandoah We fixed that. > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/CriticalNativeArgs.java.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/stress/CriticalNativeStress.java.html > > So your test will be skipped if any of -XX:+UseEpsilonGC or > -XX:+UseShenandoahGC is set. Also test might run only all run actions or > none of them. It would be better to split this test into 2 tests. So > epsilon tests might be executed if Shenandoah is absent. > > I think that (vm.bits == "64") is redundant in tag (you set x64 or arm64). > > Please use 4-space indentation? in java code. All fixed. > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestFullGCCount.java.sdiff.html > > Even original requires seems confusing to me (but it works for CMS/G1 pair) > > 28? * @requires !(vm.gc.ConcMarkSweep & > vm.opt.ExplicitGCInvokesConcurrent == true)? > > So currently test is executed if GC is CMS or default GC and > ExplicitGCInvokesConcurrent is not set to true. > > With your additional requirements 'vm.gc == "Shenandoah"' test is not > selected if ANY GC is set. Test doesn't set any GC itself so only > default GC might be tested.? See [1]. > I split them and fixed the @requires to run with Shenandoah but only with -ExplicitGCInvokesConcurrent. > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.sdiff.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestSystemGC.java.sdiff.html > > Tests will always just fail if -XX:+UseShenandoahGC? is not supported. > (invalid option) You might want to split test is it done for CMS GC. Done. > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestAlignmentToUseLargePages.java.sdiff.html > > I think? > 56? * @requires vm.gc=="null" & !vm.graal.enabled > should be something like @requires vm.gc.Shenandoah & !vm.graal.enabled Yes. Fixed them. > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestUseCompressedOopsErgo.java.sdiff.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/class_unloading/TestClassUnloadingDisabled.java.sdiff.html > > The same for? 62? * @requires vm.gc=="null" & !vm.graal.enabled > and > 72? * @requires vm.gc=="null" & !vm.graal.enabled > Fixed them too. > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/ergonomics/TestDynamicNumberOfGCThreads.java.sdiff.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/logging/TestGCId.java.sdiff.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/CompressedOops/UseCompressedOops.java.sdiff.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/MemberName/MemberNameLeak.java.sdiff.html > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html > Also these tests are going to be run with all GC and fails if Shenandoah > is not supported. Right. I fixed those and a few others I have found. Those which drive Shenandoah at runtime now have a runtime check (GC.Shenandoah.isSupported() which uses WB). Others have split test sections. I'll upload round 2 of review changesets, which contains the fixes in a bit. Thanks a *LOT* for detailed review. Roman > Leonid > > [1] http://openjdk.java.net/jtreg/tag-spec.html > > On 11/26/18 1:39 PM, Roman Kennke wrote: >> >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the mailing >> lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is >> to eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> ? https://openjdk.java.net/jeps/189 >> Bug entry: >> >> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> which is what I intend to commit (and which should be equivalent to >> the 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> ?*) shenandoah-gc >> >> ??? - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> ?*) shared-gc >> >> ??? - This is mostly boilerplate that is common to any GC >> ??? - referenceProcessor.cpp has a little change to make one assert >> not fail (next to CMS and G1) >> ??? - taskqueue.hpp has some small adjustments to enable subclassing >> >> ?*) shared-serviceability >> >> ??? - The usual code to support another GC >> >> ?*) shared-runtime >> >> ??? - A number of friends declarations to allow Shenandoah iterators >> to hook up with, >> ????? e.g. ClassLoaderData, CodeCache, etc >> ??? - Warning and disabling JFR LeakProfiler >> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> ??? - Locks initialization in mutexLocker.cpp as usual >> ??? - VM operations defines for Shenandoah's VM ops >> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> ??? - The usual macros in macro.hpp >> >> ?*) shared-build >> >> ??? - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> ??? - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> ????? useful to get the whole marking loop inlined (observed >> significant regression if we >> ????? don't) >> >> ?*) shared-tests >> >> ??? - Test infrastructure to support Shenandoah >> ??? - Shenandoah test groups >> ??? - Exclude Shenandoah in various tests that can be run with selected GC >> ??? - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> ?*) shenandoah-tests >> >> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >> ??? - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case >> you want to take >> a peek at it. >> >> We have regular builds on: >> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> ? - {Windows} x {x86_64}, >> ? - {MacOS X} x {x86_64} >> >> This also routinely passes: >> ? - the new Shenandoah tests >> ? - jcstress with/without aggressive Shenandoah verification >> ? - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she >> deserves the credit for being the original inventor of Shenandoah, >> Aleksey Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface >> and related changes, and of course the many early adopters for >> reporting bugs and success stories and feature requests: we wouldn't >> be here without any of you! >> >> Best regards, >> Roman >> From alexey.menkov at oracle.com Wed Nov 28 20:36:48 2018 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 28 Nov 2018 12:36:48 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> Message-ID: Hi guys, I was able to reproduce the issue only with Graal + -Xcomp and only with debug build (I tried Graal + -Xint, Graal without Xcomp/Xint, just -Xcomp (i.e. C1 is used)). --alex On 11/28/2018 01:27, serguei.spitsyn at oracle.com wrote: > On 11/28/18 01:07, dean.long at oracle.com wrote: >> OK, reconsidering solution 1, I have a couple more questions: >> >> The bug report says the problem is reproducible with -Xcomp, but I >> don't see the policy()->event() path taken with -Xcomp. Instead, >> -Xcomp uses CompilationPolicy::compile_if_required, so I don't see how >> Solution 1 fixes the problem with -Xcomp. >> >> > CompilerRuntime always calls policy()->event with CompLevel == >> CompLevel_aot. >> >> As far as I can tell, CompLevel_aot is only used when there is active >> AOT code.? Is this an AOT-specific problem? > > Tracing showed that the CompLevel_aot is always used for JVMCI which is > pretty confusing. > Most likely, it is because now the AOT is enabled by default (is it > correct?). > I'm not sure it actually has anything to do with the AOT mode. > > >> ? I would expect there to be a problem with c1 as well. > > Not sure about the c1. > I hope, Alex could comment on this tomorrow. > > > Thanks, > Serguei > >> dl >> >> On 11/27/18 3:38 PM, serguei.spitsyn at oracle.com wrote: >>> Good summary with a list of solutions below! >>> >>> On 11/27/18 2:50 PM, dean.long at oracle.com wrote: >>>> Let me list the proposed solutions: >>>> >>>> 1. short-circuit compiles in CompilationPolicy::event in >>>> is_interp_only_mode >>>> 2. detect is_interp_only_mode in resolve stubs and force a c2i call >>>> 3. don't offer a JVMTI suspend point in resolve stubs (or >>>> JavaCallWrapper) >>>> >>>> Solution 1 penalizes other threads, and don't prevent other threads >>>> from compiling the method, so while it may help a little, it's >>>> incomplete. >>> >>> I doubt, the Solution 1 penalizes other threads. >>> First, it is an important pretty frequent/common case that allows to >>> save on compilations. >>> Second, there is no point to compile methods on a thread executed in >>> interp_only_mode. >>> Also, there is no big advantage to compile methods even for other >>> threads >>> when an active debugging (e.g. single stepping) is in progress. >>> Third, this would make it consistent with what the the interpreter is >>> doing. >>> >>> So, I think, the Solution 1 is needed independently of other >>> solutions we take. >>> My suggestion is to apply it in the JDK-8195639 bug fix. >>> >>>> Solutions 1 and 2 allow the thread to resume in a different method >>>> (the callee method), causing other problems. >>> >>> Agreed. >>> >>>> With Solution 3, the frame we suspend is the same as the frame we >>>> end up in after the resume, so I believe it solves all the problems. >>>> IMHO this is the correct solution to pursue. >>> >>> I agree in general. >>> This solution has some risks involved as it impacts a lot of code >>> including platform-independent. >>> It is also going to fix the other bugs: JDK-8195635, JDK-8214093, >>> JDK-8207013 (not sure about all of them yet). >>> >>> My suggestion is to pursue it as the JDK-8195635 fix. >>> Could you, please, confirm if works for you? >>> >>> Also, we have no consensus yet on the Solution 1. >>> >>> Thanks, >>> Serguei >>> >>>> dl >>>> >>>> On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: >>>>> Hi Dean, >>>>> >>>>> You also asked the questions: >>>>> >>>>> > Doesn't is_interp_only_mode() only apply to the current thread? >>>>> >>>>> Yes it does. >>>>> >>>>> >>>>> > I don't think it's safe to assume no compiles will happen in >>>>> other threads, >>>>> > or that a method call target is not already compiled, because as far >>>>> > as I can tell, JVMTI only deoptimizes the active frames. >>>>> >>>>> I agree with it. >>>>> However, compilations on the thread with the interp_only_mode >>>>> enabled is the most important case. >>>>> Disabling compilations such thread improves testing stability. >>>>> >>>>> > The bug report describes a case where the caller has been >>>>> deoptimized >>>>> > while resolving a call.? I don't see how this fix prevents the >>>>> target >>>>> > from being a previously compiled method. >>>>> > But we do need to make sure not to call into compiled code, so I >>>>> think >>>>> > the fix needs to be in code like >>>>> SharedRuntime::resolve_static_call_C(), >>>>> > where it returns get_c2i_entry() if is_interp_only_mode() is true. >>>>> >>>>> Thank you for making this point. >>>>> >>>>> It would be nice to attack this as well. >>>>> We can try to investigate this approach further. >>>>> One problem is that there are more cases like resolve_static_call_C, >>>>> for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. >>>>> We may need to fix the same in other places, like >>>>> JavaCallWrapper::JavaCallWrapper. >>>>> >>>>> We can start from fixing it in the resolve_static_call_C. >>>>> If it is successful then continue this effort for other cases as well. >>>>> >>>>> What do you think? >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>> >>>>> On 11/27/18 1:01 PM, Alex Menkov wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Thank you for the feedback and for the comments in Jira. >>>>>> >>>>>> >> How does this solve the problem of >>>>>> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>> >>>>>> It doesn't. It fixes another issue. >>>>>> The test suspend test thread and ensures that top frame is a >>>>>> "fibonachi" method. Then it turns SingleStep on, does PopFrame and >>>>>> resumes the thread. >>>>>> The problem is the thread continues execution of compiled code >>>>>> (ignoring SingleStep) and we get SindleStep event only when >>>>>> adjustCompilationLevel method is called (it's interpreted code). >>>>>> >>>>>> There are several other bugs which are (most likely) caused by >>>>>> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >>>>>> >>>>>> --alex >>>>>> >>>>>> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> Thank you a lot for looking at this! >>>>>>> Just a couple of points from me (it is up to Alex to provide a >>>>>>> full answer). >>>>>>> >>>>>>> >>>>>>> I think, Alex in this RFR missed to tell that we knew about this >>>>>>> issue that an incorrect frame will be popped. >>>>>>> But after some discussion we decided to file a separate issue on >>>>>>> this. >>>>>>> Alex wanted to create a good stand-alone test case demonstrating >>>>>>> this problem before filing it. >>>>>>> >>>>>>> Now, as I can see, the JDK-8195635 >>>>>>> looks very >>>>>>> close to a bug that we wanted to file. >>>>>>> The only difference is that our scenario includes the >>>>>>> SharedRuntime::resolve_static_call_C instead of the >>>>>>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>>>>>> invocation. >>>>>>> If I understand corrctly (Alex will fix me if needed), the jtreg >>>>>>> test we used to chase this issue did not catch a problem with the >>>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel. >>>>>>> >>>>>>> >>>>>>> The suggested fix was to fix the mismatch in the >>>>>>> TieredThresholdPolicy::even with the comment in the interpreter >>>>>>> code: >>>>>>> nmethod* >>>>>>> InterpreterRuntime::frequency_counter_overflow(JavaThread* >>>>>>> thread, address branch_bcp) { >>>>>>> ?????. . . >>>>>>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>>>>>> ?????// Normally we never get an nm if is_interp_only_mode() is >>>>>>> true, because >>>>>>> ?????// policy()->event has a check for this and won't compile >>>>>>> the method when >>>>>>> ?????// true. However, it's possible for is_interp_only_mode() to >>>>>>> become true >>>>>>> ?????// during the compilation. We don't want to return the nm in >>>>>>> that case >>>>>>> ?????// because we want to continue to execute interpreted. >>>>>>> ?????nm = NULL; >>>>>>> ???} >>>>>>> >>>>>>> ?> So I think the fix needs to be in code like >>>>>>> SharedRuntime::resolve_static_call_C(), >>>>>>> ?> where it returns get_c2i_entry() if is_interp_only_mode() is >>>>>>> true. >>>>>>> >>>>>>> I'm not sure, the adding this condition and returning the >>>>>>> get_c2i_entry() is always correct. >>>>>>> We need some help from the Compiler team to make it right. >>>>>>> >>>>>>> BTW, the interp_only_mode has been enabled only when some >>>>>>> interp_only events are enabled. >>>>>>> It is not set by either PopFrame or ForceEarlyReturn. >>>>>>> But the popframe009 test enables single stepping, so we wanted to >>>>>>> make this part right. >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>>>>>> How does this solve the problem of >>>>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>>>> >>>>>>>> I don't think this fix is the right approach.? Doesn't >>>>>>>> is_interp_only_mode() only apply to the current thread? I don't >>>>>>>> think it's safe to assume no compiles will happen in other >>>>>>>> threads, or that a method call target is not already compiled, >>>>>>>> because as far as I can tell, JVMTI only deoptimizes the active >>>>>>>> frames. The bug report describes a case where the caller has >>>>>>>> been deoptimized while resolving a call.? I don't see how this >>>>>>>> fix prevents the target from being a previously compiled >>>>>>>> method.? But we do need to make sure not to call into compiled >>>>>>>> code, so I think the fix needs to be in code like >>>>>>>> SharedRuntime::resolve_static_call_C(), where it returns >>>>>>>> get_c2i_entry() if is_interp_only_mode() is true. However, there >>>>>>>> is still another problem.? By allowing JVMTI to suspend the >>>>>>>> thread during call setup, but reporting the frame as still in >>>>>>>> the caller instead of the callee, we confuse JVMTI into thinking >>>>>>>> that execution will resume in the caller instead of the callee. >>>>>>>> We may want to restrict where we offer JVMTI suspend points, and >>>>>>>> not offer a JVMTI suspend point in >>>>>>>> SharedRuntime::resolve_static_call_C and friends at all. >>>>>>>> >>>>>>>> dl >>>>>>>> >>>>>>>> >>>>>>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please review the fix for >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>>>>>> webrev: >>>>>>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>>>>>> >>>>>>>>> Description: >>>>>>>>> The test suspends a thread, turns on single stepping and then >>>>>>>>> calls PopFrame. SingleStep event is expected as soon as the >>>>>>>>> thread is resumed and PopFrame is processed (so we have call >>>>>>>>> stack with the depth 1 less than it was before PopFrame). >>>>>>>>> Instead SingleStep event is received with much deeper call >>>>>>>>> stack (and PopFrame processes wrong frame). >>>>>>>>> Research shown that this is caused by missed deoptimization of >>>>>>>>> the current frame. >>>>>>>>> As far as I understand CompilationPolicy::event should handle >>>>>>>>> the case when the thread has is_interp_only_mode() set, but >>>>>>>>> TieredThresholdPolicy::event checks this only then CompLevel is >>>>>>>>> CompLevel_none. >>>>>>>>> CompilerRuntime always calls policy()->event with CompLevel == >>>>>>>>> CompLevel_aot. >>>>>>>>> >>>>>>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>>>>>> runtime and compiler teams as I'm not sure I completely >>>>>>>>> understand all the details of the "PopFrame dance". >>>>>>>>> >>>>>>>>> --alex >>>>>>>> >>>>>>> >>>>> >>>> >>> >> > From leonid.mesnik at oracle.com Wed Nov 28 21:23:58 2018 From: leonid.mesnik at oracle.com (leonid.mesnik at oracle.com) Date: Wed, 28 Nov 2018 13:23:58 -0800 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: <45c5d4dd-4edd-b67f-28aa-38ca48de8cfd@oracle.com> Hi Thank you for your fixes and additional testing. I'll check new version? in the round-2. Leonid On 11/28/18 10:36 AM, Roman Kennke wrote: > Hi Leonid, > >> I looked at the tests changes only. It seems that? a some tests might >> start failing if JDK built without Shenandoah GC. Also some test are not >> going to be selected as expected. >> >> Unfortunately logic of '@requires' and @run in jtreg tests doesn't >> always work well for specific cases of different GC selection. The >> '@requires' tags are combined with '&' and whole test is selected or >> not. Test always execute ALL @run actions so it fails if option >> -XX:+UseShenandoahGC is not supported (not valid). The only way to split >> 'run' actions is to add more @test with same sources. They could be in >> the same file. >> >> See detailed info about jtreg tags >> here:http://openjdk.java.net/jtreg/tag-spec.html >> > Thanks for pointing this out. > > I fixed all of what you pointed out (I think) and some more: > http://cr.openjdk.java.net/~rkennke/fix-shared-tests/webrev.02/ > > It will show up in round2 of this review. > Some more comments inline: > >> Could you please run your tests with with JDK which built without >> Shenandoah GC. It helps to verify that Shenandoah-specific tests/runs >> are not selected when this GC is not supported. > I did now, and they are good (with above changes). > >> ? Also it would be nice >> to verify that there are no any valid tests which became filtered out >> with your patch. The running of all test suite with available but not >> selected Shenandoah GC and enabled Graal also might help to verify >> @requires settings. (It doesn't help when Shenandoah GCis not supported >> though.) > > I'll see into running more combinations. So far I did hotspot_gc and a > few others with and without Shenandoah. > >> I haven't looked at the tests in directory gc/shenandoah in details. But >> all of them should be guarded with @requires. Placing them in separate >> directory is not enough. See G1 tests as example: >> >> http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestEagerReclaimHumongousRegions.java >> > Right. We've done that now. > >> See more detailed comments about shared tests: >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/TEST.groups.sdiff.html >> >> 219: tier2_gc_shenandoah = \ >> >> Usually tierN doesn't include tests from tierN-1. TierN is executed >> after TierN-1 completed so no need to re-run the same tests.? The >> typical groups might looks like: >> >> tier1_gc_shenandoah = \ >> ?gc/shenandoah/ \ >> ?other-tests >> >> tier2_gc_shenandoah = \ >> ?? gc/shenandoah/\ >> ? -:tier1_gc_shenandoah >> >> tier3_gc_shenandoah = \ >> ?? gc/shenandoah/ \? //all-other-tests >> ? -:tier2_gc_shenandoah > We fixed that. > >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/CriticalNativeArgs.java.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/stress/CriticalNativeStress.java.html >> >> So your test will be skipped if any of -XX:+UseEpsilonGC or >> -XX:+UseShenandoahGC is set. Also test might run only all run actions or >> none of them. It would be better to split this test into 2 tests. So >> epsilon tests might be executed if Shenandoah is absent. >> >> I think that (vm.bits == "64") is redundant in tag (you set x64 or arm64). >> >> Please use 4-space indentation? in java code. > All fixed. > >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestFullGCCount.java.sdiff.html >> >> Even original requires seems confusing to me (but it works for CMS/G1 pair) >> >> 28? * @requires !(vm.gc.ConcMarkSweep & >> vm.opt.ExplicitGCInvokesConcurrent == true) >> >> So currently test is executed if GC is CMS or default GC and >> ExplicitGCInvokesConcurrent is not set to true. >> >> With your additional requirements 'vm.gc == "Shenandoah"' test is not >> selected if ANY GC is set. Test doesn't set any GC itself so only >> default GC might be tested.? See [1]. >> > I split them and fixed the @requires to run with Shenandoah but only > with -ExplicitGCInvokesConcurrent. > >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.sdiff.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestSystemGC.java.sdiff.html >> >> Tests will always just fail if -XX:+UseShenandoahGC? is not supported. >> (invalid option) You might want to split test is it done for CMS GC. > Done. > >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestAlignmentToUseLargePages.java.sdiff.html >> >> I think >> 56? * @requires vm.gc=="null" & !vm.graal.enabled >> should be something like @requires vm.gc.Shenandoah & !vm.graal.enabled > Yes. Fixed them. > >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestUseCompressedOopsErgo.java.sdiff.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/class_unloading/TestClassUnloadingDisabled.java.sdiff.html >> >> The same for? 62? * @requires vm.gc=="null" & !vm.graal.enabled >> and >> 72? * @requires vm.gc=="null" & !vm.graal.enabled >> > Fixed them too. > >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/ergonomics/TestDynamicNumberOfGCThreads.java.sdiff.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/logging/TestGCId.java.sdiff.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/CompressedOops/UseCompressedOops.java.sdiff.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/MemberName/MemberNameLeak.java.sdiff.html >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html >> Also these tests are going to be run with all GC and fails if Shenandoah >> is not supported. > Right. I fixed those and a few others I have found. Those which drive > Shenandoah at runtime now have a runtime check > (GC.Shenandoah.isSupported() which uses WB). Others have split test > sections. > > I'll upload round 2 of review changesets, which contains the fixes in a bit. > > Thanks a *LOT* for detailed review. > > Roman > > >> Leonid >> >> [1] http://openjdk.java.net/jtreg/tag-spec.html >> >> On 11/26/18 1:39 PM, Roman Kennke wrote: >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the mailing >>> lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is >>> to eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> which is what I intend to commit (and which should be equivalent to >>> the 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ?*) shenandoah-gc >>> >>> ??? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ?*) shared-gc >>> >>> ??? - This is mostly boilerplate that is common to any GC >>> ??? - referenceProcessor.cpp has a little change to make one assert >>> not fail (next to CMS and G1) >>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ?*) shared-serviceability >>> >>> ??? - The usual code to support another GC >>> >>> ?*) shared-runtime >>> >>> ??? - A number of friends declarations to allow Shenandoah iterators >>> to hook up with, >>> ????? e.g. ClassLoaderData, CodeCache, etc >>> ??? - Warning and disabling JFR LeakProfiler >>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ??? - Locks initialization in mutexLocker.cpp as usual >>> ??? - VM operations defines for Shenandoah's VM ops >>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ??? - The usual macros in macro.hpp >>> >>> ?*) shared-build >>> >>> ??? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ??? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ????? useful to get the whole marking loop inlined (observed >>> significant regression if we >>> ????? don't) >>> >>> ?*) shared-tests >>> >>> ??? - Test infrastructure to support Shenandoah >>> ??? - Shenandoah test groups >>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ?*) shenandoah-tests >>> >>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>> ??? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case >>> you want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ? - {Windows} x {x86_64}, >>> ? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ? - the new Shenandoah tests >>> ? - jcstress with/without aggressive Shenandoah verification >>> ? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she >>> deserves the credit for being the original inventor of Shenandoah, >>> Aleksey Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface >>> and related changes, and of course the many early adopters for >>> reporting bugs and success stories and feature requests: we wouldn't >>> be here without any of you! >>> >>> Best regards, >>> Roman >>> From dean.long at oracle.com Wed Nov 28 21:43:31 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Wed, 28 Nov 2018 13:43:31 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> Message-ID: I guess even with -Xcomp, there are still compiles triggered by profiled code, otherwise I don't see why changing policy()->event() helps. Another problem I see with this change is that it will make the real problem harder to reproduce. dl On 11/28/18 12:36 PM, Alex Menkov wrote: > Hi guys, > > I was able to reproduce the issue only with Graal + -Xcomp and only > with debug build (I tried Graal + -Xint, Graal without Xcomp/Xint, > just -Xcomp (i.e. C1 is used)). > > --alex > > On 11/28/2018 01:27, serguei.spitsyn at oracle.com wrote: >> On 11/28/18 01:07, dean.long at oracle.com wrote: >>> OK, reconsidering solution 1, I have a couple more questions: >>> >>> The bug report says the problem is reproducible with -Xcomp, but I >>> don't see the policy()->event() path taken with -Xcomp. Instead, >>> -Xcomp uses CompilationPolicy::compile_if_required, so I don't see >>> how Solution 1 fixes the problem with -Xcomp. >>> >>> > CompilerRuntime always calls policy()->event with CompLevel == >>> CompLevel_aot. >>> >>> As far as I can tell, CompLevel_aot is only used when there is >>> active AOT code.? Is this an AOT-specific problem? >> >> Tracing showed that the CompLevel_aot is always used for JVMCI which >> is pretty confusing. >> Most likely, it is because now the AOT is enabled by default (is it >> correct?). >> I'm not sure it actually has anything to do with the AOT mode. >> >> >>> ? I would expect there to be a problem with c1 as well. >> >> Not sure about the c1. >> I hope, Alex could comment on this tomorrow. >> >> >> Thanks, >> Serguei >> >>> dl >>> >>> On 11/27/18 3:38 PM, serguei.spitsyn at oracle.com wrote: >>>> Good summary with a list of solutions below! >>>> >>>> On 11/27/18 2:50 PM, dean.long at oracle.com wrote: >>>>> Let me list the proposed solutions: >>>>> >>>>> 1. short-circuit compiles in CompilationPolicy::event in >>>>> is_interp_only_mode >>>>> 2. detect is_interp_only_mode in resolve stubs and force a c2i call >>>>> 3. don't offer a JVMTI suspend point in resolve stubs (or >>>>> JavaCallWrapper) >>>>> >>>>> Solution 1 penalizes other threads, and don't prevent other >>>>> threads from compiling the method, so while it may help a little, >>>>> it's incomplete. >>>> >>>> I doubt, the Solution 1 penalizes other threads. >>>> First, it is an important pretty frequent/common case that allows >>>> to save on compilations. >>>> Second, there is no point to compile methods on a thread executed >>>> in interp_only_mode. >>>> Also, there is no big advantage to compile methods even for other >>>> threads >>>> when an active debugging (e.g. single stepping) is in progress. >>>> Third, this would make it consistent with what the the interpreter >>>> is doing. >>>> >>>> So, I think, the Solution 1 is needed independently of other >>>> solutions we take. >>>> My suggestion is to apply it in the JDK-8195639 bug fix. >>>> >>>>> Solutions 1 and 2 allow the thread to resume in a different method >>>>> (the callee method), causing other problems. >>>> >>>> Agreed. >>>> >>>>> With Solution 3, the frame we suspend is the same as the frame we >>>>> end up in after the resume, so I believe it solves all the problems. >>>>> IMHO this is the correct solution to pursue. >>>> >>>> I agree in general. >>>> This solution has some risks involved as it impacts a lot of code >>>> including platform-independent. >>>> It is also going to fix the other bugs: JDK-8195635, JDK-8214093, >>>> JDK-8207013 (not sure about all of them yet). >>>> >>>> My suggestion is to pursue it as the JDK-8195635 fix. >>>> Could you, please, confirm if works for you? >>>> >>>> Also, we have no consensus yet on the Solution 1. >>>> >>>> Thanks, >>>> Serguei >>>> >>>>> dl >>>>> >>>>> On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: >>>>>> Hi Dean, >>>>>> >>>>>> You also asked the questions: >>>>>> >>>>>> > Doesn't is_interp_only_mode() only apply to the current thread? >>>>>> >>>>>> Yes it does. >>>>>> >>>>>> >>>>>> > I don't think it's safe to assume no compiles will happen in >>>>>> other threads, >>>>>> > or that a method call target is not already compiled, because >>>>>> as far >>>>>> > as I can tell, JVMTI only deoptimizes the active frames. >>>>>> >>>>>> I agree with it. >>>>>> However, compilations on the thread with the interp_only_mode >>>>>> enabled is the most important case. >>>>>> Disabling compilations such thread improves testing stability. >>>>>> >>>>>> > The bug report describes a case where the caller has been >>>>>> deoptimized >>>>>> > while resolving a call.? I don't see how this fix prevents the >>>>>> target >>>>>> > from being a previously compiled method. >>>>>> > But we do need to make sure not to call into compiled code, so >>>>>> I think >>>>>> > the fix needs to be in code like >>>>>> SharedRuntime::resolve_static_call_C(), >>>>>> > where it returns get_c2i_entry() if is_interp_only_mode() is true. >>>>>> >>>>>> Thank you for making this point. >>>>>> >>>>>> It would be nice to attack this as well. >>>>>> We can try to investigate this approach further. >>>>>> One problem is that there are more cases like resolve_static_call_C, >>>>>> for instance, resolve_virtual_call_C and resolve_opt_virtual_call_C. >>>>>> We may need to fix the same in other places, like >>>>>> JavaCallWrapper::JavaCallWrapper. >>>>>> >>>>>> We can start from fixing it in the resolve_static_call_C. >>>>>> If it is successful then continue this effort for other cases as >>>>>> well. >>>>>> >>>>>> What do you think? >>>>>> >>>>>> Thanks, >>>>>> Serguei >>>>>> >>>>>> >>>>>> On 11/27/18 1:01 PM, Alex Menkov wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> Thank you for the feedback and for the comments in Jira. >>>>>>> >>>>>>> >> How does this solve the problem of >>>>>>> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>>> >>>>>>> It doesn't. It fixes another issue. >>>>>>> The test suspend test thread and ensures that top frame is a >>>>>>> "fibonachi" method. Then it turns SingleStep on, does PopFrame >>>>>>> and resumes the thread. >>>>>>> The problem is the thread continues execution of compiled code >>>>>>> (ignoring SingleStep) and we get SindleStep event only when >>>>>>> adjustCompilationLevel method is called (it's interpreted code). >>>>>>> >>>>>>> There are several other bugs which are (most likely) caused by >>>>>>> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >>>>>>> >>>>>>> --alex >>>>>>> >>>>>>> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>>>>>>> Hi Dean, >>>>>>>> >>>>>>>> Thank you a lot for looking at this! >>>>>>>> Just a couple of points from me (it is up to Alex to provide a >>>>>>>> full answer). >>>>>>>> >>>>>>>> >>>>>>>> I think, Alex in this RFR missed to tell that we knew about >>>>>>>> this issue that an incorrect frame will be popped. >>>>>>>> But after some discussion we decided to file a separate issue >>>>>>>> on this. >>>>>>>> Alex wanted to create a good stand-alone test case >>>>>>>> demonstrating this problem before filing it. >>>>>>>> >>>>>>>> Now, as I can see, the JDK-8195635 >>>>>>>> looks very >>>>>>>> close to a bug that we wanted to file. >>>>>>>> The only difference is that our scenario includes the >>>>>>>> SharedRuntime::resolve_static_call_C instead of the >>>>>>>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>>>>>>> invocation. >>>>>>>> If I understand corrctly (Alex will fix me if needed), the >>>>>>>> jtreg test we used to chase this issue did not catch a problem >>>>>>>> with the HotSpotJVMCIRuntime.adjustCompilationLevel. >>>>>>>> >>>>>>>> >>>>>>>> The suggested fix was to fix the mismatch in the >>>>>>>> TieredThresholdPolicy::even with the comment in the interpreter >>>>>>>> code: >>>>>>>> nmethod* >>>>>>>> InterpreterRuntime::frequency_counter_overflow(JavaThread* >>>>>>>> thread, address branch_bcp) { >>>>>>>> ?????. . . >>>>>>>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>>>>>>> ?????// Normally we never get an nm if is_interp_only_mode() is >>>>>>>> true, because >>>>>>>> ?????// policy()->event has a check for this and won't compile >>>>>>>> the method when >>>>>>>> ?????// true. However, it's possible for is_interp_only_mode() >>>>>>>> to become true >>>>>>>> ?????// during the compilation. We don't want to return the nm >>>>>>>> in that case >>>>>>>> ?????// because we want to continue to execute interpreted. >>>>>>>> ?????nm = NULL; >>>>>>>> ???} >>>>>>>> >>>>>>>> ?> So I think the fix needs to be in code like >>>>>>>> SharedRuntime::resolve_static_call_C(), >>>>>>>> ?> where it returns get_c2i_entry() if is_interp_only_mode() is >>>>>>>> true. >>>>>>>> >>>>>>>> I'm not sure, the adding this condition and returning the >>>>>>>> get_c2i_entry() is always correct. >>>>>>>> We need some help from the Compiler team to make it right. >>>>>>>> >>>>>>>> BTW, the interp_only_mode has been enabled only when some >>>>>>>> interp_only events are enabled. >>>>>>>> It is not set by either PopFrame or ForceEarlyReturn. >>>>>>>> But the popframe009 test enables single stepping, so we wanted >>>>>>>> to make this part right. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Serguei >>>>>>>> >>>>>>>> >>>>>>>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>>>>>>> How does this solve the problem of >>>>>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>>>>> >>>>>>>>> I don't think this fix is the right approach. Doesn't >>>>>>>>> is_interp_only_mode() only apply to the current thread? I >>>>>>>>> don't think it's safe to assume no compiles will happen in >>>>>>>>> other threads, or that a method call target is not already >>>>>>>>> compiled, because as far as I can tell, JVMTI only deoptimizes >>>>>>>>> the active frames. The bug report describes a case where the >>>>>>>>> caller has been deoptimized while resolving a call.? I don't >>>>>>>>> see how this fix prevents the target from being a previously >>>>>>>>> compiled method.? But we do need to make sure not to call into >>>>>>>>> compiled code, so I think the fix needs to be in code like >>>>>>>>> SharedRuntime::resolve_static_call_C(), where it returns >>>>>>>>> get_c2i_entry() if is_interp_only_mode() is true. However, >>>>>>>>> there is still another problem. By allowing JVMTI to suspend >>>>>>>>> the thread during call setup, but reporting the frame as still >>>>>>>>> in the caller instead of the callee, we confuse JVMTI into >>>>>>>>> thinking that execution will resume in the caller instead of >>>>>>>>> the callee. We may want to restrict where we offer JVMTI >>>>>>>>> suspend points, and not offer a JVMTI suspend point in >>>>>>>>> SharedRuntime::resolve_static_call_C and friends at all. >>>>>>>>> >>>>>>>>> dl >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Please review the fix for >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>>>>>>> webrev: >>>>>>>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>>>>>>> >>>>>>>>>> Description: >>>>>>>>>> The test suspends a thread, turns on single stepping and then >>>>>>>>>> calls PopFrame. SingleStep event is expected as soon as the >>>>>>>>>> thread is resumed and PopFrame is processed (so we have call >>>>>>>>>> stack with the depth 1 less than it was before PopFrame). >>>>>>>>>> Instead SingleStep event is received with much deeper call >>>>>>>>>> stack (and PopFrame processes wrong frame). >>>>>>>>>> Research shown that this is caused by missed deoptimization >>>>>>>>>> of the current frame. >>>>>>>>>> As far as I understand CompilationPolicy::event should handle >>>>>>>>>> the case when the thread has is_interp_only_mode() set, but >>>>>>>>>> TieredThresholdPolicy::event checks this only then CompLevel >>>>>>>>>> is CompLevel_none. >>>>>>>>>> CompilerRuntime always calls policy()->event with CompLevel >>>>>>>>>> == CompLevel_aot. >>>>>>>>>> >>>>>>>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>>>>>>> runtime and compiler teams as I'm not sure I completely >>>>>>>>>> understand all the details of the "PopFrame dance". >>>>>>>>>> >>>>>>>>>> --alex >>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >> From serguei.spitsyn at oracle.com Wed Nov 28 21:52:06 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 28 Nov 2018 13:52:06 -0800 Subject: RFR JDK-8195639: [Graal] nsk/jvmti/PopFrame/popframe009 fail with Graal in -Xcomp In-Reply-To: References: <15b2057a-6e8b-4f67-f03e-c45720ea0fde@oracle.com> <5897ce86-dda6-dd44-ee53-359627c18353@oracle.com> <38bd82c5-f92b-03ea-eb1b-1bcfd58b68f3@oracle.com> <58ccd968-9fa0-09f1-e908-b691068fd1e8@oracle.com> <7b2c4b0b-d958-e225-99e2-81eb037eeb7f@oracle.com> <24d0b704-e58a-af5d-f78c-f91dd283b591@oracle.com> Message-ID: <1a9f8c15-9fb3-a969-22fc-9529487ce931@oracle.com> On 11/28/18 13:43, dean.long at oracle.com wrote: > I guess even with -Xcomp, there are still compiles triggered by > profiled code, otherwise I don't see why changing policy()->event() > helps. > > Another problem I see with this change is that it will make the real > problem harder to reproduce. True. But on the other hand the tests become more stable. :) Thanks, Serguei > > dl > > On 11/28/18 12:36 PM, Alex Menkov wrote: >> Hi guys, >> >> I was able to reproduce the issue only with Graal + -Xcomp and only >> with debug build (I tried Graal + -Xint, Graal without Xcomp/Xint, >> just -Xcomp (i.e. C1 is used)). >> >> --alex >> >> On 11/28/2018 01:27, serguei.spitsyn at oracle.com wrote: >>> On 11/28/18 01:07, dean.long at oracle.com wrote: >>>> OK, reconsidering solution 1, I have a couple more questions: >>>> >>>> The bug report says the problem is reproducible with -Xcomp, but I >>>> don't see the policy()->event() path taken with -Xcomp. Instead, >>>> -Xcomp uses CompilationPolicy::compile_if_required, so I don't see >>>> how Solution 1 fixes the problem with -Xcomp. >>>> >>>> > CompilerRuntime always calls policy()->event with CompLevel == >>>> CompLevel_aot. >>>> >>>> As far as I can tell, CompLevel_aot is only used when there is >>>> active AOT code.? Is this an AOT-specific problem? >>> >>> Tracing showed that the CompLevel_aot is always used for JVMCI which >>> is pretty confusing. >>> Most likely, it is because now the AOT is enabled by default (is it >>> correct?). >>> I'm not sure it actually has anything to do with the AOT mode. >>> >>> >>>> ? I would expect there to be a problem with c1 as well. >>> >>> Not sure about the c1. >>> I hope, Alex could comment on this tomorrow. >>> >>> >>> Thanks, >>> Serguei >>> >>>> dl >>>> >>>> On 11/27/18 3:38 PM, serguei.spitsyn at oracle.com wrote: >>>>> Good summary with a list of solutions below! >>>>> >>>>> On 11/27/18 2:50 PM, dean.long at oracle.com wrote: >>>>>> Let me list the proposed solutions: >>>>>> >>>>>> 1. short-circuit compiles in CompilationPolicy::event in >>>>>> is_interp_only_mode >>>>>> 2. detect is_interp_only_mode in resolve stubs and force a c2i call >>>>>> 3. don't offer a JVMTI suspend point in resolve stubs (or >>>>>> JavaCallWrapper) >>>>>> >>>>>> Solution 1 penalizes other threads, and don't prevent other >>>>>> threads from compiling the method, so while it may help a little, >>>>>> it's incomplete. >>>>> >>>>> I doubt, the Solution 1 penalizes other threads. >>>>> First, it is an important pretty frequent/common case that allows >>>>> to save on compilations. >>>>> Second, there is no point to compile methods on a thread executed >>>>> in interp_only_mode. >>>>> Also, there is no big advantage to compile methods even for other >>>>> threads >>>>> when an active debugging (e.g. single stepping) is in progress. >>>>> Third, this would make it consistent with what the the interpreter >>>>> is doing. >>>>> >>>>> So, I think, the Solution 1 is needed independently of other >>>>> solutions we take. >>>>> My suggestion is to apply it in the JDK-8195639 bug fix. >>>>> >>>>>> Solutions 1 and 2 allow the thread to resume in a different >>>>>> method (the callee method), causing other problems. >>>>> >>>>> Agreed. >>>>> >>>>>> With Solution 3, the frame we suspend is the same as the frame we >>>>>> end up in after the resume, so I believe it solves all the problems. >>>>>> IMHO this is the correct solution to pursue. >>>>> >>>>> I agree in general. >>>>> This solution has some risks involved as it impacts a lot of code >>>>> including platform-independent. >>>>> It is also going to fix the other bugs: JDK-8195635, JDK-8214093, >>>>> JDK-8207013 (not sure about all of them yet). >>>>> >>>>> My suggestion is to pursue it as the JDK-8195635 fix. >>>>> Could you, please, confirm if works for you? >>>>> >>>>> Also, we have no consensus yet on the Solution 1. >>>>> >>>>> Thanks, >>>>> Serguei >>>>> >>>>>> dl >>>>>> >>>>>> On 11/27/18 2:19 PM, serguei.spitsyn at oracle.com wrote: >>>>>>> Hi Dean, >>>>>>> >>>>>>> You also asked the questions: >>>>>>> >>>>>>> > Doesn't is_interp_only_mode() only apply to the current thread? >>>>>>> >>>>>>> Yes it does. >>>>>>> >>>>>>> >>>>>>> > I don't think it's safe to assume no compiles will happen in >>>>>>> other threads, >>>>>>> > or that a method call target is not already compiled, because >>>>>>> as far >>>>>>> > as I can tell, JVMTI only deoptimizes the active frames. >>>>>>> >>>>>>> I agree with it. >>>>>>> However, compilations on the thread with the interp_only_mode >>>>>>> enabled is the most important case. >>>>>>> Disabling compilations such thread improves testing stability. >>>>>>> >>>>>>> > The bug report describes a case where the caller has been >>>>>>> deoptimized >>>>>>> > while resolving a call.? I don't see how this fix prevents the >>>>>>> target >>>>>>> > from being a previously compiled method. >>>>>>> > But we do need to make sure not to call into compiled code, so >>>>>>> I think >>>>>>> > the fix needs to be in code like >>>>>>> SharedRuntime::resolve_static_call_C(), >>>>>>> > where it returns get_c2i_entry() if is_interp_only_mode() is >>>>>>> true. >>>>>>> >>>>>>> Thank you for making this point. >>>>>>> >>>>>>> It would be nice to attack this as well. >>>>>>> We can try to investigate this approach further. >>>>>>> One problem is that there are more cases like >>>>>>> resolve_static_call_C, >>>>>>> for instance, resolve_virtual_call_C and >>>>>>> resolve_opt_virtual_call_C. >>>>>>> We may need to fix the same in other places, like >>>>>>> JavaCallWrapper::JavaCallWrapper. >>>>>>> >>>>>>> We can start from fixing it in the resolve_static_call_C. >>>>>>> If it is successful then continue this effort for other cases as >>>>>>> well. >>>>>>> >>>>>>> What do you think? >>>>>>> >>>>>>> Thanks, >>>>>>> Serguei >>>>>>> >>>>>>> >>>>>>> On 11/27/18 1:01 PM, Alex Menkov wrote: >>>>>>>> Hi Dean, >>>>>>>> >>>>>>>> Thank you for the feedback and for the comments in Jira. >>>>>>>> >>>>>>>> >> How does this solve the problem of >>>>>>>> >> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>>>> >>>>>>>> It doesn't. It fixes another issue. >>>>>>>> The test suspend test thread and ensures that top frame is a >>>>>>>> "fibonachi" method. Then it turns SingleStep on, does PopFrame >>>>>>>> and resumes the thread. >>>>>>>> The problem is the thread continues execution of compiled code >>>>>>>> (ignoring SingleStep) and we get SindleStep event only when >>>>>>>> adjustCompilationLevel method is called (it's interpreted code). >>>>>>>> >>>>>>>> There are several other bugs which are (most likely) caused by >>>>>>>> suspend during call setup (JDK-8195635, JDK-8214093, JDK-8207013) >>>>>>>> >>>>>>>> --alex >>>>>>>> >>>>>>>> On 11/27/2018 01:39, serguei.spitsyn at oracle.com wrote: >>>>>>>>> Hi Dean, >>>>>>>>> >>>>>>>>> Thank you a lot for looking at this! >>>>>>>>> Just a couple of points from me (it is up to Alex to provide a >>>>>>>>> full answer). >>>>>>>>> >>>>>>>>> >>>>>>>>> I think, Alex in this RFR missed to tell that we knew about >>>>>>>>> this issue that an incorrect frame will be popped. >>>>>>>>> But after some discussion we decided to file a separate issue >>>>>>>>> on this. >>>>>>>>> Alex wanted to create a good stand-alone test case >>>>>>>>> demonstrating this problem before filing it. >>>>>>>>> >>>>>>>>> Now, as I can see, the JDK-8195635 >>>>>>>>> looks very >>>>>>>>> close to a bug that we wanted to file. >>>>>>>>> The only difference is that our scenario includes the >>>>>>>>> SharedRuntime::resolve_static_call_C instead of the >>>>>>>>> JavaCallWrapper::JavaCallWrapper as a helper for Java method >>>>>>>>> invocation. >>>>>>>>> If I understand corrctly (Alex will fix me if needed), the >>>>>>>>> jtreg test we used to chase this issue did not catch a problem >>>>>>>>> with the HotSpotJVMCIRuntime.adjustCompilationLevel. >>>>>>>>> >>>>>>>>> >>>>>>>>> The suggested fix was to fix the mismatch in the >>>>>>>>> TieredThresholdPolicy::even with the comment in the >>>>>>>>> interpreter code: >>>>>>>>> nmethod* >>>>>>>>> InterpreterRuntime::frequency_counter_overflow(JavaThread* >>>>>>>>> thread, address branch_bcp) { >>>>>>>>> ?????. . . >>>>>>>>> ???if (nm != NULL && thread->is_interp_only_mode()) { >>>>>>>>> ?????// Normally we never get an nm if is_interp_only_mode() >>>>>>>>> is true, because >>>>>>>>> ?????// policy()->event has a check for this and won't compile >>>>>>>>> the method when >>>>>>>>> ?????// true. However, it's possible for is_interp_only_mode() >>>>>>>>> to become true >>>>>>>>> ?????// during the compilation. We don't want to return the nm >>>>>>>>> in that case >>>>>>>>> ?????// because we want to continue to execute interpreted. >>>>>>>>> ?????nm = NULL; >>>>>>>>> ???} >>>>>>>>> >>>>>>>>> ?> So I think the fix needs to be in code like >>>>>>>>> SharedRuntime::resolve_static_call_C(), >>>>>>>>> ?> where it returns get_c2i_entry() if is_interp_only_mode() >>>>>>>>> is true. >>>>>>>>> >>>>>>>>> I'm not sure, the adding this condition and returning the >>>>>>>>> get_c2i_entry() is always correct. >>>>>>>>> We need some help from the Compiler team to make it right. >>>>>>>>> >>>>>>>>> BTW, the interp_only_mode has been enabled only when some >>>>>>>>> interp_only events are enabled. >>>>>>>>> It is not set by either PopFrame or ForceEarlyReturn. >>>>>>>>> But the popframe009 test enables single stepping, so we wanted >>>>>>>>> to make this part right. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Serguei >>>>>>>>> >>>>>>>>> >>>>>>>>> On 11/26/18 16:05, dean.long at oracle.com wrote: >>>>>>>>>> How does this solve the problem of >>>>>>>>>> HotSpotJVMCIRuntime.adjustCompilationLevel being called? >>>>>>>>>> >>>>>>>>>> I don't think this fix is the right approach. Doesn't >>>>>>>>>> is_interp_only_mode() only apply to the current thread? I >>>>>>>>>> don't think it's safe to assume no compiles will happen in >>>>>>>>>> other threads, or that a method call target is not already >>>>>>>>>> compiled, because as far as I can tell, JVMTI only >>>>>>>>>> deoptimizes the active frames. The bug report describes a >>>>>>>>>> case where the caller has been deoptimized while resolving a >>>>>>>>>> call.? I don't see how this fix prevents the target from >>>>>>>>>> being a previously compiled method.? But we do need to make >>>>>>>>>> sure not to call into compiled code, so I think the fix needs >>>>>>>>>> to be in code like SharedRuntime::resolve_static_call_C(), >>>>>>>>>> where it returns get_c2i_entry() if is_interp_only_mode() is >>>>>>>>>> true. However, there is still another problem. By allowing >>>>>>>>>> JVMTI to suspend the thread during call setup, but reporting >>>>>>>>>> the frame as still in the caller instead of the callee, we >>>>>>>>>> confuse JVMTI into thinking that execution will resume in the >>>>>>>>>> caller instead of the callee. We may want to restrict where >>>>>>>>>> we offer JVMTI suspend points, and not offer a JVMTI suspend >>>>>>>>>> point in SharedRuntime::resolve_static_call_C and friends at >>>>>>>>>> all. >>>>>>>>>> >>>>>>>>>> dl >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 11/26/18 11:14 AM, Alex Menkov wrote: >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Please review the fix for >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195639 >>>>>>>>>>> webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~amenkov/popframe009/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> Description: >>>>>>>>>>> The test suspends a thread, turns on single stepping and >>>>>>>>>>> then calls PopFrame. SingleStep event is expected as soon as >>>>>>>>>>> the thread is resumed and PopFrame is processed (so we have >>>>>>>>>>> call stack with the depth 1 less than it was before >>>>>>>>>>> PopFrame). Instead SingleStep event is received with much >>>>>>>>>>> deeper call stack (and PopFrame processes wrong frame). >>>>>>>>>>> Research shown that this is caused by missed deoptimization >>>>>>>>>>> of the current frame. >>>>>>>>>>> As far as I understand CompilationPolicy::event should >>>>>>>>>>> handle the case when the thread has is_interp_only_mode() >>>>>>>>>>> set, but TieredThresholdPolicy::event checks this only then >>>>>>>>>>> CompLevel is CompLevel_none. >>>>>>>>>>> CompilerRuntime always calls policy()->event with CompLevel >>>>>>>>>>> == CompLevel_aot. >>>>>>>>>>> >>>>>>>>>>> The fix looks quite simple, but I'd appreciate feedback from >>>>>>>>>>> runtime and compiler teams as I'm not sure I completely >>>>>>>>>>> understand all the details of the "PopFrame dance". >>>>>>>>>>> >>>>>>>>>>> --alex >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> > From rkennke at redhat.com Wed Nov 28 22:29:12 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 28 Nov 2018 23:29:12 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> Message-ID: <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Alright, we fixed: - The minor issues that Kim reported in shared-gc - A lot of fixes in shared-tests according to Leonid's review - Disabled SA heapdumping similar to ZGC as Per suggested Some notes: Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually correct. The @requires there means to exclude runs with both CMS and ExplicitGCInvokesConcurrent at the same time, because that would be (expectedly) failing. It can run CMS, default GC and any other GC just fine. Adding the same clause for Shenandoah means the same, and filters the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I made the condition a bit clearer by avoiding triple-negation. See: http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html Per: Disabling the SA part for heapdumping makes 2 tests fail: - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java we filter them for Shenandoah now. I'm wondering: how do you get past those with ZGC? See: http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html (Note to Leonid and tests reviewers: I'll add those related filters in next round). Vladimir: Roland integrated a bunch of changes to make loop* code look better. I can tell that we're not done with C2 yet. Can you look over the code and see what is ok, and especially what is not ok, so that we can focus our efforts on the relevant parts? Updated set of webrevs: http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ Thanks, Roman > Hi, > > This is the first round of changes for including Shenandoah GC into > mainline. > I divided the review into parts that roughly correspond to the mailing lists > that would normally review it, and I divided it into 'shared' code > changes and > 'shenandoah' code changes (actually, mostly additions). The intend is to > eventually > push them as single 'combined' changeset, once reviewed. > > JEP: > ? https://openjdk.java.net/jeps/189 > Bug entry: > > ?https://bugs.openjdk.java.net/browse/JDK-8214259 > > Webrevs: > ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ > > For those who want to see the full change, have a look at the > shenandoah-complete > > directory, > it contains the full combined webrev. Alternatively, there is the file > shenandoah-master.patch > , > which is what I intend to commit (and which should be equivalent to the > 'shenandoah-complete' webrev). > > Sections to review (at this point) are the following: > ?*) shenandoah-gc > > ??? - Actual Shenandoah implementation, almost completely residing in > gc/shenandoah > > ?*) shared-gc > > ??? - This is mostly boilerplate that is common to any GC > ??? - referenceProcessor.cpp has a little change to make one assert not > fail (next to CMS and G1) > ??? - taskqueue.hpp has some small adjustments to enable subclassing > > ?*) shared-serviceability > > ??? - The usual code to support another GC > > ?*) shared-runtime > > ??? - A number of friends declarations to allow Shenandoah iterators to > hook up with, > ????? e.g. ClassLoaderData, CodeCache, etc > ??? - Warning and disabling JFR LeakProfiler > ??? - fieldDescriptor.hpp added is_stable() accessor, for use in > Shenandoah C2 optimizations > ??? - Locks initialization in mutexLocker.cpp as usual > ??? - VM operations defines for Shenandoah's VM ops > ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in > Shenandoah's logging > ??? - The usual macros in macro.hpp > > ?*) shared-build > > ??? - Add shenandoah feature, enabled by default, as agreed with > Vladimir K. beforehand > ??? - Some flags for shenandoah-enabled compilation to get > SUPPORT_BARRIER_ON_PRIMITIVES > ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for > Shenandoah's barriers > ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source > files, which is > ????? useful to get the whole marking loop inlined (observed significant > regression if we > ????? don't) > > ?*) shared-tests > > ??? - Test infrastructure to support Shenandoah > ??? - Shenandoah test groups > ??? - Exclude Shenandoah in various tests that can be run with selected GC > ??? - Enable/add configure for Shenandoah for tests that make sense to > run with it > > ?*) shenandoah-tests > > ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory > ??? - A couple of tests configurations have been added, e.g. > TestGCBasherWithShenandoah.java > > I intentionally left out shared-compiler for now, because we have some > work left to do > there, but if you click around you'll find the patch anyway, in case you > want to take > a peek at it. > > We have regular builds on: > ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} > ? - {Windows} x {x86_64}, > ? - {MacOS X} x {x86_64} > > This also routinely passes: > ? - the new Shenandoah tests > ? - jcstress with/without aggressive Shenandoah verification > ? - specjvm2008 with/without aggressive Shenandoah verification > > > I'd like to thank my collegues at Red Hat: Christine Flood, she deserves > the credit for being the original inventor of Shenandoah, Aleksey > Shipl?v, Roland Westrelin & Zhengyu Gu for their countless > contributions, everybody else in Red Hat's OpenJDK team for testing, > advice and support, my collegues in Oracle's GC, runtime and compiler > teams for tirelessly helping with and reviewing all the GC interface and > related changes, and of course the many early adopters for reporting > bugs and success stories and feature requests: we wouldn't be here > without any of you! > > Best regards, > Roman > From coleen.phillimore at oracle.com Wed Nov 28 22:29:14 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 28 Nov 2018 17:29:14 -0500 Subject: RFR(S) [downport 11]: 8213410: UseCompressedOops requirement check fails fails on 32-bit system In-Reply-To: <0a52708b571f4c729664900b4a7e5bd1@sap.com> References: <0a52708b571f4c729664900b4a7e5bd1@sap.com> Message-ID: <9665eca5-7d15-4499-d2b2-5f927506cad3@oracle.com> This seems good. Coleen On 11/28/18 4:00 AM, Lindenmaier, Goetz wrote: > Hi, > > I need a review to downport 8213410. > 8213410 fixes disabling tests for UseCompressedOops on 32-bit platforms. > > In jdk12, "@requires vm.bits == 64" was introduced to CompressedClassPointers.java > by "8208655: use JTreg skipped status in hotspot tests". Because of this change, > 8213410 does not apply properly. > To fix this, I introduce "@requires vm.bits == 64" to this test in jdk11, too. > http://cr.openjdk.java.net/~goetz/wr18/8213410-32bit_cOops-jdk11/01/ > > Best regards, > Goetz. > From vladimir.kozlov at oracle.com Thu Nov 29 00:55:02 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 28 Nov 2018 16:55:02 -0800 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Message-ID: About C2 changes. .ad files. I am thinking may be we should put new code in separate _shenandoah.ad files and merge them only if Shenandoah is included in built [1]. I don't like #ifdefs - you still generate this mach nodes even without Shenandoah. debug_final_field_at() and debug_stable_field_at() are not used - could be excluded from changes. loopnode.cpp Sometimes you add #if INCLUDE_SHENANDOAHGC and sometimes don't. Be consistent. I don't know why you need #ifdef if such nodes should be generated without Shenandoah compile.cpp - no need to include shenandoahBarrierSetC2.hpp type.hpp - cast_to_nonconst() are not used. phasetype.hpp, subnode.cpp - don't add files with only cleanup changes. I wish loopnode.cpp and loopopts.cpp changes were more clean. I like the idea where you expose only one ShenandoahBarrier node to C2 and hide all subnodes behind it. I really don't want to see a lot of Shenandoah*Node new classes in C2 shared code. Thanks, Vladimir [1] http://hg.openjdk.java.net/jdk/jdk/file/157c1130b46e/make/hotspot/gensrc/GensrcAdlc.gmk#l122 On 11/28/18 2:29 PM, Roman Kennke wrote: > Alright, we fixed: > - The minor issues that Kim reported in shared-gc > - A lot of fixes in shared-tests according to Leonid's review > - Disabled SA heapdumping similar to ZGC as Per suggested > > Some notes: > Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually > correct. The @requires there means to exclude runs with both CMS and > ExplicitGCInvokesConcurrent at the same time, because that would be > (expectedly) failing. It can run CMS, default GC and any other GC just > fine. Adding the same clause for Shenandoah means the same, and filters > the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I > made the condition a bit clearer by avoiding triple-negation. > > See: > http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html > > Per: Disabling the SA part for heapdumping makes 2 tests fail: > - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java > - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java > > we filter them for Shenandoah now. I'm wondering: how do you get past > those with ZGC? > > See: > http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html > > (Note to Leonid and tests reviewers: I'll add those related filters in > next round). > > Vladimir: Roland integrated a bunch of changes to make loop* code look > better. I can tell that we're not done with C2 yet. Can you look over > the code and see what is ok, and especially what is not ok, so that we > can focus our efforts on the relevant parts? > > Updated set of webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ > > Thanks, > Roman > > >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the mailing lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is to >> eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> ? https://openjdk.java.net/jeps/189 >> Bug entry: >> >> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> which is what I intend to commit (and which should be equivalent to the >> 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> ?*) shenandoah-gc >> >> ??? - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> ?*) shared-gc >> >> ??? - This is mostly boilerplate that is common to any GC >> ??? - referenceProcessor.cpp has a little change to make one assert not >> fail (next to CMS and G1) >> ??? - taskqueue.hpp has some small adjustments to enable subclassing >> >> ?*) shared-serviceability >> >> ??? - The usual code to support another GC >> >> ?*) shared-runtime >> >> ??? - A number of friends declarations to allow Shenandoah iterators to >> hook up with, >> ????? e.g. ClassLoaderData, CodeCache, etc >> ??? - Warning and disabling JFR LeakProfiler >> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> ??? - Locks initialization in mutexLocker.cpp as usual >> ??? - VM operations defines for Shenandoah's VM ops >> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> ??? - The usual macros in macro.hpp >> >> ?*) shared-build >> >> ??? - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> ??? - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> ????? useful to get the whole marking loop inlined (observed significant >> regression if we >> ????? don't) >> >> ?*) shared-tests >> >> ??? - Test infrastructure to support Shenandoah >> ??? - Shenandoah test groups >> ??? - Exclude Shenandoah in various tests that can be run with selected GC >> ??? - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> ?*) shenandoah-tests >> >> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >> ??? - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case you >> want to take >> a peek at it. >> >> We have regular builds on: >> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> ? - {Windows} x {x86_64}, >> ? - {MacOS X} x {x86_64} >> >> This also routinely passes: >> ? - the new Shenandoah tests >> ? - jcstress with/without aggressive Shenandoah verification >> ? - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >> the credit for being the original inventor of Shenandoah, Aleksey >> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface and >> related changes, and of course the many early adopters for reporting >> bugs and success stories and feature requests: we wouldn't be here >> without any of you! >> >> Best regards, >> Roman >> > From jini.george at oracle.com Thu Nov 29 04:39:31 2018 From: jini.george at oracle.com (Jini George) Date: Thu, 29 Nov 2018 10:09:31 +0530 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Message-ID: Hi Roman, The SA tests which iterate over the heap (ClhsdbJhisto.java, TestHeapDumpFor*.java) fail if ZGC is passed as an option to these also (Am planning on fixing these). It might be better to avoid invoking those tests for Shenandoah now with an @requires tag. Thank you, Jini On 11/29/2018 3:59 AM, Roman Kennke wrote: > Alright, we fixed: > - The minor issues that Kim reported in shared-gc > - A lot of fixes in shared-tests according to Leonid's review > - Disabled SA heapdumping similar to ZGC as Per suggested > > Some notes: > Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually > correct. The @requires there means to exclude runs with both CMS and > ExplicitGCInvokesConcurrent at the same time, because that would be > (expectedly) failing. It can run CMS, default GC and any other GC just > fine. Adding the same clause for Shenandoah means the same, and filters > the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I > made the condition a bit clearer by avoiding triple-negation. > > See: > http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html > > Per: Disabling the SA part for heapdumping makes 2 tests fail: > - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java > - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java > > we filter them for Shenandoah now. I'm wondering: how do you get past > those with ZGC? > > See: > http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html > > (Note to Leonid and tests reviewers: I'll add those related filters in > next round). > > Vladimir: Roland integrated a bunch of changes to make loop* code look > better. I can tell that we're not done with C2 yet. Can you look over > the code and see what is ok, and especially what is not ok, so that we > can focus our efforts on the relevant parts? > > Updated set of webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ > > Thanks, > Roman > > >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the mailing lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is to >> eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> ? https://openjdk.java.net/jeps/189 >> Bug entry: >> >> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> which is what I intend to commit (and which should be equivalent to the >> 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> ?*) shenandoah-gc >> >> ??? - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> ?*) shared-gc >> >> ??? - This is mostly boilerplate that is common to any GC >> ??? - referenceProcessor.cpp has a little change to make one assert not >> fail (next to CMS and G1) >> ??? - taskqueue.hpp has some small adjustments to enable subclassing >> >> ?*) shared-serviceability >> >> ??? - The usual code to support another GC >> >> ?*) shared-runtime >> >> ??? - A number of friends declarations to allow Shenandoah iterators to >> hook up with, >> ????? e.g. ClassLoaderData, CodeCache, etc >> ??? - Warning and disabling JFR LeakProfiler >> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> ??? - Locks initialization in mutexLocker.cpp as usual >> ??? - VM operations defines for Shenandoah's VM ops >> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> ??? - The usual macros in macro.hpp >> >> ?*) shared-build >> >> ??? - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> ??? - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> ????? useful to get the whole marking loop inlined (observed significant >> regression if we >> ????? don't) >> >> ?*) shared-tests >> >> ??? - Test infrastructure to support Shenandoah >> ??? - Shenandoah test groups >> ??? - Exclude Shenandoah in various tests that can be run with selected GC >> ??? - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> ?*) shenandoah-tests >> >> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >> ??? - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case you >> want to take >> a peek at it. >> >> We have regular builds on: >> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> ? - {Windows} x {x86_64}, >> ? - {MacOS X} x {x86_64} >> >> This also routinely passes: >> ? - the new Shenandoah tests >> ? - jcstress with/without aggressive Shenandoah verification >> ? - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >> the credit for being the original inventor of Shenandoah, Aleksey >> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface and >> related changes, and of course the many early adopters for reporting >> bugs and success stories and feature requests: we wouldn't be here >> without any of you! >> >> Best regards, >> Roman >> > From ioi.lam at oracle.com Thu Nov 29 05:56:06 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 28 Nov 2018 21:56:06 -0800 Subject: What's the best way to append an oop to a GrowableArray? Message-ID: <49ca3441-7cf2-ba30-7cb5-107441f9ea53@oracle.com> I have a memory model question: I need to create lots of objects and append them into a GrowableArray GrowableArray* arr = new (ResourceObj::C_HEAP, mtInternal) ??? GrowableArray(10000, true);; void doit() { ?? while (...) { ?????? oop o = allocate_from_heap(.....); ?????? arr->append(o); ??? } } There is only a single thread that can append to the array, so I am not using any locks. However, GC could happen inside this loop, so I have written a do_oops() function that would scan the array during GC. (For convenience, I am calling my_oops_do() from within SystemDictionary:oops_do() for now ...). void my_oops_do(OopClosure* f) { ??? int len = arr->length(); ??? for (int i = 0; i < len; i++) { ????? f->do_oop(arr->adr_at(i)); ??? } ? } } This seems to work fine on my dual socket Xeon, but will it work everywhere? Do I need to do anything to ensure that the GC threads can read the up-to-date values of the array elements, and the array length? Thanks - Ioi From Yang.Zhang at arm.com Thu Nov 29 06:39:50 2018 From: Yang.Zhang at arm.com (Yang Zhang (Arm Technology China)) Date: Thu, 29 Nov 2018 06:39:50 +0000 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> Message-ID: Hi Vladimir I run all the jtreg test(hotspot/langtools/jdk, compiler/c2/cr634086 is included). They are passed. Regards Yang -----Original Message----- From: Vladimir Kozlov Sent: Thursday, November 29, 2018 2:13 AM To: Yang Zhang (Arm Technology China) ; Andrew Haley ; hotspot-dev at openjdk.java.net Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 Looks good. What testing you did? Unfortunately compiler/c2/cr634086 test is not part of tier1 testing and not executed by 'submit' repo testing. You need to run it. Thanks, Vladimir On 11/28/18 12:32 AM, Yang Zhang (Arm Technology China) wrote: > Hi Andrew, Vladimir > > I update the reviewers and remove the new assert in Matcher::vector_shift_count_ideal_reg(). Could you please help review again? > > http://cr.openjdk.java.net/~yzhang/8213134/webrev.02/ > > Regards > Yang > > -----Original Message----- > From: Vladimir Kozlov > Sent: Thursday, November 22, 2018 1:56 AM > To: Yang Zhang (Arm Technology China) ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with > MaxVectorSize=8 > > You are right. I think I was confused that you are not limiting MaxVectorSize but simple use min(16, MaxVectorSize). > > The assert is fine. > > Thanks, > Vladimir > > On 11/20/18 10:28 PM, Yang Zhang (Arm Technology China) wrote: >> >>> I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. >>> In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). >>> aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. >> >> Matcher::vector_width_in_bytes() limits vector size to min(16, MaxVectorSize). >> Matcher::vector_shift_count_ideal_reg() assert(MaxVectorSize >= size, >> "Length isn't supported"); When MaxVectorSize is 8, size is 8. >> When MaxVectorSize is greater than or equal 16, size is 16. >> So (MaxVectorSize >= size) is always true. >> >> Test cases of cr6340864 are passed with or without this new assert. >> >> Do I need to remove this new assert? >> >> Regards >> Yang >> >> >> On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: >>> Hi, >>> >>> When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? >>> >>> Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 >>> >>> This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. >>> Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. >>> compiler/c2/cr6340864/TestByteVect.java >>> compiler/c2/cr6340864/TestIntVect.java >>> compiler/c2/cr6340864/TestShortVect.java >>> compiler/codegen/TestCharVect2.java >>> >>> Regards >>> Yang >>> >>> >> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. >> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. From tobias.hartmann at oracle.com Thu Nov 29 08:07:21 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 29 Nov 2018 09:07:21 +0100 Subject: RFR: 8214257: IC cache not clean after cleaning assertion failure In-Reply-To: References: Message-ID: <39e41533-4bd5-fb5a-4a9f-cc305953a367@oracle.com> Hi Erik, looks good to me. Please run at least hs-tier1,hs-tier2,hs-precheckin-comp before pushing. Best regards, Tobias On 27.11.18 14:00, Erik ?sterlund wrote: > Hi, > > Back in 8212681, an abstract CompiledICLocker was introduce to protect inline caches from concurrent > patching. > > Unfortunately, not all patching of ICs started using the CompiledICLocker. There were still some > callsites using the Patching_lock. Therefore, the assert may trigger sometimes saying the IC cache > is not clean right after cleaning, due to other concurrent modifications under the Patching_lock. > > Also, in CompiledIC::set_to_clean(), the condition for safe_transition was altered incorrectly from > is_at_safepoint() to CompiledICLocker::is_safe(), causing cleaning to not used transition, so I > reinstated that in this patch. > > This patch depends on 8214338 by assuming that the CompiledICLocker does not use safepoint checks, > as it is now used from leaf calls where safepointing is not allowed. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8214257 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8214257/webrev.00/ > > Thanks, > /Erik From goetz.lindenmaier at sap.com Thu Nov 29 08:19:48 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 29 Nov 2018 08:19:48 +0000 Subject: RFR(S) [downport 11]: 8213410: UseCompressedOops requirement check fails fails on 32-bit system In-Reply-To: <9665eca5-7d15-4499-d2b2-5f927506cad3@oracle.com> References: <0a52708b571f4c729664900b4a7e5bd1@sap.com> <9665eca5-7d15-4499-d2b2-5f927506cad3@oracle.com> Message-ID: <9dd31591ed214ab6858e7365f7b1b662@sap.com> Hi Coleen, thanks for looking at this! Best regards, Goetz. > -----Original Message----- > From: hotspot-dev On Behalf Of > coleen.phillimore at oracle.com > Sent: Mittwoch, 28. November 2018 23:29 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(S) [downport 11]: 8213410: UseCompressedOops > requirement check fails fails on 32-bit system > > > This seems good. > Coleen > > On 11/28/18 4:00 AM, Lindenmaier, Goetz wrote: > > Hi, > > > > I need a review to downport 8213410. > > 8213410 fixes disabling tests for UseCompressedOops on 32-bit platforms. > > > > In jdk12, "@requires vm.bits == 64" was introduced to > CompressedClassPointers.java > > by "8208655: use JTreg skipped status in hotspot tests". Because of this > change, > > 8213410 does not apply properly. > > To fix this, I introduce "@requires vm.bits == 64" to this test in jdk11, too. > > http://cr.openjdk.java.net/~goetz/wr18/8213410-32bit_cOops-jdk11/01/ > > > > Best regards, > > Goetz. > > From rkennke at redhat.com Thu Nov 29 09:59:18 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Nov 2018 10:59:18 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Message-ID: Hi Vladimir, thanks for reviewing! > .ad files. I am thinking may be we should put new code in separate > _shenandoah.ad files and merge them only if Shenandoah is included > in built [1]. I don't like #ifdefs - you still generate this mach nodes > even without Shenandoah. We're looking into improving the situation. > debug_final_field_at() and debug_stable_field_at() are not used - could > be excluded from changes. Those are used from shenandoahBarrierSetC2.cpp and related code in in the shenandoah-gc or shenandoah-complete webrevs. > loopnode.cpp Sometimes you add #if INCLUDE_SHENANDOAHGC and sometimes > don't. Be consistent. I don't know why you need #ifdef if such nodes > should be generated without Shenandoah We'll look into improving that. > compile.cpp - no need to include shenandoahBarrierSetC2.hpp Right. Will fix it. > type.hpp - cast_to_nonconst() are not used. As above, this is used from shenandoahBarrierSetC2.cpp > phasetype.hpp, subnode.cpp - don't add files with only cleanup changes. right. Those are leftovers and I'll remove them. > I wish loopnode.cpp and loopopts.cpp changes were more clean. We'll see what we can do. Some of those could possibly be turned into some sort of GC interface, but 1. it would be very Shenandoah-specific and 2. require significant duplication of code. > I like the idea where you expose only one ShenandoahBarrier node to C2 > and hide all subnodes behind it. I really don't want to see a lot of > Shenandoah*Node new classes in C2 shared code. I understand that. We have ShenandoahBarrierNode superclass and a bunch of ShenandoahReadBarrier, WriteBarrier and some other nodes 'behind' this. The crux is probably all the various ShenandoahCompareAndSwap etc variants, which need to be subclasses of their non-Shenandoah-counterparts, and I don't see how to change that. Thanks for reviewing and helping! Roman > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/157c1130b46e/make/hotspot/gensrc/GensrcAdlc.gmk#l122 > > > On 11/28/18 2:29 PM, Roman Kennke wrote: >> Alright, we fixed: >> - The minor issues that Kim reported in shared-gc >> - A lot of fixes in shared-tests according to Leonid's review >> - Disabled SA heapdumping similar to ZGC as Per suggested >> >> Some notes: >> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >> correct. The @requires there means to exclude runs with both CMS and >> ExplicitGCInvokesConcurrent at the same time, because that would be >> (expectedly) failing. It can run CMS, default GC and any other GC just >> fine. Adding the same clause for Shenandoah means the same, and filters >> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >> made the condition a bit clearer by avoiding triple-negation. >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >> >> >> Per: Disabling the SA part for heapdumping makes 2 tests fail: >> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >> >> we filter them for Shenandoah now. I'm wondering: how do you get past >> those with ZGC? >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >> >> >> (Note to Leonid and tests reviewers: I'll add those related filters in >> next round). >> >> Vladimir: Roland integrated a bunch of changes to make loop* code look >> better. I can tell that we're not done with C2 yet. Can you look over >> the code and see what is ok, and especially what is not ok, so that we >> can focus our efforts on the relevant parts? >> >> Updated set of webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >> >> Thanks, >> Roman >> >> >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the >>> mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ?? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ??*) shenandoah-gc >>> >>> >>> ???? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ??*) shared-gc >>> >>> ???? - This is mostly boilerplate that is common to any GC >>> ???? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >>> >>> ??*) shared-runtime >>> >>> >>> ???? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ?????? e.g. ClassLoaderData, CodeCache, etc >>> ???? - Warning and disabling JFR LeakProfiler >>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ???? - Locks initialization in mutexLocker.cpp as usual >>> ???? - VM operations defines for Shenandoah's VM ops >>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ???? - The usual macros in macro.hpp >>> >>> ??*) shared-build >>> >>> >>> ???? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ???? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ?????? useful to get the whole marking loop inlined (observed >>> significant >>> regression if we >>> ?????? don't) >>> >>> ??*) shared-tests >>> >>> >>> ???? - Test infrastructure to support Shenandoah >>> ???? - Shenandoah test groups >>> ???? - Exclude Shenandoah in various tests that can be run with >>> selected GC >>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ??*) shenandoah-tests >>> >>> >>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>> subdirectory >>> ???? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ?? - {Windows} x {x86_64}, >>> ?? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ?? - the new Shenandoah tests >>> ?? - jcstress with/without aggressive Shenandoah verification >>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> >> From rkennke at redhat.com Thu Nov 29 10:42:23 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Nov 2018 11:42:23 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Message-ID: <95ceb785-25e0-9a0b-7ca4-ca0a2af765a7@redhat.com> Hi Jini, > The SA tests which iterate over the heap (ClhsdbJhisto.java, > TestHeapDumpFor*.java) fail if ZGC is passed as an option to these also > (Am planning on fixing these). It might be better to avoid invoking > those tests for Shenandoah now with an @requires tag. Thanks for clarifying. Yeah, we already have a fix in shenandoah/jdk and it will appear in next round of reviews. Thanks, Roman > Thank you, > Jini > > On 11/29/2018 3:59 AM, Roman Kennke wrote: >> Alright, we fixed: >> - The minor issues that Kim reported in shared-gc >> - A lot of fixes in shared-tests according to Leonid's review >> - Disabled SA heapdumping similar to ZGC as Per suggested >> >> Some notes: >> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >> correct. The @requires there means to exclude runs with both CMS and >> ExplicitGCInvokesConcurrent at the same time, because that would be >> (expectedly) failing. It can run CMS, default GC and any other GC just >> fine. Adding the same clause for Shenandoah means the same, and filters >> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >> made the condition a bit clearer by avoiding triple-negation. >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >> >> >> Per: Disabling the SA part for heapdumping makes 2 tests fail: >> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >> >> we filter them for Shenandoah now. I'm wondering: how do you get past >> those with ZGC? >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >> >> >> (Note to Leonid and tests reviewers: I'll add those related filters in >> next round). >> >> Vladimir: Roland integrated a bunch of changes to make loop* code look >> better. I can tell that we're not done with C2 yet. Can you look over >> the code and see what is ok, and especially what is not ok, so that we >> can focus our efforts on the relevant parts? >> >> Updated set of webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >> >> Thanks, >> Roman >> >> >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the >>> mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ?? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ??*) shenandoah-gc >>> >>> >>> ???? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ??*) shared-gc >>> >>> ???? - This is mostly boilerplate that is common to any GC >>> ???? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >>> >>> ??*) shared-runtime >>> >>> >>> ???? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ?????? e.g. ClassLoaderData, CodeCache, etc >>> ???? - Warning and disabling JFR LeakProfiler >>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ???? - Locks initialization in mutexLocker.cpp as usual >>> ???? - VM operations defines for Shenandoah's VM ops >>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ???? - The usual macros in macro.hpp >>> >>> ??*) shared-build >>> >>> >>> ???? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ???? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ?????? useful to get the whole marking loop inlined (observed >>> significant >>> regression if we >>> ?????? don't) >>> >>> ??*) shared-tests >>> >>> >>> ???? - Test infrastructure to support Shenandoah >>> ???? - Shenandoah test groups >>> ???? - Exclude Shenandoah in various tests that can be run with >>> selected GC >>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ??*) shenandoah-tests >>> >>> >>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>> subdirectory >>> ???? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ?? - {Windows} x {x86_64}, >>> ?? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ?? - the new Shenandoah tests >>> ?? - jcstress with/without aggressive Shenandoah verification >>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> >> From magnus.ihse.bursie at oracle.com Thu Nov 29 10:53:47 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 29 Nov 2018 11:53:47 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> Message-ID: <1264c130-6bf1-f71f-8811-a2bf716245af@oracle.com> On 2018-11-26 23:47, Erik Joelsson wrote: > Build changes look ok to me. I agree, but I'd like to point out a (possible) style improvement. In hotspot.m4, if test "x$OPENJDK_TARGET_CPU" = "xx86_64" || test "x$OPENJDK_TARGET_CPU" = "xaarch64" || test "x$OPENJDK_TARGET_CPU" == "xx86"; then can be more succinctly expressed as if test "x$OPENJDK_TARGET_CPU_ARCH" = "xx86" || test "x$OPENJDK_TARGET_CPU" = "xaarch64" ; then But it's a matter of preference, if you do not want to change. Otherwise it looks good! /Magnus > > /Erik > > On 2018-11-26 13:39, Roman Kennke wrote: >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the >> mailing lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is to >> eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> https://openjdk.java.net/jeps/189 >> Bug entry: >> >> https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> >> which is what I intend to commit (and which should be equivalent to the >> 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> *) shenandoah-gc >> >> >> - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> *) shared-gc >> >> - This is mostly boilerplate that is common to any GC >> - referenceProcessor.cpp has a little change to make one assert not >> fail (next to CMS and G1) >> - taskqueue.hpp has some small adjustments to enable subclassing >> >> *) shared-serviceability >> >> >> - The usual code to support another GC >> >> *) shared-runtime >> >> >> - A number of friends declarations to allow Shenandoah iterators to >> hook up with, >> e.g. ClassLoaderData, CodeCache, etc >> - Warning and disabling JFR LeakProfiler >> - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> - Locks initialization in mutexLocker.cpp as usual >> - VM operations defines for Shenandoah's VM ops >> - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> - The usual macros in macro.hpp >> >> *) shared-build >> >> >> - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> useful to get the whole marking loop inlined (observed >> significant >> regression if we >> don't) >> >> *) shared-tests >> >> >> - Test infrastructure to support Shenandoah >> - Shenandoah test groups >> - Exclude Shenandoah in various tests that can be run with >> selected GC >> - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> *) shenandoah-tests >> >> >> - Shenandoah specific tests, most reside in gc/shenandoah >> subdirectory >> - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case you >> want to take >> a peek at it. >> >> We have regular builds on: >> - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> - {Windows} x {x86_64}, >> - {MacOS X} x {x86_64} >> >> This also routinely passes: >> - the new Shenandoah tests >> - jcstress with/without aggressive Shenandoah verification >> - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >> the credit for being the original inventor of Shenandoah, Aleksey >> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface and >> related changes, and of course the many early adopters for reporting >> bugs and success stories and feature requests: we wouldn't be here >> without any of you! >> >> Best regards, >> Roman >> From per.liden at oracle.com Thu Nov 29 11:07:44 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 29 Nov 2018 12:07:44 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> Message-ID: <87720eda-7239-ba21-b93d-674e6cee1f7e@oracle.com> Small comment on the build changes. Instead of checking for zero variant here: + # Only enable Shenandoah on supported arches + AC_MSG_CHECKING([if shenandoah can be built]) + if HOTSPOT_CHECK_JVM_VARIANT(zero); then + DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES shenandoahgc" + AC_MSG_RESULT([no, this JVM variant not supported]) + else I think you should just add shanandoahgc to the existing check, here: # Disable unsupported GCs for Zero if HOTSPOT_CHECK_JVM_VARIANT(zero); then DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES epsilongc g1gc zgc" fi cheers, Per On 11/26/18 11:47 PM, Erik Joelsson wrote: > Build changes look ok to me. > > /Erik > > On 2018-11-26 13:39, Roman Kennke wrote: >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the mailing >> lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is to >> eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> ?? https://openjdk.java.net/jeps/189 >> Bug entry: >> >> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> >> which is what I intend to commit (and which should be equivalent to the >> 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> ??*) shenandoah-gc >> >> >> ???? - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> ??*) shared-gc >> >> ???? - This is mostly boilerplate that is common to any GC >> ???? - referenceProcessor.cpp has a little change to make one assert not >> fail (next to CMS and G1) >> ???? - taskqueue.hpp has some small adjustments to enable subclassing >> >> ??*) shared-serviceability >> >> >> ???? - The usual code to support another GC >> >> ??*) shared-runtime >> >> >> ???? - A number of friends declarations to allow Shenandoah iterators to >> hook up with, >> ?????? e.g. ClassLoaderData, CodeCache, etc >> ???? - Warning and disabling JFR LeakProfiler >> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> ???? - Locks initialization in mutexLocker.cpp as usual >> ???? - VM operations defines for Shenandoah's VM ops >> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> ???? - The usual macros in macro.hpp >> >> ??*) shared-build >> >> >> ???? - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> ???? - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> ?????? useful to get the whole marking loop inlined (observed significant >> regression if we >> ?????? don't) >> >> ??*) shared-tests >> >> >> ???? - Test infrastructure to support Shenandoah >> ???? - Shenandoah test groups >> ???? - Exclude Shenandoah in various tests that can be run with >> selected GC >> ???? - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> ??*) shenandoah-tests >> >> >> ???? - Shenandoah specific tests, most reside in gc/shenandoah >> subdirectory >> ???? - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case you >> want to take >> a peek at it. >> >> We have regular builds on: >> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> ?? - {Windows} x {x86_64}, >> ?? - {MacOS X} x {x86_64} >> >> This also routinely passes: >> ?? - the new Shenandoah tests >> ?? - jcstress with/without aggressive Shenandoah verification >> ?? - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >> the credit for being the original inventor of Shenandoah, Aleksey >> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface and >> related changes, and of course the many early adopters for reporting >> bugs and success stories and feature requests: we wouldn't be here >> without any of you! >> >> Best regards, >> Roman >> From amith.pawar at gmail.com Thu Nov 29 11:08:14 2018 From: amith.pawar at gmail.com (amith pawar) Date: Thu, 29 Nov 2018 16:38:14 +0530 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> Message-ID: Hi Thomas, Sorry for the late reply and please check my inline comments below. On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl wrote: > Hi Amith, > > welcome to the OpenJDK community! :) > Thanks > > On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: > > Hi all, > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, > > S1 and S2 region (if any other ) memory areas on requested Numa nodes > > and it should not configure itself to access other Numa nodes. This > > issue is observed only when Java is allowed to run on fewer NUMA > > nodes than available on the system with numactl membind and > > interleave options. Running on all the nodes does not have any > > effect. This will cause some applications (objects residing in old > > gen and survivor region) to run slower on system with large Numa > > nodes. > > > [... long explanation...] > > Is it possible to summarize the problem and the changes with the > following few lines: > > "NUMA interleaving of memory of old gen and survivor spaces (for > parallel GC) tells the OS to interleave memory across all nodes of a > NUMA system. However the VM process may be configured to be limited to > run only on a few nodes, which means that large parts of the heap will > be located on foreign nodes. This can incurs a large performance > penalty. > > The proposed solution is to tell the OS to interleave memory only > across available nodes when enabling NUMA." > OK. Since, I dont have write access to the defect link so can I request you do the same. > > We have had trouble understanding the problem statement and purpose of > this patch when triaging (making sure the issue is understandable and > can be worked on) as the text is rather technical. Having an easily > understandable text also helps reviewers a lot. > > Assuming my summary is appropriate, I have several other unrelated > questions: > > - could you propose a better subject for this work? "JDK NUMA > Interleaving issue" seems very generic. Something like "NUMA heap > allocation does not respect VM membind/interleave settings" maybe? > This is also OK. > > - there have been other NUMA related patches from AMD recently, in > particular JDK-what is the relation to JDK-8205051? The text there > reads awfully similar to this one, but I *think* JDK-8205051 is > actually about making sure that the parallel gc eden is not put on > inactive nodes. > Can you confirm this (talk to your colleague) so that we can fix the > description too? > JDK-8205051 is related to early GC. JDK-8189922 specific to membind case where heap was allocated on non requested NUMA nodes. The proposed patch fixes two following issues 1. Similar to JDK-8189922 but for interleave case. 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer NUMA nodes. > - fyi, we are currently working on NUMA aware memory allocation support > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to > not step on each other's toes (see below). > We are not working anything related to G1. It may effect G1 if numa_make_global function is called. > > [... broken patch...] > > I tried to apply the patch to the jdk/jdk tree, which failed; I then > started to manually apply it but stopped after not being able to find > the context of some hunks. I do not think this change applies to the > latest source tree. > > Please make sure that the patch applies to the latest jdk/jdk tree with > errors. All changes generally must first go into the latest dev tree > before you can apply for backporting. > > Could you please send the patch as attachment (not copy&pasted) to this > list and cc me? Then I can create a webrev out of it. > > I did look a bit over the patch as much as I could (it's very hard > trying to review a diff), some comments: > Sorry. Please check the attached patch. > > - the _numa_interleave_memory_v2 function pointer is never assigned > to in the patch in the CR, so it will not be used. Please make sure the > patch is complete. > Actually it is never defined anywhere, ie. the patch unlikely actually > compiles even if I could apply it somewhere. > > Please avoid frustrating reviewers by sending incomplete patches. > Sorry again. Faced same issue when copied from the defect link but worked for me from the mail. Now i have attached the patch. Can you please update defect link with this patch ? > > - I am not sure set_numa_interleave, i.e. the interleaving, should be > done without UseNUMAInterleaving enabled. Some collectors may do their > own interleaving in the future (JDK-8210473 / JEP-345) so this would > massively interfere in how they work. (That issue may be because I am > looking at a potentially incomplete diff, so forgive me if the patch > already handles this). > Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect when java externally invoked in interleave mode. It helps membind case. > > - if some of the actions (interleaving/membind) fail, and it had been > requested, it would be nice to print a log message. > Following two NUMA API's are used and they return nothing. Right now not sure which cases to handle. Please suggest void numa_tonode_memory(void *start, size_t size, int node); void numa_interleave_memory(void *start, size_t size, struct bitmask *nodemask); > > Actually it would be nice to print information about e.g. the bitmask > anyway in the log so that the user can easily verify that what he > specified on the command line has been properly picked up by the VM - > instead of looking through the proc filesystem. > As per the suggestion, patch is updated to print following information. [0.001s][info][os] UseNUMA is enabled [0.001s][info][os] Java is configured to run in interleave mode [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, 2, 3 > > Thanks, > Thomas > > > > Thanks Amit Pawar -------------- next part -------------- diff -r d537553ed639 src/hotspot/os/linux/os_linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp Wed Nov 28 22:29:35 2018 -0500 +++ b/src/hotspot/os/linux/os_linux.cpp Thu Nov 29 16:56:15 2018 +0530 @@ -2724,6 +2724,8 @@ } void os::numa_make_global(char *addr, size_t bytes) { + if (!UseNUMAInterleaving) + return ; Linux::numa_interleave_memory(addr, bytes); } @@ -2789,6 +2791,16 @@ ids[i++] = node; } } + + // If externally invoked in interleave mode then get node bitmasks from interleave mode pointer. + if (Linux::_numa_interleave_ptr != NULL ) { + i = 0; + for (int node = 0; node <= highest_node_number; node++) { + if (Linux::_numa_bitmask_isbitset(Linux::_numa_interleave_ptr, node)) { + ids[i++] = node; + } + } + } return i; } @@ -2888,11 +2900,18 @@ libnuma_dlsym(handle, "numa_distance"))); set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, libnuma_v2_dlsym(handle, "numa_get_membind"))); + set_numa_get_interleave_mask(CAST_TO_FN_PTR(numa_get_interleave_mask_func_t, + libnuma_v2_dlsym(handle, "numa_get_interleave_mask"))); if (numa_available() != -1) { + struct bitmask *bmp; set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); set_numa_all_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_all_nodes_ptr")); set_numa_nodes_ptr((struct bitmask **)libnuma_dlsym(handle, "numa_nodes_ptr")); + bmp = _numa_get_interleave_mask(); + set_numa_interleave_ptr(&bmp); + bmp = _numa_get_membind(); + set_numa_membind_ptr(&bmp); // Create an index -> node mapping, since nodes are not always consecutive _nindex_to_node = new (ResourceObj::C_HEAP, mtInternal) GrowableArray(0, true); rebuild_nindex_to_node_map(); @@ -3019,9 +3038,12 @@ os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; os::Linux::numa_distance_func_t os::Linux::_numa_distance; os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; +os::Linux::numa_get_interleave_mask_func_t os::Linux::_numa_get_interleave_mask; unsigned long* os::Linux::_numa_all_nodes; struct bitmask* os::Linux::_numa_all_nodes_ptr; struct bitmask* os::Linux::_numa_nodes_ptr; +struct bitmask* os::Linux::_numa_interleave_ptr; +struct bitmask* os::Linux::_numa_membind_ptr; bool os::pd_uncommit_memory(char* addr, size_t size) { uintptr_t res = (uintptr_t) ::mmap(addr, size, PROT_NONE, @@ -5005,13 +5027,55 @@ if (!Linux::libnuma_init()) { UseNUMA = false; } else { - if ((Linux::numa_max_node() < 1) || Linux::isbound_to_single_node()) { - // If there's only one node (they start from 0) or if the process - // is bound explicitly to a single node using membind, disable NUMA. - UseNUMA = false; + + // Identify whether running in membind or interleave mode. + struct bitmask *bmp; + bool _is_membind = false; + bool _is_interleaved = false; + char _buf[BUFSIZ] = {'\0'}; + char *_bufptr = _buf; + + log_info(os)("UseNUMA is enabled"); + // Check for membind mode. + bmp = Linux::_numa_membind_ptr; + for (int node = 0; node <= Linux::numa_max_node() ; node++) { + if (Linux::_numa_bitmask_isbitset(bmp, node)) { + _is_membind = true; + } } + + // Check for interleave mode. + bmp = Linux::_numa_interleave_ptr; + for (int node = 0; node <= Linux::numa_max_node() ; node++) { + if (Linux::_numa_bitmask_isbitset(bmp, node)) { + _is_interleaved = true; + // Set membind to false as interleave mode allows all nodes to be used. + _is_membind = false; + } + } + + if (_is_membind) { + bmp = Linux::_numa_membind_ptr; + Linux::set_numa_interleave_ptr (NULL); + log_info(os) (" Java is configured to run in membind mode"); + } + + if (_is_interleaved) { + bmp = Linux::_numa_interleave_ptr; + Linux::set_numa_membind_ptr (NULL); + log_info(os) (" Java is configured to run in interleave mode"); + } + + for (int node = 0; node <= Linux::numa_max_node() ; node++) { + if (Linux::_numa_bitmask_isbitset(bmp, node)) { + _bufptr += sprintf (_bufptr, "%d, ", node); + } + } + _bufptr[-2 ] = '\0'; + log_info(os) (" Heap will be configured using NUMA memory nodes: %s", _buf); } + if (UseParallelGC && UseNUMA && UseLargePages && !can_commit_large_page_memory()) { // With SHM and HugeTLBFS large pages we cannot uncommit a page, so there's no way // we can make the adaptive lgrp chunk resizing work. If the user specified both diff -r d537553ed639 src/hotspot/os/linux/os_linux.hpp --- a/src/hotspot/os/linux/os_linux.hpp Wed Nov 28 22:29:35 2018 -0500 +++ b/src/hotspot/os/linux/os_linux.hpp Thu Nov 29 16:56:15 2018 +0530 @@ -222,6 +222,7 @@ typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); typedef struct bitmask* (*numa_get_membind_func_t)(void); + typedef struct bitmask* (*numa_get_interleave_mask_func_t)(void); typedef void (*numa_set_bind_policy_func_t)(int policy); typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); @@ -239,9 +240,12 @@ static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; static numa_distance_func_t _numa_distance; static numa_get_membind_func_t _numa_get_membind; + static numa_get_interleave_mask_func_t _numa_get_interleave_mask; static unsigned long* _numa_all_nodes; static struct bitmask* _numa_all_nodes_ptr; static struct bitmask* _numa_nodes_ptr; + static struct bitmask* _numa_interleave_ptr; + static struct bitmask* _numa_membind_ptr; static void set_sched_getcpu(sched_getcpu_func_t func) { _sched_getcpu = func; } static void set_numa_node_to_cpus(numa_node_to_cpus_func_t func) { _numa_node_to_cpus = func; } @@ -255,9 +259,12 @@ static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } + static void set_numa_get_interleave_mask(numa_get_interleave_mask_func_t func) { _numa_get_interleave_mask = func; } static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } + static void set_numa_interleave_ptr(struct bitmask **ptr) { _numa_interleave_ptr = (ptr == NULL ? NULL : *ptr); } + static void set_numa_membind_ptr(struct bitmask **ptr) { _numa_membind_ptr = (ptr == NULL ? NULL : *ptr); } static int sched_getcpu_syscall(void); public: static int sched_getcpu() { return _sched_getcpu != NULL ? _sched_getcpu() : -1; } @@ -275,7 +282,11 @@ static void numa_interleave_memory(void *start, size_t size) { // Use v2 api if available if (_numa_interleave_memory_v2 != NULL && _numa_all_nodes_ptr != NULL) { - _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr); + if (_numa_interleave_ptr != NULL) + // If externally invoked in interleave mode then use interleave bitmask. + _numa_interleave_memory_v2(start, size, _numa_interleave_ptr); + else + _numa_interleave_memory_v2(start, size, _numa_membind_ptr); } else if (_numa_interleave_memory != NULL && _numa_all_nodes != NULL) { _numa_interleave_memory(start, size, _numa_all_nodes); } From rkennke at redhat.com Thu Nov 29 11:12:11 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Nov 2018 12:12:11 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <1264c130-6bf1-f71f-8811-a2bf716245af@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <1264c130-6bf1-f71f-8811-a2bf716245af@oracle.com> Message-ID: Thanks, Magnus, this makes sense. I'll incorporate that simplification. Thanks, Roman Am 29.11.18 um 11:53 schrieb Magnus Ihse Bursie: > On 2018-11-26 23:47, Erik Joelsson wrote: > >> Build changes look ok to me. > I agree, but I'd like to point out a (possible) style improvement. In > hotspot.m4, > > if test "x$OPENJDK_TARGET_CPU" = "xx86_64" || test > "x$OPENJDK_TARGET_CPU" = "xaarch64" || test "x$OPENJDK_TARGET_CPU" == > "xx86"; then > > can be more succinctly expressed as > > if test "x$OPENJDK_TARGET_CPU_ARCH" = "xx86" || test > "x$OPENJDK_TARGET_CPU" = "xaarch64" ; then > > > But it's a matter of preference, if you do not want to change. Otherwise > it looks good! > > /Magnus >> >> /Erik >> >> On 2018-11-26 13:39, Roman Kennke wrote: >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the >>> mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ?? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ??*) shenandoah-gc >>> >>> >>> ???? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ??*) shared-gc >>> >>> ???? - This is mostly boilerplate that is common to any GC >>> ???? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >>> >>> ??*) shared-runtime >>> >>> >>> ???? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ?????? e.g. ClassLoaderData, CodeCache, etc >>> ???? - Warning and disabling JFR LeakProfiler >>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ???? - Locks initialization in mutexLocker.cpp as usual >>> ???? - VM operations defines for Shenandoah's VM ops >>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ???? - The usual macros in macro.hpp >>> >>> ??*) shared-build >>> >>> >>> ???? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ???? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ?????? useful to get the whole marking loop inlined (observed >>> significant >>> regression if we >>> ?????? don't) >>> >>> ??*) shared-tests >>> >>> >>> ???? - Test infrastructure to support Shenandoah >>> ???? - Shenandoah test groups >>> ???? - Exclude Shenandoah in various tests that can be run with >>> selected GC >>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ??*) shenandoah-tests >>> >>> >>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>> subdirectory >>> ???? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ?? - {Windows} x {x86_64}, >>> ?? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ?? - the new Shenandoah tests >>> ?? - jcstress with/without aggressive Shenandoah verification >>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> > From rkennke at redhat.com Thu Nov 29 11:12:39 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Nov 2018 12:12:39 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <87720eda-7239-ba21-b93d-674e6cee1f7e@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <38e57ae2-dce8-6d79-b172-302899312ff9@oracle.com> <87720eda-7239-ba21-b93d-674e6cee1f7e@oracle.com> Message-ID: <55f17364-b32f-4f59-6546-fc403d118a3b@redhat.com> Thanks, Per! I will do this. Check out round 3 when it arrives. ;-) Roman Am 29.11.18 um 12:07 schrieb Per Liden: > Small comment on the build changes. Instead of checking for zero variant > here: > > +? # Only enable Shenandoah on supported arches > +? AC_MSG_CHECKING([if shenandoah can be built]) > +? if HOTSPOT_CHECK_JVM_VARIANT(zero); then > +??? DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES shenandoahgc" > +??? AC_MSG_RESULT([no, this JVM variant not supported]) > +? else > > I think you should just add shanandoahgc to the existing check, here: > > ? # Disable unsupported GCs for Zero > ? if HOTSPOT_CHECK_JVM_VARIANT(zero); then > ??? DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES epsilongc g1gc zgc" > ? fi > > cheers, > Per > > On 11/26/18 11:47 PM, Erik Joelsson wrote: >> Build changes look ok to me. >> >> /Erik >> >> On 2018-11-26 13:39, Roman Kennke wrote: >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the >>> mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ?? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ??*) shenandoah-gc >>> >>> >>> ???? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ??*) shared-gc >>> >>> ???? - This is mostly boilerplate that is common to any GC >>> ???? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >>> >>> ??*) shared-runtime >>> >>> >>> ???? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ?????? e.g. ClassLoaderData, CodeCache, etc >>> ???? - Warning and disabling JFR LeakProfiler >>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ???? - Locks initialization in mutexLocker.cpp as usual >>> ???? - VM operations defines for Shenandoah's VM ops >>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ???? - The usual macros in macro.hpp >>> >>> ??*) shared-build >>> >>> >>> ???? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ???? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ?????? useful to get the whole marking loop inlined (observed >>> significant >>> regression if we >>> ?????? don't) >>> >>> ??*) shared-tests >>> >>> >>> ???? - Test infrastructure to support Shenandoah >>> ???? - Shenandoah test groups >>> ???? - Exclude Shenandoah in various tests that can be run with >>> selected GC >>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ??*) shenandoah-tests >>> >>> >>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>> subdirectory >>> ???? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ?? - {Windows} x {x86_64}, >>> ?? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ?? - the new Shenandoah tests >>> ?? - jcstress with/without aggressive Shenandoah verification >>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> From per.liden at oracle.com Thu Nov 29 14:44:20 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 29 Nov 2018 15:44:20 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Message-ID: Hi Roman, On 11/28/18 11:29 PM, Roman Kennke wrote: [...] > Per: Disabling the SA part for heapdumping makes 2 tests fail: > - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java > - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java > > we filter them for Shenandoah now. I'm wondering: how do you get past > those with ZGC? Good catch! These don't work with ZGC either. As far as I can tell, these tests are not part of any test tier, so they are never executed in our CI pipeline and have gone under the radar. I'll file a bug and add the appropriate @require tag to exclude ZGC. I'm not sure if there's a reason they are excluded from our CI, or if they should be added. Leonid, do you know? Thanks! /Per From thomas.stuefe at gmail.com Thu Nov 29 15:20:54 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 29 Nov 2018 16:20:54 +0100 Subject: ShowRegistersOnAssert on by default Message-ID: Hi all, short note that we enabled ShowRegistersOnAssert (https://bugs.openjdk.java.net/browse/JDK-8191101). We did add this feature some months ago and ran our nightlies since then with it enabled locally. Our compiler folks liked to have this on by default so on it is. I do not expect any problems, but should something go wrong you would see a false segfault at the place where an assertion would normally happen. Should you see this, please open a bug report and put it on my name. Note that for now, this only affects Linux (and not zero). Should there be any interest and I have the time I will extend it to other Posix platforms too. Thanks Thomas From per.liden at oracle.com Thu Nov 29 15:36:58 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 29 Nov 2018 16:36:58 +0100 Subject: RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Message-ID: <840e1174-cb61-3c27-c5f0-bf2bf3336192@oracle.com> Oh, I filed a bug [1] and sent out a fix for review, before noticing Jini's mail. [1] https://bugs.openjdk.java.net/browse/JDK-8214484 /Per On 11/29/18 5:39 AM, Jini George wrote: > Hi Roman, > > The SA tests which iterate over the heap (ClhsdbJhisto.java, > TestHeapDumpFor*.java) fail if ZGC is passed as an option to these also > (Am planning on fixing these). It might be better to avoid invoking > those tests for Shenandoah now with an @requires tag. > > Thank you, > Jini > > On 11/29/2018 3:59 AM, Roman Kennke wrote: >> Alright, we fixed: >> - The minor issues that Kim reported in shared-gc >> - A lot of fixes in shared-tests according to Leonid's review >> - Disabled SA heapdumping similar to ZGC as Per suggested >> >> Some notes: >> Leonid:? test/hotspot/jtreg/gc/TestFullGCCount.java was actually >> correct. The @requires there means to exclude runs with both CMS and >> ExplicitGCInvokesConcurrent at the same time, because that would be >> (expectedly) failing. It can run CMS, default GC and any other GC just >> fine. Adding the same clause for Shenandoah means the same, and filters >> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >> made the condition a bit clearer by avoiding triple-negation. >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >> >> >> Per: Disabling the SA part for heapdumping makes 2 tests fail: >> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >> >> we filter them for Shenandoah now. I'm wondering: how do you get past >> those with ZGC? >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >> >> >> (Note to Leonid and tests reviewers: I'll add those related filters in >> next round). >> >> Vladimir: Roland integrated a bunch of changes to make loop* code look >> better. I can tell that we're not done with C2 yet. Can you look over >> the code and see what is ok, and especially what is not ok, so that we >> can focus our efforts on the relevant parts? >> >> Updated set of webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >> >> Thanks, >> Roman >> >> >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the >>> mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ?? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ??https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ?? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ??*) shenandoah-gc >>> >>> >>> ???? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ??*) shared-gc >>> >>> ???? - This is mostly boilerplate that is common to any GC >>> ???? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ???? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ??*) shared-serviceability >>> >>> >>> ???? - The usual code to support another GC >>> >>> ??*) shared-runtime >>> >>> >>> ???? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ?????? e.g. ClassLoaderData, CodeCache, etc >>> ???? - Warning and disabling JFR LeakProfiler >>> ???? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ???? - Locks initialization in mutexLocker.cpp as usual >>> ???? - VM operations defines for Shenandoah's VM ops >>> ???? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ???? - The usual macros in macro.hpp >>> >>> ??*) shared-build >>> >>> >>> ???? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ???? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ?????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ???? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ?????? useful to get the whole marking loop inlined (observed >>> significant >>> regression if we >>> ?????? don't) >>> >>> ??*) shared-tests >>> >>> >>> ???? - Test infrastructure to support Shenandoah >>> ???? - Shenandoah test groups >>> ???? - Exclude Shenandoah in various tests that can be run with >>> selected GC >>> ???? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ??*) shenandoah-tests >>> >>> >>> ???? - Shenandoah specific tests, most reside in gc/shenandoah >>> subdirectory >>> ???? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ?? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ?? - {Windows} x {x86_64}, >>> ?? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ?? - the new Shenandoah tests >>> ?? - jcstress with/without aggressive Shenandoah verification >>> ?? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> >> From ioi.lam at oracle.com Thu Nov 29 17:58:33 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 29 Nov 2018 09:58:33 -0800 Subject: What's the best way to append an oop to a GrowableArray? In-Reply-To: <49ca3441-7cf2-ba30-7cb5-107441f9ea53@oracle.com> References: <49ca3441-7cf2-ba30-7cb5-107441f9ea53@oracle.com> Message-ID: <9fe5756d-8db9-d3c6-a6d5-ae7f7f2f669b@oracle.com> As suggested to me offline by Dean, the right way to do this is with a GrowableArray :-) Thanks - Ioi On 11/28/18 9:56 PM, Ioi Lam wrote: > I have a memory model question: > > I need to create lots of objects and append them into a GrowableArray > > ?? GrowableArray* arr = new (ResourceObj::C_HEAP, mtInternal) > ??? ??? GrowableArray(10000, true);; > > ?? void doit() { > ??? ?? while (...) { > ??? ?????? oop o = allocate_from_heap(.....); > ??? ?????? arr->append(o); > ??? ??? } > ?? } > > There is only a single thread that can append to the array, so I am > not using any locks. > > However, GC could happen inside this loop, so I have written a > do_oops() function that would scan the array during GC. (For > convenience, I am calling my_oops_do() from within > SystemDictionary:oops_do() for now ...). > > ?? void my_oops_do(OopClosure* f) { > ??? ??? int len = arr->length(); > ??? ??? for (int i = 0; i < len; i++) { > ??? ????? f->do_oop(arr->adr_at(i)); > ??? ??? } > ??? ? } > ?? } > > This seems to work fine on my dual socket Xeon, but will it work > everywhere? Do I need to do anything to ensure that the GC threads can > read the up-to-date values of the array elements, and the array length? > > Thanks > - Ioi > > From vladimir.kozlov at oracle.com Thu Nov 29 18:02:49 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 29 Nov 2018 10:02:49 -0800 Subject: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 In-Reply-To: References: <1a76811e-a056-02df-58e6-dcc13603abdf@oracle.com> <56039258-c077-e488-ac3b-f5eaee56777f@oracle.com> <0793836a-383b-e3c2-d044-26876da1d645@oracle.com> Message-ID: <754b4943-559f-31d7-c564-a1103ac6d50d@oracle.com> Thank you, Yang, for running testing. No more comments from me. Vladimir On 11/28/18 10:39 PM, Yang Zhang (Arm Technology China) wrote: > Hi Vladimir > > I run all the jtreg test(hotspot/langtools/jdk, compiler/c2/cr634086 is included). They are passed. > > Regards > Yang > > -----Original Message----- > From: Vladimir Kozlov > Sent: Thursday, November 29, 2018 2:13 AM > To: Yang Zhang (Arm Technology China) ; Andrew Haley ; hotspot-dev at openjdk.java.net > Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with MaxVectorSize=8 > > Looks good. > > What testing you did? > Unfortunately compiler/c2/cr634086 test is not part of tier1 testing and not executed by 'submit' repo testing. You need to run it. > > Thanks, > Vladimir > > On 11/28/18 12:32 AM, Yang Zhang (Arm Technology China) wrote: >> Hi Andrew, Vladimir >> >> I update the reviewers and remove the new assert in Matcher::vector_shift_count_ideal_reg(). Could you please help review again? >> >> http://cr.openjdk.java.net/~yzhang/8213134/webrev.02/ >> >> Regards >> Yang >> >> -----Original Message----- >> From: Vladimir Kozlov >> Sent: Thursday, November 22, 2018 1:56 AM >> To: Yang Zhang (Arm Technology China) ; >> hotspot-dev at openjdk.java.net >> Subject: Re: RFR(M): 8213134: AArch64: vector shift failed with >> MaxVectorSize=8 >> >> You are right. I think I was confused that you are not limiting MaxVectorSize but simple use min(16, MaxVectorSize). >> >> The assert is fine. >> >> Thanks, >> Vladimir >> >> On 11/20/18 10:28 PM, Yang Zhang (Arm Technology China) wrote: >>> >>>> I don't see any checking or setting for MaxVectorSize in vm_version_aarch64.cpp. >>>> In such case you will definitely hit your new assert Matcher::vector_shift_count_ideal_reg(). >>>> aarch64.ad code Matcher::vector_width_in_bytes() has limit vector size to 16 bytes. So using MaxVectorSize > 16 will hit the assert. >>> >>> Matcher::vector_width_in_bytes() limits vector size to min(16, MaxVectorSize). >>> Matcher::vector_shift_count_ideal_reg() assert(MaxVectorSize >= size, >>> "Length isn't supported"); When MaxVectorSize is 8, size is 8. >>> When MaxVectorSize is greater than or equal 16, size is 16. >>> So (MaxVectorSize >= size) is always true. >>> >>> Test cases of cr6340864 are passed with or without this new assert. >>> >>> Do I need to remove this new assert? >>> >>> Regards >>> Yang >>> >>> >>> On 11/18/18 11:53 PM, Yang Zhang (Arm Technology China) wrote: >>>> Hi, >>>> >>>> When I implemented AArch64 NEON for Vector API (http://openjdk.java.net/jeps/338), I found an issue about vector shift. I have a patch which could fix this issue. Could anyone help to review this patch? >>>> >>>> Webrev: http://cr.openjdk.java.net/~yzhang/8213134/webrev.01/ >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8213134 >>>> >>>> This patch is verified both in jdk/jdk master and panama/vectorIntrinsics, and tests are passed. >>>> Hotspt jtreg contains the following test cases that can cover all the patterns of vector shift. But only default option "-XX:MaxVectorSize=64" is tested, so this issue isn't detected. Now I add "-XX:MaxVectorSize=8", "-XX:MaxVectorSize=16" and "-XX:MaxVectorSize=32" to these jtreg tests in this patch. >>>> compiler/c2/cr6340864/TestByteVect.java >>>> compiler/c2/cr6340864/TestIntVect.java >>>> compiler/c2/cr6340864/TestShortVect.java >>>> compiler/codegen/TestCharVect2.java >>>> >>>> Regards >>>> Yang >>>> >>>> >>> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. >>> >> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. >> > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. > From rkennke at redhat.com Thu Nov 29 20:41:27 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 29 Nov 2018 21:41:27 +0100 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> Message-ID: <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> Hello all, Thanks so far for all the reviews and support! I forgot to update the 'round' yesterday. We are in round 3 now :-) Also, I fixed the numbering of my webrevs to match the review-round. ;-) Things we've changed today: - We moved shenandoah-specific code out of .ad files into our own .ad files under gc/shenandoah (in shenandoah-gc), how cool is that? This requires an addition in build machinery though, see make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). - Improved zero-disabling and build-code-simplification as suggested by Magnus and Per - Cleaned up some leftovers in C2 - Improved C2 loop opts code by introducing another APIs in BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. - We would all very much prefer to keep ShenandoahXYZNode names, as noted earlier. This stuff is Shenandoah-specific, so let's just call it that. - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) - Rebased on jdk-12+22 - Question: let us know if you need separate RFE for the new BSC2 APIs? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ Thanks, Roman > Alright, we fixed: > - The minor issues that Kim reported in shared-gc > - A lot of fixes in shared-tests according to Leonid's review > - Disabled SA heapdumping similar to ZGC as Per suggested > > Some notes: > Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually > correct. The @requires there means to exclude runs with both CMS and > ExplicitGCInvokesConcurrent at the same time, because that would be > (expectedly) failing. It can run CMS, default GC and any other GC just > fine. Adding the same clause for Shenandoah means the same, and filters > the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I > made the condition a bit clearer by avoiding triple-negation. > > See: > http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html > > Per: Disabling the SA part for heapdumping makes 2 tests fail: > - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java > - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java > > we filter them for Shenandoah now. I'm wondering: how do you get past > those with ZGC? > > See: > http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html > > (Note to Leonid and tests reviewers: I'll add those related filters in > next round). > > Vladimir: Roland integrated a bunch of changes to make loop* code look > better. I can tell that we're not done with C2 yet. Can you look over > the code and see what is ok, and especially what is not ok, so that we > can focus our efforts on the relevant parts? > > Updated set of webrevs: > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ > > Thanks, > Roman > > >> Hi, >> >> This is the first round of changes for including Shenandoah GC into >> mainline. >> I divided the review into parts that roughly correspond to the mailing lists >> that would normally review it, and I divided it into 'shared' code >> changes and >> 'shenandoah' code changes (actually, mostly additions). The intend is to >> eventually >> push them as single 'combined' changeset, once reviewed. >> >> JEP: >> ? https://openjdk.java.net/jeps/189 >> Bug entry: >> >> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >> >> Webrevs: >> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >> >> For those who want to see the full change, have a look at the >> shenandoah-complete >> >> directory, >> it contains the full combined webrev. Alternatively, there is the file >> shenandoah-master.patch >> , >> which is what I intend to commit (and which should be equivalent to the >> 'shenandoah-complete' webrev). >> >> Sections to review (at this point) are the following: >> ?*) shenandoah-gc >> >> ??? - Actual Shenandoah implementation, almost completely residing in >> gc/shenandoah >> >> ?*) shared-gc >> >> ??? - This is mostly boilerplate that is common to any GC >> ??? - referenceProcessor.cpp has a little change to make one assert not >> fail (next to CMS and G1) >> ??? - taskqueue.hpp has some small adjustments to enable subclassing >> >> ?*) shared-serviceability >> >> ??? - The usual code to support another GC >> >> ?*) shared-runtime >> >> ??? - A number of friends declarations to allow Shenandoah iterators to >> hook up with, >> ????? e.g. ClassLoaderData, CodeCache, etc >> ??? - Warning and disabling JFR LeakProfiler >> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >> Shenandoah C2 optimizations >> ??? - Locks initialization in mutexLocker.cpp as usual >> ??? - VM operations defines for Shenandoah's VM ops >> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >> Shenandoah's logging >> ??? - The usual macros in macro.hpp >> >> ?*) shared-build >> >> ??? - Add shenandoah feature, enabled by default, as agreed with >> Vladimir K. beforehand >> ??? - Some flags for shenandoah-enabled compilation to get >> SUPPORT_BARRIER_ON_PRIMITIVES >> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >> Shenandoah's barriers >> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >> files, which is >> ????? useful to get the whole marking loop inlined (observed significant >> regression if we >> ????? don't) >> >> ?*) shared-tests >> >> ??? - Test infrastructure to support Shenandoah >> ??? - Shenandoah test groups >> ??? - Exclude Shenandoah in various tests that can be run with selected GC >> ??? - Enable/add configure for Shenandoah for tests that make sense to >> run with it >> >> ?*) shenandoah-tests >> >> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >> ??? - A couple of tests configurations have been added, e.g. >> TestGCBasherWithShenandoah.java >> >> I intentionally left out shared-compiler for now, because we have some >> work left to do >> there, but if you click around you'll find the patch anyway, in case you >> want to take >> a peek at it. >> >> We have regular builds on: >> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >> ? - {Windows} x {x86_64}, >> ? - {MacOS X} x {x86_64} >> >> This also routinely passes: >> ? - the new Shenandoah tests >> ? - jcstress with/without aggressive Shenandoah verification >> ? - specjvm2008 with/without aggressive Shenandoah verification >> >> >> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >> the credit for being the original inventor of Shenandoah, Aleksey >> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >> contributions, everybody else in Red Hat's OpenJDK team for testing, >> advice and support, my collegues in Oracle's GC, runtime and compiler >> teams for tirelessly helping with and reviewing all the GC interface and >> related changes, and of course the many early adopters for reporting >> bugs and success stories and feature requests: we wouldn't be here >> without any of you! >> >> Best regards, >> Roman >> > From per.liden at oracle.com Fri Nov 30 08:56:16 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 30 Nov 2018 09:56:16 +0100 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> Message-ID: Hi Roman, On 11/29/18 9:41 PM, Roman Kennke wrote: [...] > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ Some comments (I didn't look at the compiler part): src/hotspot/share/gc/shared/barrierSetConfig.hpp src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp ------------------------------------------------------- Please insert the Shenandoah lines so that the lists remain alphabetically sorted on GC names. src/hotspot/share/gc/shared/gcCause.cpp --------------------------------------- Add the Shenandoah stuff after _dcmd_gc_run to match the order in the header file. src/hotspot/share/gc/shared/gcConfig.cpp ---------------------------------------- Please insert the Shenandoah lines so that the lists remain alphabetically sorted on GC names. Only the last change is in the right place now. src/hotspot/share/gc/shared/gcTrace.hpp --------------------------------------- Please move this new class to a gc/shenandoah/shanandoahTrace.hpp file (or something similar). src/hotspot/share/gc/shared/gc_globals.hpp src/hotspot/share/gc/shared/vmStructs_gc.hpp ------------------------------------------ Please insert the INCLUDE_SHENANDOAHGC/SHENANDOAHGC_ONLY stuff so that the lists remain alphabetically sorted on GC names. src/hotspot/share/utilities/globalDefinitions.hpp ------------------------------------------------- I think you want to call the new macro UINT64_FORMAT_X_W, to maintain similarity to it's sibling UINT64_FORMAT_X. Also please adjust the indentation/alignment so the list of macros remains nicely structured. src/hotspot/share/utilities/macros.hpp -------------------------------------- Please insert the Shenandoah lines so that the lists remain alphabetically sorted on GC names. src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeap.java src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeapRegion.java ------------------------------------------------------------------------------- Since the heap walking is now disabled, it looks like there's quite a bit of code here that can be removed. test/hotspot/jtreg/* -------------------- As Aleksey and I discussed in a separate thread. Using 'vm.gc.Shenandoah' does not mean the same thing as 'vm.gc == "Shenandoah"', and you typically want to use 'vm.gc == "Shenandoah"' in most cases (I did't check all), like in tests that aren't Shenandoah specific. src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp ------------------------------------------------------ I don't think you want to expose the following flags. Setting any of them to false will crash the VM, right? If you really want to keep them I'd suggest you make them develop-flags. 346 diagnostic(bool, ShenandoahSATBBarrier, true, \ 347 "Turn on/off SATB barriers in Shenandoah") \ 348 \ 349 diagnostic(bool, ShenandoahKeepAliveBarrier, true, \ 350 "Turn on/off keep alive barriers in Shenandoah") \ 351 \ 352 diagnostic(bool, ShenandoahWriteBarrier, true, \ 353 "Turn on/off write barriers in Shenandoah") \ 354 \ 355 diagnostic(bool, ShenandoahReadBarrier, true, \ 356 "Turn on/off read barriers in Shenandoah") \ 357 \ 358 diagnostic(bool, ShenandoahStoreValEnqueueBarrier, false, \ 359 "Turn on/off enqueuing of oops for storeval barriers") \ 360 \ 361 diagnostic(bool, ShenandoahStoreValReadBarrier, true, \ 362 "Turn on/off store val read barriers in Shenandoah") \ 363 \ 364 diagnostic(bool, ShenandoahCASBarrier, true, \ 365 "Turn on/off CAS barriers in Shenandoah") \ 366 \ 367 diagnostic(bool, ShenandoahAcmpBarrier, true, \ 368 "Turn on/off acmp barriers in Shenandoah") \ 369 \ 370 diagnostic(bool, ShenandoahCloneBarrier, true, \ 371 "Turn on/off clone barriers in Shenandoah") \ cheers, Per From rkennke at redhat.com Fri Nov 30 09:09:14 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 30 Nov 2018 10:09:14 +0100 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> Message-ID: <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> Hi Per, Thanks for taking time to look at this! We will take care of all the issues you mentioned. Regarding the flags, I think they are useful as they are now. Shenandoah can be run in 'passive' mode, which doesn't involve any concurrent GC (iow, it runs kindof like Parallel GC). In this mode we can selectively turn on/off different kinds of barriers. This is useful: - for debugging. if we see a crash and we suspect it's caused by a certain type of barrier, we can turn on/off those barriers to narrow down - for performance experiments: passive w/o any barriers give a good baseline against which we can measure impact of types of barriers. Debugging may or may not be useful in develop mode (some bugs only show up in product build), performance work quite definitely is not useful in develop builds, and therefore I think it makes sense to keep them as diagnostic, because that is what they are: diagnostic flags. Makes sense? Thanks, Roman > On 11/29/18 9:41 PM, Roman Kennke wrote: > [...] >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ > > Some comments (I didn't look at the compiler part): > > > src/hotspot/share/gc/shared/barrierSetConfig.hpp > src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp > ------------------------------------------------------- > Please insert the Shenandoah lines so that the lists remain > alphabetically sorted on GC names. > > > src/hotspot/share/gc/shared/gcCause.cpp > --------------------------------------- > Add the Shenandoah stuff after _dcmd_gc_run to match the order in the > header file. > > > src/hotspot/share/gc/shared/gcConfig.cpp > ---------------------------------------- > Please insert the Shenandoah lines so that the lists remain > alphabetically sorted on GC names. Only the last change is in the right > place now. > > > src/hotspot/share/gc/shared/gcTrace.hpp > --------------------------------------- > Please move this new class to a gc/shenandoah/shanandoahTrace.hpp file > (or something similar). > > > src/hotspot/share/gc/shared/gc_globals.hpp > src/hotspot/share/gc/shared/vmStructs_gc.hpp > ------------------------------------------ > Please insert the INCLUDE_SHENANDOAHGC/SHENANDOAHGC_ONLY stuff so that > the lists remain alphabetically sorted on GC names. > > > src/hotspot/share/utilities/globalDefinitions.hpp > ------------------------------------------------- > I think you want to call the new macro UINT64_FORMAT_X_W, to maintain > similarity to it's sibling UINT64_FORMAT_X. Also please adjust the > indentation/alignment so the list of macros remains nicely structured. > > > src/hotspot/share/utilities/macros.hpp > -------------------------------------- > Please insert the Shenandoah lines so that the lists remain > alphabetically sorted on GC names. > > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeap.java > > src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeapRegion.java > > ------------------------------------------------------------------------------- > > Since the heap walking is now disabled, it looks like there's quite a > bit of code here that can be removed. > > > test/hotspot/jtreg/* > -------------------- > As Aleksey and I discussed in a separate thread. Using > 'vm.gc.Shenandoah' does not mean the same thing as 'vm.gc == > "Shenandoah"', and you typically want to use 'vm.gc == "Shenandoah"' in > most cases (I did't check all), like in tests that aren't Shenandoah > specific. > > > src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp > ------------------------------------------------------ > I don't think you want to expose the following flags. Setting any of > them to false will crash the VM, right? If you really want to keep them > I'd suggest you make them develop-flags. > > ?346?? diagnostic(bool, ShenandoahSATBBarrier, true, ???????? \ > ?347?????????? "Turn on/off SATB barriers in Shenandoah") ???????? \ > ?348 ???????? \ > ?349?? diagnostic(bool, ShenandoahKeepAliveBarrier, true, ???????? \ > ?350?????????? "Turn on/off keep alive barriers in Shenandoah") ???????? \ > ?351 ???????? \ > ?352?? diagnostic(bool, ShenandoahWriteBarrier, true, ???????? \ > ?353?????????? "Turn on/off write barriers in Shenandoah") ???????? \ > ?354 ???????? \ > ?355?? diagnostic(bool, ShenandoahReadBarrier, true, ???????? \ > ?356?????????? "Turn on/off read barriers in Shenandoah") ???????? \ > ?357 ???????? \ > ?358?? diagnostic(bool, ShenandoahStoreValEnqueueBarrier, false, ???????? \ > ?359?????????? "Turn on/off enqueuing of oops for storeval barriers") > ???????? \ > ?360 ???????? \ > ?361?? diagnostic(bool, ShenandoahStoreValReadBarrier, true, ???????? \ > ?362?????????? "Turn on/off store val read barriers in Shenandoah") > ???????? \ > ?363 ???????? \ > ?364?? diagnostic(bool, ShenandoahCASBarrier, true, ???????? \ > ?365?????????? "Turn on/off CAS barriers in Shenandoah") ???????? \ > ?366 ???????? \ > ?367?? diagnostic(bool, ShenandoahAcmpBarrier, true, ???????? \ > ?368?????????? "Turn on/off acmp barriers in Shenandoah") ???????? \ > ?369 ???????? \ > ?370?? diagnostic(bool, ShenandoahCloneBarrier, true, ???????? \ > ?371?????????? "Turn on/off clone barriers in Shenandoah") ???????? \ > > > cheers, > Per From per.liden at oracle.com Fri Nov 30 09:34:19 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 30 Nov 2018 10:34:19 +0100 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> Message-ID: Hi Roman, On 11/30/18 10:09 AM, Roman Kennke wrote: > Hi Per, > > Thanks for taking time to look at this! > > We will take care of all the issues you mentioned. > > Regarding the flags, I think they are useful as they are now. Shenandoah > can be run in 'passive' mode, which doesn't involve any concurrent GC > (iow, it runs kindof like Parallel GC). In this mode we can selectively > turn on/off different kinds of barriers. This is useful: > - for debugging. if we see a crash and we suspect it's caused by a > certain type of barrier, we can turn on/off those barriers to narrow down > - for performance experiments: passive w/o any barriers give a good > baseline against which we can measure impact of types of barriers. > > Debugging may or may not be useful in develop mode (some bugs only show > up in product build), performance work quite definitely is not useful in > develop builds, and therefore I think it makes sense to keep them as > diagnostic, because that is what they are: diagnostic flags. > > Makes sense? I can see how these flags can be useful. But I think you might be in violating-spec-territory here, if you're allowing users to turn on flags that crash the VM. If they are safe to use in 'passive' mode, then I think there should be code in shenandoahArguments.cpp that rejects/corrects flag combinations that are unstable. I think the correct way to think about this is: We should pass the TCK, regardless of which features are enabled/disabled (unless the VM barfs at start-up because the chosen combination of flags are incompatible or isn't supported in the current environment, etc). CC:ing Mikael here, who might be able to shed some light on what's ok and not ok here. cheers, Per > > Thanks, > Roman > > >> On 11/29/18 9:41 PM, Roman Kennke wrote: >> [...] >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Some comments (I didn't look at the compiler part): >> >> >> src/hotspot/share/gc/shared/barrierSetConfig.hpp >> src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp >> ------------------------------------------------------- >> Please insert the Shenandoah lines so that the lists remain >> alphabetically sorted on GC names. >> >> >> src/hotspot/share/gc/shared/gcCause.cpp >> --------------------------------------- >> Add the Shenandoah stuff after _dcmd_gc_run to match the order in the >> header file. >> >> >> src/hotspot/share/gc/shared/gcConfig.cpp >> ---------------------------------------- >> Please insert the Shenandoah lines so that the lists remain >> alphabetically sorted on GC names. Only the last change is in the right >> place now. >> >> >> src/hotspot/share/gc/shared/gcTrace.hpp >> --------------------------------------- >> Please move this new class to a gc/shenandoah/shanandoahTrace.hpp file >> (or something similar). >> >> >> src/hotspot/share/gc/shared/gc_globals.hpp >> src/hotspot/share/gc/shared/vmStructs_gc.hpp >> ------------------------------------------ >> Please insert the INCLUDE_SHENANDOAHGC/SHENANDOAHGC_ONLY stuff so that >> the lists remain alphabetically sorted on GC names. >> >> >> src/hotspot/share/utilities/globalDefinitions.hpp >> ------------------------------------------------- >> I think you want to call the new macro UINT64_FORMAT_X_W, to maintain >> similarity to it's sibling UINT64_FORMAT_X. Also please adjust the >> indentation/alignment so the list of macros remains nicely structured. >> >> >> src/hotspot/share/utilities/macros.hpp >> -------------------------------------- >> Please insert the Shenandoah lines so that the lists remain >> alphabetically sorted on GC names. >> >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeap.java >> >> src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeapRegion.java >> >> ------------------------------------------------------------------------------- >> >> Since the heap walking is now disabled, it looks like there's quite a >> bit of code here that can be removed. >> >> >> test/hotspot/jtreg/* >> -------------------- >> As Aleksey and I discussed in a separate thread. Using >> 'vm.gc.Shenandoah' does not mean the same thing as 'vm.gc == >> "Shenandoah"', and you typically want to use 'vm.gc == "Shenandoah"' in >> most cases (I did't check all), like in tests that aren't Shenandoah >> specific. >> >> >> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp >> ------------------------------------------------------ >> I don't think you want to expose the following flags. Setting any of >> them to false will crash the VM, right? If you really want to keep them >> I'd suggest you make them develop-flags. >> >> ?346?? diagnostic(bool, ShenandoahSATBBarrier, true, ???????? \ >> ?347?????????? "Turn on/off SATB barriers in Shenandoah") ???????? \ >> ?348 ???????? \ >> ?349?? diagnostic(bool, ShenandoahKeepAliveBarrier, true, ???????? \ >> ?350?????????? "Turn on/off keep alive barriers in Shenandoah") ???????? \ >> ?351 ???????? \ >> ?352?? diagnostic(bool, ShenandoahWriteBarrier, true, ???????? \ >> ?353?????????? "Turn on/off write barriers in Shenandoah") ???????? \ >> ?354 ???????? \ >> ?355?? diagnostic(bool, ShenandoahReadBarrier, true, ???????? \ >> ?356?????????? "Turn on/off read barriers in Shenandoah") ???????? \ >> ?357 ???????? \ >> ?358?? diagnostic(bool, ShenandoahStoreValEnqueueBarrier, false, ???????? \ >> ?359?????????? "Turn on/off enqueuing of oops for storeval barriers") >> ???????? \ >> ?360 ???????? \ >> ?361?? diagnostic(bool, ShenandoahStoreValReadBarrier, true, ???????? \ >> ?362?????????? "Turn on/off store val read barriers in Shenandoah") >> ???????? \ >> ?363 ???????? \ >> ?364?? diagnostic(bool, ShenandoahCASBarrier, true, ???????? \ >> ?365?????????? "Turn on/off CAS barriers in Shenandoah") ???????? \ >> ?366 ???????? \ >> ?367?? diagnostic(bool, ShenandoahAcmpBarrier, true, ???????? \ >> ?368?????????? "Turn on/off acmp barriers in Shenandoah") ???????? \ >> ?369 ???????? \ >> ?370?? diagnostic(bool, ShenandoahCloneBarrier, true, ???????? \ >> ?371?????????? "Turn on/off clone barriers in Shenandoah") ???????? \ >> >> >> cheers, >> Per > From erik.osterlund at oracle.com Fri Nov 30 09:50:22 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 30 Nov 2018 10:50:22 +0100 Subject: RFR: 8213209: [REDO] Allow Klass::_subklass and _next_sibling to have unloaded classes In-Reply-To: References: <5BF2B1F2.8070004@oracle.com> Message-ID: <11c36871-fe98-1cbc-f86b-cb5044bb2cec@oracle.com> Hi Dean, Thank you for the review! /Erik On 2018-11-27 07:11, dean.long at oracle.com wrote: > Looks OK to me too.? The lazy initialization of _has_subklass probably > isn't needed anymore, but it does seem to be an improvement. > > dl > > > On 11/19/18 4:52 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thanks for the review. >> >> Full webrev: >> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.01/ >> >> Incremental webrev: >> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00_01/ >> >> On 2018-11-16 22:08, coleen.phillimore at oracle.com wrote: >>> >>> Erik, thank you for taking this change and making it lock-free. >>> >>> On 11/16/18 1:12 PM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> This is a redo of Coleen's earlier patch to clean weak metadata >>>> links using the Compile_lock. >>>> >>>> We noticed the proposed solution could cause deadlocks. At least one >>>> of the reasons for that is that the Compile_lock was taken during >>>> printing and verification in safepoints. And the Compile_lock is >>>> sometimes held while issuing safepoints. The latter problem also >>>> forces us to take the Compile_lock before joining the suspendible >>>> thread set, to avoid another deadlock situation. But that is really >>>> nasty for other reasons. And there have been a few more problems as >>>> well. >>>> >>>> So in this case, we think it is both safer, less error prone, and >>>> better performing (although less important) to perform this cleaning >>>> in a lock-free fashion rather than trying to dodge all the issues >>>> related to the locking protocol. >>>> >>>> Inserts still require a lock. Therefore, we are dealing with >>>> lock-free reads, and lock-free cleaning. The lock-free cleaning is >>>> interacting with multiple lock-free readers and a single writer. >>>> >>>> The single inserter always helps with cleaning the subclass list >>>> head before prepending new nodes to the chain. That allows an >>>> invariant that the siblings link is never inserted pointing at a >>>> Klass that is unloading, which simplifies things a lot. The head is >>>> inserted in a classic load + CAS in a loop. >>>> >>>> Readers require load_acquire when reading the head, due to competing >>>> inserts putting new Klass pointers there. The siblings, however, >>>> only need relaxed consistency, because they are linked to data >>>> strictly older than the head, which has already been acquired. >>>> >>>> Unlinked entries are all inserted into a purge list, because freeing >>>> them immediately is not safe. A global handshake operation is >>>> performed, and after that ClassLoaderDataGraph::purge() will go >>>> through the list and delete the entries safely. >>> >>> I don't see the purge list in your change, and it's not needed since >>> this is unlinking in place.? I think this is your other change you're >>> thinking of. >> >> Yes you are right, of course. It's the unloading list and not a >> separate purge list... >> >>> Should the Compile_lock be removed in ciInstanceKlass.cpp >>> ciInstanceKlass* ciInstanceKlass::implementor() { >>> >>> and in jvmciCompilerToVM.cpp >>> C2V_VMENTRY(jobject, getImplementor, (JNIEnv *, jobject, jobject >>> jvmci_type)) >>> >>> The former does a bit more than calling implementor().? Or do you >>> want to leave these locks to be conservative? >> >> While I think your analysis is correct, I would indeed like to leave >> them in this RFE to be conservative. >> >>> ? JVMCIKlassHandle handle(THREAD); >>> ? { >>> ??? // Need Compile_lock around implementor() >>> ??? MutexLocker locker(Compile_lock); >>> ??? handle = iklass->implementor(); >>> ? } >>> >>> >>> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/oops/instanceKlass.cpp.udiff.html >>> >>> >>> nof_implementors() has an assert_locked_or_safepoint(Compile_lock); >>> which it no longer needs.? You'll hit this from printing because >>> you've removed the locking here: >>> >>> 3093?? if (is_interface()) { >>> 3094 MutexLocker ml(Compile_lock); >>> 3095???? st->print_cr(BULLET"nof implementors:? %d", >>> nof_implementors()); >>> 3096???? if (nof_implementors() == 1) { >>> 3097?????? st->print_cr(BULLET"implementor:??? "); >>> 3098?????? st->print("?? "); >>> 3099?????? implementor()->print_value_on(st); >>> 3100?????? st->cr(); >>> 3101???? } >>> >>> >>> We don't test printing very well. >>> >> >> Ahh yes - good catch. I removed the assert as it is no longer required. >> >>> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/src/hotspot/share/utilities/vmError.cpp.udiff.html >>> >>> >>> This file is unchanged except a line removed. >> >> I put it back the way it was before. >> >>> This change looks good.? I've run through the tests with it and >>> didn't find any problems that doomed my earlier version. >> >> Thank you for running additional tests on this patch! >> >> Thanks, >> /Erik >> >>> Thanks!! >>> Coleen >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8213209/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8213209 >>>> >>>> Thanks, >>>> /Erik >>> >> > From erik.osterlund at oracle.com Fri Nov 30 09:50:07 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 30 Nov 2018 10:50:07 +0100 Subject: RFR: 8214257: IC cache not clean after cleaning assertion failure In-Reply-To: <39e41533-4bd5-fb5a-4a9f-cc305953a367@oracle.com> References: <39e41533-4bd5-fb5a-4a9f-cc305953a367@oracle.com> Message-ID: <3618d283-90b0-963c-f137-713dbcdae0f8@oracle.com> Hi Tobias, Thanks for the review! /Erik On 2018-11-29 09:07, Tobias Hartmann wrote: > Hi Erik, > > looks good to me. Please run at least hs-tier1,hs-tier2,hs-precheckin-comp before pushing. > > Best regards, > Tobias > > On 27.11.18 14:00, Erik ?sterlund wrote: >> Hi, >> >> Back in 8212681, an abstract CompiledICLocker was introduce to protect inline caches from concurrent >> patching. >> >> Unfortunately, not all patching of ICs started using the CompiledICLocker. There were still some >> callsites using the Patching_lock. Therefore, the assert may trigger sometimes saying the IC cache >> is not clean right after cleaning, due to other concurrent modifications under the Patching_lock. >> >> Also, in CompiledIC::set_to_clean(), the condition for safe_transition was altered incorrectly from >> is_at_safepoint() to CompiledICLocker::is_safe(), causing cleaning to not used transition, so I >> reinstated that in this patch. >> >> This patch depends on 8214338 by assuming that the CompiledICLocker does not use safepoint checks, >> as it is now used from leaf calls where safepointing is not allowed. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8214257 >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8214257/webrev.00/ >> >> Thanks, >> /Erik From rkennke at redhat.com Fri Nov 30 09:55:26 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 30 Nov 2018 10:55:26 +0100 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> Message-ID: <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> >> Hi Per, >> >> Thanks for taking time to look at this! >> >> We will take care of all the issues you mentioned. >> >> Regarding the flags, I think they are useful as they are now. Shenandoah >> can be run in 'passive' mode, which doesn't involve any concurrent GC >> (iow, it runs kindof like Parallel GC). In this mode we can selectively >> turn on/off different kinds of barriers. This is useful: >> - for debugging. if we see a crash and we suspect it's caused by a >> certain type of barrier, we can turn on/off those barriers to narrow down >> - for performance experiments: passive w/o any barriers give a good >> baseline against which we can measure impact of types of barriers. >> >> Debugging may or may not be useful in develop mode (some bugs only show >> up in product build), performance work quite definitely is not useful in >> develop builds, and therefore I think it makes sense to keep them as >> diagnostic, because that is what they are: diagnostic flags. >> >> Makes sense? > > I can see how these flags can be useful. But I think you might be in > violating-spec-territory here, if you're allowing users to turn on flags > that crash the VM. If they are safe to use in 'passive' mode, then I > think there should be code in shenandoahArguments.cpp that > rejects/corrects flag combinations that are unstable. Let us see if we can do that. If you think this violates the spec, then please point to the part that says diagnostic (!!) options must pass the TCK and/or not crash the JVM? I mean, it's called diagnostic for a reason. It seems counter-useful to bury diagnostic flags that we would be using for diagnostics. > I think the correct way to think about this is: We should pass the TCK, > regardless of which features are enabled/disabled (unless the VM barfs > at start-up because the chosen combination of flags are incompatible or > isn't supported in the current environment, etc). > > CC:ing Mikael here, who might be able to shed some light on what's ok > and not ok here. Yeah, again, where is it said that diagnostic flags must even pass the TCK? That is totally new to me. Thanks, Roman From thomas.schatzl at oracle.com Fri Nov 30 10:11:18 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 30 Nov 2018 11:11:18 +0100 Subject: RFR (S/M): 8213827: NUMA heap allocation does not respect process membind/interleave settings [Was: Re: [PATCH] JDK NUMA Interleaving issue] In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com> Message-ID: <9bea7b0957bbfc2f0ac34306ee162f2d98e44bfe.camel@oracle.com> Hi Amit, On Thu, 2018-11-29 at 16:38 +0530, amith pawar wrote: > Hi Thomas, > > Sorry for the late reply and please check my inline comments below. > > On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl < > thomas.schatzl at oracle.com> wrote: > > Hi Amith, > > > > welcome to the OpenJDK community! :) > > Thanks > > > > On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: > > > Hi all, > > > > > >[...] > > > > Is it possible to summarize the problem and the changes with the > > following few lines: > > [...] > > This is also OK. I updated JDK-8213827. > > - there have been other NUMA related patches from AMD recently, in > > particular JDK-what is the relation to JDK-8205051? The text there > > reads awfully similar to this one, but I *think* JDK-8205051 is > > actually about making sure that the parallel gc eden is not put on > > inactive nodes. > > Can you confirm this (talk to your colleague) so that we can fix > > the description too? > > JDK-8205051 is related to early GC. > JDK-8189922 specific to membind case where heap was allocated on non > requested NUMA nodes. > The proposed patch fixes two following issues > 1. Similar to JDK-8189922 but for interleave case. > 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on > fewer NUMA nodes. Okay. > > - fyi, we are currently working on NUMA aware memory allocation > > support for G1 in JEP 345 (JDK-8210473). It may be useful to sync > > up a bit to not step on each other's toes (see below). > > We are not working anything related to G1. It may effect G1 if > numa_make_global function is called. That's why we need to be a bit careful here. > > [... broken patch...] > > > > I tried to apply the patch to the jdk/jdk tree, which failed; I > > then started to manually apply it but stopped after not being able > > to find the context of some hunks. I do not think this change > > applies to the latest source tree. > > > > Please make sure that the patch applies to the latest jdk/jdk tree > > with errors. All changes generally must first go into the latest > > dev tree before you can apply for backporting. > > > > Could you please send the patch as attachment (not copy&pasted) to > > this list and cc me? Then I can create a webrev out of it. > > > > I did look a bit over the patch as much as I could (it's very hard > > trying to review a diff), some comments: > > Sorry. Please check the attached patch. I created a webrev at http://cr.openjdk.java.net/~tschatzl/8213827/webrev/ . Let the reviews begin :) > > - the _numa_interleave_mmory_v2 function pointer is never > > assigned to in the patch in the CR, so it will not be used. Please > > make sure the patch is complete. > > Actually it is never defined anywhere, ie. the patch unlikely > > actually compiles even if I could apply it somewhere. > > > > Please avoid frustrating reviewers by sending incomplete patches. > > Sorry again. Faced same issue when copied from the defect link but > worked for me from the mail. Now i have attached the patch. Can you > please update defect link with this patch ? This one applies correctly. I will provide review comments in a separate email. Thanks, Thomas From per.liden at oracle.com Fri Nov 30 10:40:13 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 30 Nov 2018 11:40:13 +0100 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> Message-ID: <2828302a-3e42-d35b-864d-1bbadba7a233@oracle.com> Hi, On 11/30/18 10:55 AM, Roman Kennke wrote: >>> Hi Per, >>> >>> Thanks for taking time to look at this! >>> >>> We will take care of all the issues you mentioned. >>> >>> Regarding the flags, I think they are useful as they are now. Shenandoah >>> can be run in 'passive' mode, which doesn't involve any concurrent GC >>> (iow, it runs kindof like Parallel GC). In this mode we can selectively >>> turn on/off different kinds of barriers. This is useful: >>> - for debugging. if we see a crash and we suspect it's caused by a >>> certain type of barrier, we can turn on/off those barriers to narrow down >>> - for performance experiments: passive w/o any barriers give a good >>> baseline against which we can measure impact of types of barriers. >>> >>> Debugging may or may not be useful in develop mode (some bugs only show >>> up in product build), performance work quite definitely is not useful in >>> develop builds, and therefore I think it makes sense to keep them as >>> diagnostic, because that is what they are: diagnostic flags. >>> >>> Makes sense? >> >> I can see how these flags can be useful. But I think you might be in >> violating-spec-territory here, if you're allowing users to turn on flags >> that crash the VM. If they are safe to use in 'passive' mode, then I >> think there should be code in shenandoahArguments.cpp that >> rejects/corrects flag combinations that are unstable. > > Let us see if we can do that. > > If you think this violates the spec, then please point to the part that > says diagnostic (!!) options must pass the TCK and/or not crash the JVM? > I mean, it's called diagnostic for a reason. It seems counter-useful to > bury diagnostic flags that we would be using for diagnostics. > >> I think the correct way to think about this is: We should pass the TCK, >> regardless of which features are enabled/disabled (unless the VM barfs >> at start-up because the chosen combination of flags are incompatible or >> isn't supported in the current environment, etc). >> >> CC:ing Mikael here, who might be able to shed some light on what's ok >> and not ok here. > > Yeah, again, where is it said that diagnostic flags must even pass the > TCK? That is totally new to me. I'm not really the right person to speak authoritatively about this. However, from what I understand it doesn't matter what kind of flag it is. If a user can enable it in a production/shipped build I think it must pass the TCK. But let's have someone who knows more about the rules around that weight in on this. cheers, Per From erik.osterlund at oracle.com Fri Nov 30 11:19:36 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 30 Nov 2018 12:19:36 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> Message-ID: Hi Dean, Thanks for reviewing this! On 2018-11-27 23:55, dean.long at oracle.com wrote: > There's a typo in the new guarantee message: "tranisition". Fixed in-place. > Regarding the infinite loops, can we detect if progress isn't being made > and assert/panic, rather than hanging? I'm afraid not. It's a similar analogy to the problem of progress guarantees of a lock-free algorithm. They have a global progress guarantee for the system, but no local guarantee of progress for each operation, that can in theory be starved indefinitely due to other operations winning every race. The situation is similar here, at least in theory, that the thread refilling the IC stubs didn't get a single stub back, because another thread started depleted the IC stubs again. Naturally, if this was ever to happen, we would get time outs in our tests though, to indicate that there is a problem. Thanks, /Erik > dl > > On 11/27/18 1:29 PM, Erik ?sterlund wrote: >> Hi Dean, >> >> Thank you for reviewing this. >> >> Full webrev: >> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >> >> Incremental webrev: >> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >> >> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>> Hi Erik.? Can you explain why you removed the pre-allocated "next >>> stub"?? I guess it was no longer necessary?? If so, then you should >>> update the "always has sentinel" comment in is_empty. >> >> Updated. Indeed, the sentinel no longer serves a purpose so I removed >> it. Found some more traces of the sentinel that I removed in the last >> webrev. >> >>> In your guarantee messages, some use "should not fail" and some use >>> "should never fail".? It's not a big deal but maybe they should be >>> the same. >> >> Sure. Updated. >> >>> You introduced a couple of infinite loops.? Are we guaranteed to exit >>> these loops eventually?? Is there an upper bound on how many >>> iterations are needed? >> >> In my latest webrev I removed some unnecessary set_to_clean() where IC >> caches are already is_clean(). With that in place, there should be >> global progress guarantees and a single IC stub in the buffer should >> be all you really "need". Although, you might want more. In fact, I'm >> questioning if the 10K size buffer is big enough today, but I'll leave >> that for another day. >> >> Thanks, >> /Erik >> >>> The rest looks good. >>> >>> dl >>> >>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> IC stub refilling currently requires a safepoint operation. When >>>> done right at the point where an CompiledIC is about to get patched >>>> to a transitional state using an IC stub, locks may already be held, >>>> causing a bunch of locking issues - especially for concurrent class >>>> unloading. Therefore, the IC stub refilling ought to be moved out so >>>> that IC cache cleaning and transitioning may be done without any >>>> safepoints, and the locks in the path ought to not perform safepoint >>>> checking. >>>> >>>> This is implemented by allowing IC transitions to fail when they >>>> require IC stubs, and we run out of them. This propages back to a >>>> higher level where IC stubs are refilled after having released the >>>> IC locker. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>> >>>> Thanks, >>>> /Erik >>> > From hohensee at amazon.com Fri Nov 30 14:28:34 2018 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 30 Nov 2018 14:28:34 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> Message-ID: My bad on the definition of "intrinsic". I was thinking of "intrinsic" as a method for which C2 generates specialized IR which would then be subject to optimization (as in library_call.cpp), not hand-coded asm (as in stubGenerator_aarch64.cpp). I don't know how feasible it is to switch from the latter to the former, but imo it would be good to do if we could. Maybe a hybrid approach with the loop structure in IR and specialized operators that emit optimized code sequences for stuff the compile can't figure out. But that's not an immediate fix. On the maximum time an intrinsic should block, Roland pointed at https://bugs.openjdk.java.net/browse/JDK-8186027. Maybe use the same approach for hand-coded asm, i.e, insert safepoint polls into the hand-coded asm based on LoopStripMiningIter. Or, are the intrinsics completely separate native methods? If so, your approach sounds good to me. Thanks, Paul ?On 11/27/18, 2:42 AM, "Andrew Haley" wrote: On 11/26/18 10:01 PM, Hohensee, Paul wrote: > The compiler could estimate the cost of an iteration using some rough pessimistic heuristic and strip mine based on that. But the compiler doesn't see the intrinsics: they are hand-coded assembly language. > Measuring the safepoint time directly is fragile due to inability to find all the hardware platforms that might run the JVM. A671 Sure, but all I'm suggesting is that we take an order-of-magnitude guess and apply that to our intrinsics. It does not have to be accurate: in the case of encryption the speed difference caused by limiting the number of iterations is infinitesimal, but the time to safepoint difference is huge. IMO, this is a true no brainer. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rkennke at redhat.com Fri Nov 30 15:34:04 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 30 Nov 2018 16:34:04 +0100 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> Message-ID: > My bad on the definition of "intrinsic". > > I was thinking of "intrinsic" as a method for which C2 generates specialized IR which would then be subject to optimization (as in library_call.cpp), not hand-coded asm (as in stubGenerator_aarch64.cpp). I don't know how feasible it is to switch from the latter to the former, but imo it would be good to do if we could. Maybe a hybrid approach with the loop structure in IR and specialized operators that emit optimized code sequences for stuff the compile can't figure out. But that's not an immediate fix. > > On the maximum time an intrinsic should block, Roland pointed at https://bugs.openjdk.java.net/browse/JDK-8186027. Maybe use the same approach for hand-coded asm, i.e, insert safepoint polls into the hand-coded asm based on LoopStripMiningIter. > > Or, are the intrinsics completely separate native methods? If so, your approach sounds good to me. I assume we are talking about things like asm loops for crypto. In this case, only carefully placed safepoint polls in asm would help, right? But then we need oopmaps at those intrinsics. This seems far away from trival :-) Roman From coleen.phillimore at oracle.com Fri Nov 30 15:37:41 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 30 Nov 2018 10:37:41 -0500 Subject: RFR: 8214231: Allow concurrent cleaning of TypeStackSlotEntries and ReturnTypeEntry In-Reply-To: <0dee808b-dfee-500d-ec57-ef96ce314fa3@oracle.com> References: <5BF6B641.60302@oracle.com> <0dee808b-dfee-500d-ec57-ef96ce314fa3@oracle.com> Message-ID: <14866718-ab89-45ec-f453-ced38e439b80@oracle.com> This looks good to me also. Thanks, Coleen On 11/23/18 7:45 AM, Robbin Ehn wrote: > Hi Erik, > > This is a reasonable approach, thanks! > > /Robbin > > On 2018-11-22 14:59, Erik ?sterlund wrote: >> Hi, >> >> The TypeStackSlotEntries and ReturnTypeEntry contain weak metadata >> links that must be cleaned during class unloading. With concurrent >> class unloading, their ci counterparts must be protected from >> accessing unloading metadata. >> >> In particular, ciTypeStackSlotEntries and ciReturnTypeEntry >> counterparts copy the raw data from TypeStackSlotEntries and >> ReturnTypeEntry of the MDOs, and are subsequently queried by the >> compiler. The copied data may contain stale metadata. By checking for >> unloading metadata and replacing it with NULL during creation of the >> ci variants from their raw counterparts, we protect the compiler from >> seeing stale metadata that is being unloaded, before the GC has >> gotten around to cleaning up the MDOs. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8214231/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8214231 >> >> Thanks, >> /Erik From erik.osterlund at oracle.com Fri Nov 30 15:47:16 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 30 Nov 2018 16:47:16 +0100 Subject: RFR: 8214231: Allow concurrent cleaning of TypeStackSlotEntries and ReturnTypeEntry In-Reply-To: <14866718-ab89-45ec-f453-ced38e439b80@oracle.com> References: <5BF6B641.60302@oracle.com> <0dee808b-dfee-500d-ec57-ef96ce314fa3@oracle.com> <14866718-ab89-45ec-f453-ced38e439b80@oracle.com> Message-ID: Hi Coleen, Thanks for the review! /Erik On 2018-11-30 16:37, coleen.phillimore at oracle.com wrote: > > This looks good to me also. > Thanks, > Coleen > > On 11/23/18 7:45 AM, Robbin Ehn wrote: >> Hi Erik, >> >> This is a reasonable approach, thanks! >> >> /Robbin >> >> On 2018-11-22 14:59, Erik ?sterlund wrote: >>> Hi, >>> >>> The TypeStackSlotEntries and ReturnTypeEntry contain weak metadata >>> links that must be cleaned during class unloading. With concurrent >>> class unloading, their ci counterparts must be protected from >>> accessing unloading metadata. >>> >>> In particular, ciTypeStackSlotEntries and ciReturnTypeEntry >>> counterparts copy the raw data from TypeStackSlotEntries and >>> ReturnTypeEntry of the MDOs, and are subsequently queried by the >>> compiler. The copied data may contain stale metadata. By checking for >>> unloading metadata and replacing it with NULL during creation of the >>> ci variants from their raw counterparts, we protect the compiler from >>> seeing stale metadata that is being unloaded, before the GC has >>> gotten around to cleaning up the MDOs. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8214231/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8214231 >>> >>> Thanks, >>> /Erik > From rwestrel at redhat.com Fri Nov 30 16:05:00 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 30 Nov 2018 17:05:00 +0100 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> Message-ID: <87lg5ak0ur.fsf@redhat.com> > We could simply limit the maximum buffer size of the intrinsic and > wrap it in a Java loop with (as usual) a safepoint check. This does > not noticeably slow anything down, but it does solve the TTSP > problem. This can be important when multi-megabyte messages are being > encrypted. "a Java loop": so that would be implemented by changing the library code, not the JVM code? Is the intend to have one safepoint per iteration of that loop? I think we would need to verify this interacts correctly with loop strip mining so that loop doesn't get strip mined if the body is something that's very long. As you say, sounds like a no brainer to me too. Roland. From aph at redhat.com Fri Nov 30 16:10:09 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 30 Nov 2018 16:10:09 +0000 Subject: Problems with AES-GCM native acceleration In-Reply-To: References: <92a16009b9e047eacf102b006bc0612bac4a2cfb.camel@redhat.com> <1b1fe9e4-ad71-7d48-13eb-48dbe267962e@oracle.com> <864a763d-0d67-d894-234c-fbbef78e0320@oracle.com> Message-ID: On 11/21/18 5:37 PM, Andrew Haley wrote: > On 11/15/18 10:42 AM, Gidon Gershinsky wrote: >> Having the decryption optimized in the HotSpot engine would be ideal. > > I agree with you. I did a few experiments, and it can take a very > long time for C2 compilation to kick in, especially because GCM is > glacially slow until the intrinsics are used. > > I think this would be a generally useful enhancement to HotSpot, > and I'm kicking around an experimental patch which adds the > intrinsics to c1 and the interpreter. There's a proof-of-concept patch at http://cr.openjdk.java.net/~aph/gctr/ It's all rather hacky but it works. The patch is rather more complicated than I would have liked. We could simplify it somewhat by getting rid of the C1 intrinsic, and instead making C1 call the interpreter implementation. There also a jmh benchmark in that directory. Test results for 1Mb look like this: Interp: Benchmark Mode Cnt Score Error Units AESGCMUpdateAAD2.test avgt 5 1426.275 ? 8.778 us/op C1: Benchmark Mode Cnt Score Error Units AESGCMUpdateAAD2.test avgt 5 1359.367 ? 8.196 us/op C2: Benchmark Mode Cnt Score Error Units AESGCMUpdateAAD2.test avgt 5 1333.863 ? 18.385 us/op -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri Nov 30 16:26:10 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 30 Nov 2018 16:26:10 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: <87lg5ak0ur.fsf@redhat.com> References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <87lg5ak0ur.fsf@redhat.com> Message-ID: <3d287d09-9a32-d0ac-3b2b-9e69d015778c@redhat.com> On 11/30/18 4:05 PM, Roland Westrelin wrote: > >> We could simply limit the maximum buffer size of the intrinsic and >> wrap it in a Java loop with (as usual) a safepoint check. This does >> not noticeably slow anything down, but it does solve the TTSP >> problem. This can be important when multi-megabyte messages are being >> encrypted. > > "a Java loop": so that would be implemented by changing the library > code, not the JVM code? Exactly, yes. We just wrap the (single) call site to the intrinsic in an outer loop. > Is the intend to have one safepoint per iteration of that loop? I think > we would need to verify this interacts correctly with loop strip mining > so that loop doesn't get strip mined if the body is something that's > very long. OK, that should be doable. > As you say, sounds like a no brainer to me too. Thanks for the reply. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From aph at redhat.com Fri Nov 30 16:39:02 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 30 Nov 2018 16:39:02 +0000 Subject: Time to Safepoint and HotSpot intrinsics In-Reply-To: References: <2f3c3cb1-26a4-0921-d24c-1a662c158a75@redhat.com> <39C48A7A-7CDC-472E-8370-3A2CCEE2C7BE@amazon.com> Message-ID: <8c5da4d8-bbec-ee55-3c02-94e261fa8977@redhat.com> On 11/30/18 2:28 PM, Hohensee, Paul wrote: > My bad on the definition of "intrinsic". > > I was thinking of "intrinsic" as a method for which C2 generates specialized IR which would then be subject to optimization (as in library_call.cpp), not hand-coded asm (as in stubGenerator_aarch64.cpp). It's very much the latter. > I don't know how feasible it is to switch from the latter to the former, but imo it would be good to do if we could. Pretty near impossible for the crypto stuff, which relies on magic hand-unrolled software pipelining for maximum performance. > On the maximum time an intrinsic should block, Roland pointed at https://bugs.openjdk.java.net/browse/JDK-8186027. Maybe use the same approach for hand-coded asm, i.e, insert safepoint polls into the hand-coded asm based on LoopStripMiningIter. It'd be hard. We'd need to generate oop maps in the middle of the intrinsics. > Or, are the intrinsics completely separate native methods? If so, your approach sounds good to me. They just call out to the stubs. As a matter of maintainability I believe we should do as much as possible in Java rather than add even more hand-coded stuff. Apart from anything else it'd mean that we wouldn't have to fix this in N different back ends. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From dean.long at oracle.com Fri Nov 30 18:41:32 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 30 Nov 2018 10:41:32 -0800 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> Message-ID: <942c9ffc-9206-2b1d-5f17-363b7080f721@oracle.com> On 11/30/18 3:19 AM, Erik ?sterlund wrote: > Hi Dean, > > Thanks for reviewing this! > > On 2018-11-27 23:55, dean.long at oracle.com wrote: >> There's a typo in the new guarantee message: "tranisition". > > Fixed in-place. > >> Regarding the infinite loops, can we detect if progress isn't being >> made and assert/panic, rather than hanging? > > I'm afraid not. It's a similar analogy to the problem of progress > guarantees of a lock-free algorithm. They have a global progress > guarantee for the system, but no local guarantee of progress for each > operation, that can in theory be starved indefinitely due to other > operations winning every race. The situation is similar here, at least > in theory, that the thread refilling the IC stubs didn't get a single > stub back, because another thread started depleted the IC stubs again. > Naturally, if this was ever to happen, we would get time outs in our > tests though, to indicate that there is a problem. > Is it correct to say that the loop was there before (in new_ic_stub), but now you've moved it into the callers? dl > Thanks, > /Erik > >> dl >> >> On 11/27/18 1:29 PM, Erik ?sterlund wrote: >>> Hi Dean, >>> >>> Thank you for reviewing this. >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >>> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >>> >>> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>>> Hi Erik.? Can you explain why you removed the pre-allocated "next >>>> stub"?? I guess it was no longer necessary?? If so, then you should >>>> update the "always has sentinel" comment in is_empty. >>> >>> Updated. Indeed, the sentinel no longer serves a purpose so I >>> removed it. Found some more traces of the sentinel that I removed in >>> the last webrev. >>> >>>> In your guarantee messages, some use "should not fail" and some use >>>> "should never fail".? It's not a big deal but maybe they should be >>>> the same. >>> >>> Sure. Updated. >>> >>>> You introduced a couple of infinite loops.? Are we guaranteed to >>>> exit these loops eventually? Is there an upper bound on how many >>>> iterations are needed? >>> >>> In my latest webrev I removed some unnecessary set_to_clean() where >>> IC caches are already is_clean(). With that in place, there should >>> be global progress guarantees and a single IC stub in the buffer >>> should be all you really "need". Although, you might want more. In >>> fact, I'm questioning if the 10K size buffer is big enough today, >>> but I'll leave that for another day. >>> >>> Thanks, >>> /Erik >>> >>>> The rest looks good. >>>> >>>> dl >>>> >>>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> IC stub refilling currently requires a safepoint operation. When >>>>> done right at the point where an CompiledIC is about to get >>>>> patched to a transitional state using an IC stub, locks may >>>>> already be held, causing a bunch of locking issues - especially >>>>> for concurrent class unloading. Therefore, the IC stub refilling >>>>> ought to be moved out so that IC cache cleaning and transitioning >>>>> may be done without any safepoints, and the locks in the path >>>>> ought to not perform safepoint checking. >>>>> >>>>> This is implemented by allowing IC transitions to fail when they >>>>> require IC stubs, and we run out of them. This propages back to a >>>>> higher level where IC stubs are refilled after having released the >>>>> IC locker. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>>> >>>>> Thanks, >>>>> /Erik >>>> >> From erik.osterlund at oracle.com Fri Nov 30 18:47:00 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 30 Nov 2018 19:47:00 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <942c9ffc-9206-2b1d-5f17-363b7080f721@oracle.com> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> <942c9ffc-9206-2b1d-5f17-363b7080f721@oracle.com> Message-ID: Hi Dean, > On 30 Nov 2018, at 19:41, dean.long at oracle.com wrote: > >> On 11/30/18 3:19 AM, Erik ?sterlund wrote: >> Hi Dean, >> >> Thanks for reviewing this! >> >>> On 2018-11-27 23:55, dean.long at oracle.com wrote: >>> There's a typo in the new guarantee message: "tranisition". >> >> Fixed in-place. >> >>> Regarding the infinite loops, can we detect if progress isn't being made and assert/panic, rather than hanging? >> >> I'm afraid not. It's a similar analogy to the problem of progress guarantees of a lock-free algorithm. They have a global progress guarantee for the system, but no local guarantee of progress for each operation, that can in theory be starved indefinitely due to other operations winning every race. The situation is similar here, at least in theory, that the thread refilling the IC stubs didn't get a single stub back, because another thread started depleted the IC stubs again. Naturally, if this was ever to happen, we would get time outs in our tests though, to indicate that there is a problem. >> > > Is it correct to say that the loop was there before (in new_ic_stub), but now you've moved it into the callers? Yes, that is exactly right. /Erik > dl > >> Thanks, >> /Erik >> >>> dl >>> >>>> On 11/27/18 1:29 PM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>> Thank you for reviewing this. >>>> >>>> Full webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >>>> >>>> Incremental webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >>>> >>>>> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>>>> Hi Erik. Can you explain why you removed the pre-allocated "next stub"? I guess it was no longer necessary? If so, then you should update the "always has sentinel" comment in is_empty. >>>> >>>> Updated. Indeed, the sentinel no longer serves a purpose so I removed it. Found some more traces of the sentinel that I removed in the last webrev. >>>> >>>>> In your guarantee messages, some use "should not fail" and some use "should never fail". It's not a big deal but maybe they should be the same. >>>> >>>> Sure. Updated. >>>> >>>>> You introduced a couple of infinite loops. Are we guaranteed to exit these loops eventually? Is there an upper bound on how many iterations are needed? >>>> >>>> In my latest webrev I removed some unnecessary set_to_clean() where IC caches are already is_clean(). With that in place, there should be global progress guarantees and a single IC stub in the buffer should be all you really "need". Although, you might want more. In fact, I'm questioning if the 10K size buffer is big enough today, but I'll leave that for another day. >>>> >>>> Thanks, >>>> /Erik >>>> >>>>> The rest looks good. >>>>> >>>>> dl >>>>> >>>>>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>>>> Hi, >>>>>> >>>>>> IC stub refilling currently requires a safepoint operation. When done right at the point where an CompiledIC is about to get patched to a transitional state using an IC stub, locks may already be held, causing a bunch of locking issues - especially for concurrent class unloading. Therefore, the IC stub refilling ought to be moved out so that IC cache cleaning and transitioning may be done without any safepoints, and the locks in the path ought to not perform safepoint checking. >>>>>> >>>>>> This is implemented by allowing IC transitions to fail when they require IC stubs, and we run out of them. This propages back to a higher level where IC stubs are refilled after having released the IC locker. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>> >>> > From vladimir.kozlov at oracle.com Fri Nov 30 19:25:35 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 30 Nov 2018 11:25:35 -0800 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> Message-ID: <5eee86f0-84a8-8ba1-bb77-c61680f4a6a1@oracle.com> Can you do simple verification of flags and exit VM gracefully with error message? I think it should be fine. Crashing VM (unintentionally) is definitely not acceptable. Thanks, Vladimir On 11/30/18 1:55 AM, Roman Kennke wrote: >>> Hi Per, >>> >>> Thanks for taking time to look at this! >>> >>> We will take care of all the issues you mentioned. >>> >>> Regarding the flags, I think they are useful as they are now. Shenandoah >>> can be run in 'passive' mode, which doesn't involve any concurrent GC >>> (iow, it runs kindof like Parallel GC). In this mode we can selectively >>> turn on/off different kinds of barriers. This is useful: >>> - for debugging. if we see a crash and we suspect it's caused by a >>> certain type of barrier, we can turn on/off those barriers to narrow down >>> - for performance experiments: passive w/o any barriers give a good >>> baseline against which we can measure impact of types of barriers. >>> >>> Debugging may or may not be useful in develop mode (some bugs only show >>> up in product build), performance work quite definitely is not useful in >>> develop builds, and therefore I think it makes sense to keep them as >>> diagnostic, because that is what they are: diagnostic flags. >>> >>> Makes sense? >> >> I can see how these flags can be useful. But I think you might be in >> violating-spec-territory here, if you're allowing users to turn on flags >> that crash the VM. If they are safe to use in 'passive' mode, then I >> think there should be code in shenandoahArguments.cpp that >> rejects/corrects flag combinations that are unstable. > > Let us see if we can do that. > > If you think this violates the spec, then please point to the part that > says diagnostic (!!) options must pass the TCK and/or not crash the JVM? > I mean, it's called diagnostic for a reason. It seems counter-useful to > bury diagnostic flags that we would be using for diagnostics. > >> I think the correct way to think about this is: We should pass the TCK, >> regardless of which features are enabled/disabled (unless the VM barfs >> at start-up because the chosen combination of flags are incompatible or >> isn't supported in the current environment, etc). >> >> CC:ing Mikael here, who might be able to shed some light on what's ok >> and not ok here. > > Yeah, again, where is it said that diagnostic flags must even pass the > TCK? That is totally new to me. > > Thanks, > Roman > From rkennke at redhat.com Fri Nov 30 19:28:47 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 30 Nov 2018 20:28:47 +0100 Subject: RFR (round 3), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <5eee86f0-84a8-8ba1-bb77-c61680f4a6a1@oracle.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <2b31b813-3178-4712-7a72-b3ec62394c71@redhat.com> <13c3fa3b-dae3-0042-ce7a-27fb7b4ee4ca@redhat.com> <5eee86f0-84a8-8ba1-bb77-c61680f4a6a1@oracle.com> Message-ID: Yes, Aleksey just implemented it. It will appear in round 4 in a bit. Thanks, Roman Am 30. November 2018 20:25:35 MEZ schrieb Vladimir Kozlov : >Can you do simple verification of flags and exit VM gracefully with >error message? > >I think it should be fine. Crashing VM (unintentionally) is definitely >not acceptable. > >Thanks, >Vladimir > >On 11/30/18 1:55 AM, Roman Kennke wrote: >>>> Hi Per, >>>> >>>> Thanks for taking time to look at this! >>>> >>>> We will take care of all the issues you mentioned. >>>> >>>> Regarding the flags, I think they are useful as they are now. >Shenandoah >>>> can be run in 'passive' mode, which doesn't involve any concurrent >GC >>>> (iow, it runs kindof like Parallel GC). In this mode we can >selectively >>>> turn on/off different kinds of barriers. This is useful: >>>> - for debugging. if we see a crash and we suspect it's caused by a >>>> certain type of barrier, we can turn on/off those barriers to >narrow down >>>> - for performance experiments: passive w/o any barriers give a good >>>> baseline against which we can measure impact of types of barriers. >>>> >>>> Debugging may or may not be useful in develop mode (some bugs only >show >>>> up in product build), performance work quite definitely is not >useful in >>>> develop builds, and therefore I think it makes sense to keep them >as >>>> diagnostic, because that is what they are: diagnostic flags. >>>> >>>> Makes sense? >>> >>> I can see how these flags can be useful. But I think you might be in >>> violating-spec-territory here, if you're allowing users to turn on >flags >>> that crash the VM. If they are safe to use in 'passive' mode, then I >>> think there should be code in shenandoahArguments.cpp that >>> rejects/corrects flag combinations that are unstable. >> >> Let us see if we can do that. >> >> If you think this violates the spec, then please point to the part >that >> says diagnostic (!!) options must pass the TCK and/or not crash the >JVM? >> I mean, it's called diagnostic for a reason. It seems counter-useful >to >> bury diagnostic flags that we would be using for diagnostics. >> >>> I think the correct way to think about this is: We should pass the >TCK, >>> regardless of which features are enabled/disabled (unless the VM >barfs >>> at start-up because the chosen combination of flags are incompatible >or >>> isn't supported in the current environment, etc). >>> >>> CC:ing Mikael here, who might be able to shed some light on what's >ok >>> and not ok here. >> >> Yeah, again, where is it said that diagnostic flags must even pass >the >> TCK? That is totally new to me. >> >> Thanks, >> Roman >> From dean.long at oracle.com Fri Nov 30 20:21:47 2018 From: dean.long at oracle.com (dean.long at oracle.com) Date: Fri, 30 Nov 2018 12:21:47 -0800 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> <942c9ffc-9206-2b1d-5f17-363b7080f721@oracle.com> Message-ID: <7a1db048-4133-18b7-e877-6bb0248eff7e@oracle.com> On 11/30/18 10:47 AM, Erik Osterlund wrote: > Hi Dean, > >> On 30 Nov 2018, at 19:41, dean.long at oracle.com wrote: >> >>> On 11/30/18 3:19 AM, Erik ?sterlund wrote: >>> Hi Dean, >>> >>> Thanks for reviewing this! >>> >>>> On 2018-11-27 23:55, dean.long at oracle.com wrote: >>>> There's a typo in the new guarantee message: "tranisition". >>> Fixed in-place. >>> >>>> Regarding the infinite loops, can we detect if progress isn't being made and assert/panic, rather than hanging? >>> I'm afraid not. It's a similar analogy to the problem of progress guarantees of a lock-free algorithm. They have a global progress guarantee for the system, but no local guarantee of progress for each operation, that can in theory be starved indefinitely due to other operations winning every race. The situation is similar here, at least in theory, that the thread refilling the IC stubs didn't get a single stub back, because another thread started depleted the IC stubs again. Naturally, if this was ever to happen, we would get time outs in our tests though, to indicate that there is a problem. >>> >> Is it correct to say that the loop was there before (in new_ic_stub), but now you've moved it into the callers? > Yes, that is exactly right. Sounds good! dl > > /Erik > >> dl >> >>> Thanks, >>> /Erik >>> >>>> dl >>>> >>>>> On 11/27/18 1:29 PM, Erik ?sterlund wrote: >>>>> Hi Dean, >>>>> >>>>> Thank you for reviewing this. >>>>> >>>>> Full webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >>>>> >>>>> Incremental webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >>>>> >>>>>> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>>>>> Hi Erik. Can you explain why you removed the pre-allocated "next stub"? I guess it was no longer necessary? If so, then you should update the "always has sentinel" comment in is_empty. >>>>> Updated. Indeed, the sentinel no longer serves a purpose so I removed it. Found some more traces of the sentinel that I removed in the last webrev. >>>>> >>>>>> In your guarantee messages, some use "should not fail" and some use "should never fail". It's not a big deal but maybe they should be the same. >>>>> Sure. Updated. >>>>> >>>>>> You introduced a couple of infinite loops. Are we guaranteed to exit these loops eventually? Is there an upper bound on how many iterations are needed? >>>>> In my latest webrev I removed some unnecessary set_to_clean() where IC caches are already is_clean(). With that in place, there should be global progress guarantees and a single IC stub in the buffer should be all you really "need". Although, you might want more. In fact, I'm questioning if the 10K size buffer is big enough today, but I'll leave that for another day. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>>> The rest looks good. >>>>>> >>>>>> dl >>>>>> >>>>>>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>>>>> Hi, >>>>>>> >>>>>>> IC stub refilling currently requires a safepoint operation. When done right at the point where an CompiledIC is about to get patched to a transitional state using an IC stub, locks may already be held, causing a bunch of locking issues - especially for concurrent class unloading. Therefore, the IC stub refilling ought to be moved out so that IC cache cleaning and transitioning may be done without any safepoints, and the locks in the path ought to not perform safepoint checking. >>>>>>> >>>>>>> This is implemented by allowing IC transitions to fail when they require IC stubs, and we run out of them. This propages back to a higher level where IC stubs are refilled after having released the IC locker. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik From erik.osterlund at oracle.com Fri Nov 30 20:24:55 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 30 Nov 2018 21:24:55 +0100 Subject: RFR: 8214338: Move IC stub refilling out of IC cache transitions In-Reply-To: <7a1db048-4133-18b7-e877-6bb0248eff7e@oracle.com> References: <200407d5-2fad-ff34-a7e9-ed046ff5788f@oracle.com> <25ef1893-57db-dd09-f012-d6bc677f8808@oracle.com> <267db98d-660e-514e-8ff1-f80c59c26952@oracle.com> <942c9ffc-9206-2b1d-5f17-363b7080f721@oracle.com> <7a1db048-4133-18b7-e877-6bb0248eff7e@oracle.com> Message-ID: <35724B49-97B5-486D-A720-0571B3BD8ADC@oracle.com> Hi Dean, Thanks for the review! /Erik > On 30 Nov 2018, at 21:21, dean.long at oracle.com wrote: > >> On 11/30/18 10:47 AM, Erik Osterlund wrote: >> Hi Dean, >> >>>> On 30 Nov 2018, at 19:41, dean.long at oracle.com wrote: >>>> >>>> On 11/30/18 3:19 AM, Erik ?sterlund wrote: >>>> Hi Dean, >>>> >>>> Thanks for reviewing this! >>>> >>>>> On 2018-11-27 23:55, dean.long at oracle.com wrote: >>>>> There's a typo in the new guarantee message: "tranisition". >>>> Fixed in-place. >>>> >>>>> Regarding the infinite loops, can we detect if progress isn't being made and assert/panic, rather than hanging? >>>> I'm afraid not. It's a similar analogy to the problem of progress guarantees of a lock-free algorithm. They have a global progress guarantee for the system, but no local guarantee of progress for each operation, that can in theory be starved indefinitely due to other operations winning every race. The situation is similar here, at least in theory, that the thread refilling the IC stubs didn't get a single stub back, because another thread started depleted the IC stubs again. Naturally, if this was ever to happen, we would get time outs in our tests though, to indicate that there is a problem. >>>> >>> Is it correct to say that the loop was there before (in new_ic_stub), but now you've moved it into the callers? >> Yes, that is exactly right. > > Sounds good! > > dl > >> >> /Erik >> >>> dl >>> >>>> Thanks, >>>> /Erik >>>> >>>>> dl >>>>> >>>>>> On 11/27/18 1:29 PM, Erik ?sterlund wrote: >>>>>> Hi Dean, >>>>>> >>>>>> Thank you for reviewing this. >>>>>> >>>>>> Full webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.01/ >>>>>> >>>>>> Incremental webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00_01/ >>>>>> >>>>>>> On 2018-11-27 19:07, dean.long at oracle.com wrote: >>>>>>> Hi Erik. Can you explain why you removed the pre-allocated "next stub"? I guess it was no longer necessary? If so, then you should update the "always has sentinel" comment in is_empty. >>>>>> Updated. Indeed, the sentinel no longer serves a purpose so I removed it. Found some more traces of the sentinel that I removed in the last webrev. >>>>>> >>>>>>> In your guarantee messages, some use "should not fail" and some use "should never fail". It's not a big deal but maybe they should be the same. >>>>>> Sure. Updated. >>>>>> >>>>>>> You introduced a couple of infinite loops. Are we guaranteed to exit these loops eventually? Is there an upper bound on how many iterations are needed? >>>>>> In my latest webrev I removed some unnecessary set_to_clean() where IC caches are already is_clean(). With that in place, there should be global progress guarantees and a single IC stub in the buffer should be all you really "need". Although, you might want more. In fact, I'm questioning if the 10K size buffer is big enough today, but I'll leave that for another day. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>>> The rest looks good. >>>>>>> >>>>>>> dl >>>>>>> >>>>>>>> On 11/27/18 5:00 AM, Erik ?sterlund wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> IC stub refilling currently requires a safepoint operation. When done right at the point where an CompiledIC is about to get patched to a transitional state using an IC stub, locks may already be held, causing a bunch of locking issues - especially for concurrent class unloading. Therefore, the IC stub refilling ought to be moved out so that IC cache cleaning and transitioning may be done without any safepoints, and the locks in the path ought to not perform safepoint checking. >>>>>>>> >>>>>>>> This is implemented by allowing IC transitions to fail when they require IC stubs, and we run out of them. This propages back to a higher level where IC stubs are refilled after having released the IC locker. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8214338/webrev.00/ >>>>>>>> >>>>>>>> Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8214338 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik > From rkennke at redhat.com Fri Nov 30 21:00:33 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 30 Nov 2018 22:00:33 +0100 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> Message-ID: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Hi all, here comes round 4 of Shenandoah upstreaming review: This includes fixes for the issues that Per brought up: - Verify and gracefully reject dangerous flags combinations that disables required barriers - Revisited @requires filters in tests - Trim unused code from Shenandoah's SA impl - Move ShenandoahGCTracer to gc/shenandoah - Fix ordering of GC names in various files - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ Thanks everybody for taking time to review this! Roman > Hello all, > > Thanks so far for all the reviews and support! > > I forgot to update the 'round' yesterday. We are in round 3 now :-) > Also, I fixed the numbering of my webrevs to match the review-round. ;-) > > Things we've changed today: > - We moved shenandoah-specific code out of .ad files into our own .ad > files under gc/shenandoah (in shenandoah-gc), how cool is that? This > requires an addition in build machinery though, see > make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). > - Improved zero-disabling and build-code-simplification as suggested by > Magnus and Per > - Cleaned up some leftovers in C2 > - Improved C2 loop opts code by introducing another APIs in > BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. > - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. > - We would all very much prefer to keep ShenandoahXYZNode names, as > noted earlier. This stuff is Shenandoah-specific, so let's just call it > that. > - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) > - Rebased on jdk-12+22 > > - Question: let us know if you need separate RFE for the new BSC2 APIs? > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ > > Thanks, > Roman > >> Alright, we fixed: >> - The minor issues that Kim reported in shared-gc >> - A lot of fixes in shared-tests according to Leonid's review >> - Disabled SA heapdumping similar to ZGC as Per suggested >> >> Some notes: >> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >> correct. The @requires there means to exclude runs with both CMS and >> ExplicitGCInvokesConcurrent at the same time, because that would be >> (expectedly) failing. It can run CMS, default GC and any other GC just >> fine. Adding the same clause for Shenandoah means the same, and filters >> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >> made the condition a bit clearer by avoiding triple-negation. >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >> >> Per: Disabling the SA part for heapdumping makes 2 tests fail: >> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >> >> we filter them for Shenandoah now. I'm wondering: how do you get past >> those with ZGC? >> >> See: >> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >> >> (Note to Leonid and tests reviewers: I'll add those related filters in >> next round). >> >> Vladimir: Roland integrated a bunch of changes to make loop* code look >> better. I can tell that we're not done with C2 yet. Can you look over >> the code and see what is ok, and especially what is not ok, so that we >> can focus our efforts on the relevant parts? >> >> Updated set of webrevs: >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >> >> Thanks, >> Roman >> >> >>> Hi, >>> >>> This is the first round of changes for including Shenandoah GC into >>> mainline. >>> I divided the review into parts that roughly correspond to the mailing lists >>> that would normally review it, and I divided it into 'shared' code >>> changes and >>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>> eventually >>> push them as single 'combined' changeset, once reviewed. >>> >>> JEP: >>> ? https://openjdk.java.net/jeps/189 >>> Bug entry: >>> >>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>> >>> Webrevs: >>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>> >>> For those who want to see the full change, have a look at the >>> shenandoah-complete >>> >>> directory, >>> it contains the full combined webrev. Alternatively, there is the file >>> shenandoah-master.patch >>> , >>> which is what I intend to commit (and which should be equivalent to the >>> 'shenandoah-complete' webrev). >>> >>> Sections to review (at this point) are the following: >>> ?*) shenandoah-gc >>> >>> ??? - Actual Shenandoah implementation, almost completely residing in >>> gc/shenandoah >>> >>> ?*) shared-gc >>> >>> ??? - This is mostly boilerplate that is common to any GC >>> ??? - referenceProcessor.cpp has a little change to make one assert not >>> fail (next to CMS and G1) >>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>> >>> ?*) shared-serviceability >>> >>> ??? - The usual code to support another GC >>> >>> ?*) shared-runtime >>> >>> ??? - A number of friends declarations to allow Shenandoah iterators to >>> hook up with, >>> ????? e.g. ClassLoaderData, CodeCache, etc >>> ??? - Warning and disabling JFR LeakProfiler >>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>> Shenandoah C2 optimizations >>> ??? - Locks initialization in mutexLocker.cpp as usual >>> ??? - VM operations defines for Shenandoah's VM ops >>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>> Shenandoah's logging >>> ??? - The usual macros in macro.hpp >>> >>> ?*) shared-build >>> >>> ??? - Add shenandoah feature, enabled by default, as agreed with >>> Vladimir K. beforehand >>> ??? - Some flags for shenandoah-enabled compilation to get >>> SUPPORT_BARRIER_ON_PRIMITIVES >>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>> Shenandoah's barriers >>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>> files, which is >>> ????? useful to get the whole marking loop inlined (observed significant >>> regression if we >>> ????? don't) >>> >>> ?*) shared-tests >>> >>> ??? - Test infrastructure to support Shenandoah >>> ??? - Shenandoah test groups >>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>> run with it >>> >>> ?*) shenandoah-tests >>> >>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>> ??? - A couple of tests configurations have been added, e.g. >>> TestGCBasherWithShenandoah.java >>> >>> I intentionally left out shared-compiler for now, because we have some >>> work left to do >>> there, but if you click around you'll find the patch anyway, in case you >>> want to take >>> a peek at it. >>> >>> We have regular builds on: >>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>> ? - {Windows} x {x86_64}, >>> ? - {MacOS X} x {x86_64} >>> >>> This also routinely passes: >>> ? - the new Shenandoah tests >>> ? - jcstress with/without aggressive Shenandoah verification >>> ? - specjvm2008 with/without aggressive Shenandoah verification >>> >>> >>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>> the credit for being the original inventor of Shenandoah, Aleksey >>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>> advice and support, my collegues in Oracle's GC, runtime and compiler >>> teams for tirelessly helping with and reviewing all the GC interface and >>> related changes, and of course the many early adopters for reporting >>> bugs and success stories and feature requests: we wouldn't be here >>> without any of you! >>> >>> Best regards, >>> Roman >>> >> > From Derek.White at cavium.com Fri Nov 30 22:40:44 2018 From: Derek.White at cavium.com (White, Derek) Date: Fri, 30 Nov 2018 22:40:44 +0000 Subject: [PATCH] JDK NUMA Interleaving issue In-Reply-To: References: <6e5b102d07b4ceded09115a649be020410240fe7.camel@oracle.com>, Message-ID: Hi Amith, Thanks for tackling this! I think that your patch may fix https://bugs.openjdk.java.net/browse/JDK-8205051, although it's probably fixing more than that. I was thinking that the problem was related to calling " _numa_interleave_memory_v2(start, size, _numa_all_nodes_ptr);", and you replaced that call with more specific node sets, which seems better to me. You can try the test cases listed in the bug to see if the patch fixes the problem. The existing code disables UseNUMA if the process is bound to only one node, but it looks like your patch removes this. Is that a correct read of the patch, intentional, and/or a good idea? Thanks! - Derek ________________________________ From: hotspot-dev on behalf of amith pawar Sent: Thursday, November 29, 2018 6:08 AM To: thomas.schatzl at oracle.com Cc: hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com Subject: Re: [PATCH] JDK NUMA Interleaving issue External Email Hi Thomas, Sorry for the late reply and please check my inline comments below. On Thu, Nov 22, 2018 at 5:18 PM Thomas Schatzl wrote: > Hi Amith, > > welcome to the OpenJDK community! :) > Thanks > > On Fri, 2018-11-09 at 17:53 +0530, amith pawar wrote: > > Hi all, > > > > The flag UseNUMA (or UseNUMAInterleaving), has to interleave old gen, > > S1 and S2 region (if any other ) memory areas on requested Numa nodes > > and it should not configure itself to access other Numa nodes. This > > issue is observed only when Java is allowed to run on fewer NUMA > > nodes than available on the system with numactl membind and > > interleave options. Running on all the nodes does not have any > > effect. This will cause some applications (objects residing in old > > gen and survivor region) to run slower on system with large Numa > > nodes. > > > [... long explanation...] > > Is it possible to summarize the problem and the changes with the > following few lines: > > "NUMA interleaving of memory of old gen and survivor spaces (for > parallel GC) tells the OS to interleave memory across all nodes of a > NUMA system. However the VM process may be configured to be limited to > run only on a few nodes, which means that large parts of the heap will > be located on foreign nodes. This can incurs a large performance > penalty. > > The proposed solution is to tell the OS to interleave memory only > across available nodes when enabling NUMA." > OK. Since, I dont have write access to the defect link so can I request you do the same. > > We have had trouble understanding the problem statement and purpose of > this patch when triaging (making sure the issue is understandable and > can be worked on) as the text is rather technical. Having an easily > understandable text also helps reviewers a lot. > > Assuming my summary is appropriate, I have several other unrelated > questions: > > - could you propose a better subject for this work? "JDK NUMA > Interleaving issue" seems very generic. Something like "NUMA heap > allocation does not respect VM membind/interleave settings" maybe? > This is also OK. > > - there have been other NUMA related patches from AMD recently, in > particular JDK-what is the relation to JDK-8205051? The text there > reads awfully similar to this one, but I *think* JDK-8205051 is > actually about making sure that the parallel gc eden is not put on > inactive nodes. > Can you confirm this (talk to your colleague) so that we can fix the > description too? > JDK-8205051 is related to early GC. JDK-8189922 specific to membind case where heap was allocated on non requested NUMA nodes. The proposed patch fixes two following issues 1. Similar to JDK-8189922 but for interleave case. 2. OLDGen, S1 and S2 memory interleaving is incorrect when run on fewer NUMA nodes. > - fyi, we are currently working on NUMA aware memory allocation support > for G1 in JEP 345 (JDK-8210473). It may be useful to sync up a bit to > not step on each other's toes (see below). > We are not working anything related to G1. It may effect G1 if numa_make_global function is called. > > [... broken patch...] > > I tried to apply the patch to the jdk/jdk tree, which failed; I then > started to manually apply it but stopped after not being able to find > the context of some hunks. I do not think this change applies to the > latest source tree. > > Please make sure that the patch applies to the latest jdk/jdk tree with > errors. All changes generally must first go into the latest dev tree > before you can apply for backporting. > > Could you please send the patch as attachment (not copy&pasted) to this > list and cc me? Then I can create a webrev out of it. > > I did look a bit over the patch as much as I could (it's very hard > trying to review a diff), some comments: > Sorry. Please check the attached patch. > > - the _numa_interleave_memory_v2 function pointer is never assigned > to in the patch in the CR, so it will not be used. Please make sure the > patch is complete. > Actually it is never defined anywhere, ie. the patch unlikely actually > compiles even if I could apply it somewhere. > > Please avoid frustrating reviewers by sending incomplete patches. > Sorry again. Faced same issue when copied from the defect link but worked for me from the mail. Now i have attached the patch. Can you please update defect link with this patch ? > > - I am not sure set_numa_interleave, i.e. the interleaving, should be > done without UseNUMAInterleaving enabled. Some collectors may do their > own interleaving in the future (JDK-8210473 / JEP-345) so this would > massively interfere in how they work. (That issue may be because I am > looking at a potentially incomplete diff, so forgive me if the patch > already handles this). > Testing SPECjbb with UseNUMAInterleaving enabled/disabled has no effect when java externally invoked in interleave mode. It helps membind case. > > - if some of the actions (interleaving/membind) fail, and it had been > requested, it would be nice to print a log message. > Following two NUMA API's are used and they return nothing. Right now not sure which cases to handle. Please suggest void numa_tonode_memory(void *start, size_t size, int node); void numa_interleave_memory(void *start, size_t size, struct bitmask *nodemask); > > Actually it would be nice to print information about e.g. the bitmask > anyway in the log so that the user can easily verify that what he > specified on the command line has been properly picked up by the VM - > instead of looking through the proc filesystem. > As per the suggestion, patch is updated to print following information. [0.001s][info][os] UseNUMA is enabled [0.001s][info][os] Java is configured to run in interleave mode [0.001s][info][os] Heap will be configured using NUMA memory nodes: 0, 2, 3 > > Thanks, > Thomas > > > > Thanks Amit Pawar From coleen.phillimore at oracle.com Fri Nov 30 23:01:03 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 30 Nov 2018 18:01:03 -0500 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi, I looked at the runtime changes, which are very few, since you've upstreamed most of the runtime dependent changes already. They look good to me. Thanks, Coleen On 11/30/18 4:00 PM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>> From coleen.phillimore at oracle.com Fri Nov 30 23:22:47 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 30 Nov 2018 18:22:47 -0500 Subject: RFR (round 4), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector In-Reply-To: <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> References: <9ff4171e-9827-8710-554a-b84da309277a@redhat.com> <914ae8db-c7d6-4d14-541b-f49d670d2ffb@redhat.com> <0f7aa868-5862-47da-9d1c-98ee70036e72@redhat.com> <02c58732-e421-6eb2-d21f-50e56b5d665f@redhat.com> Message-ID: Hi, I looked at the runtime changes, which are very few, since you've upstreamed most of the runtime dependent changes already. They look good to me. Thanks, Coleen (trying again with all the lists). On 11/30/18 4:00 PM, Roman Kennke wrote: > Hi all, > > here comes round 4 of Shenandoah upstreaming review: > > This includes fixes for the issues that Per brought up: > - Verify and gracefully reject dangerous flags combinations that > disables required barriers > - Revisited @requires filters in tests > - Trim unused code from Shenandoah's SA impl > - Move ShenandoahGCTracer to gc/shenandoah > - Fix ordering of GC names in various files > - Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W > > http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/ > > Thanks everybody for taking time to review this! > Roman > >> Hello all, >> >> Thanks so far for all the reviews and support! >> >> I forgot to update the 'round' yesterday. We are in round 3 now :-) >> Also, I fixed the numbering of my webrevs to match the review-round. ;-) >> >> Things we've changed today: >> - We moved shenandoah-specific code out of .ad files into our own .ad >> files under gc/shenandoah (in shenandoah-gc), how cool is that? This >> requires an addition in build machinery though, see >> make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build). >> - Improved zero-disabling and build-code-simplification as suggested by >> Magnus and Per >> - Cleaned up some leftovers in C2 >> - Improved C2 loop opts code by introducing another APIs in >> BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp. >> - I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now. >> - We would all very much prefer to keep ShenandoahXYZNode names, as >> noted earlier. This stuff is Shenandoah-specific, so let's just call it >> that. >> - Rehashed Shenandoah tests (formatting, naming, directory layout, etc) >> - Rebased on jdk-12+22 >> >> - Question: let us know if you need separate RFE for the new BSC2 APIs? >> >> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/ >> >> Thanks, >> Roman >> >>> Alright, we fixed: >>> - The minor issues that Kim reported in shared-gc >>> - A lot of fixes in shared-tests according to Leonid's review >>> - Disabled SA heapdumping similar to ZGC as Per suggested >>> >>> Some notes: >>> Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually >>> correct. The @requires there means to exclude runs with both CMS and >>> ExplicitGCInvokesConcurrent at the same time, because that would be >>> (expectedly) failing. It can run CMS, default GC and any other GC just >>> fine. Adding the same clause for Shenandoah means the same, and filters >>> the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I >>> made the condition a bit clearer by avoiding triple-negation. >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html >>> >>> Per: Disabling the SA part for heapdumping makes 2 tests fail: >>> - test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java >>> - test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java >>> >>> we filter them for Shenandoah now. I'm wondering: how do you get past >>> those with ZGC? >>> >>> See: >>> http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html >>> >>> (Note to Leonid and tests reviewers: I'll add those related filters in >>> next round). >>> >>> Vladimir: Roland integrated a bunch of changes to make loop* code look >>> better. I can tell that we're not done with C2 yet. Can you look over >>> the code and see what is ok, and especially what is not ok, so that we >>> can focus our efforts on the relevant parts? >>> >>> Updated set of webrevs: >>> http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/ >>> >>> Thanks, >>> Roman >>> >>> >>>> Hi, >>>> >>>> This is the first round of changes for including Shenandoah GC into >>>> mainline. >>>> I divided the review into parts that roughly correspond to the mailing lists >>>> that would normally review it, and I divided it into 'shared' code >>>> changes and >>>> 'shenandoah' code changes (actually, mostly additions). The intend is to >>>> eventually >>>> push them as single 'combined' changeset, once reviewed. >>>> >>>> JEP: >>>> ? https://openjdk.java.net/jeps/189 >>>> Bug entry: >>>> >>>> ?https://bugs.openjdk.java.net/browse/JDK-8214259 >>>> >>>> Webrevs: >>>> ? http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/ >>>> >>>> For those who want to see the full change, have a look at the >>>> shenandoah-complete >>>> >>>> directory, >>>> it contains the full combined webrev. Alternatively, there is the file >>>> shenandoah-master.patch >>>> , >>>> which is what I intend to commit (and which should be equivalent to the >>>> 'shenandoah-complete' webrev). >>>> >>>> Sections to review (at this point) are the following: >>>> ?*) shenandoah-gc >>>> >>>> ??? - Actual Shenandoah implementation, almost completely residing in >>>> gc/shenandoah >>>> >>>> ?*) shared-gc >>>> >>>> ??? - This is mostly boilerplate that is common to any GC >>>> ??? - referenceProcessor.cpp has a little change to make one assert not >>>> fail (next to CMS and G1) >>>> ??? - taskqueue.hpp has some small adjustments to enable subclassing >>>> >>>> ?*) shared-serviceability >>>> >>>> ??? - The usual code to support another GC >>>> >>>> ?*) shared-runtime >>>> >>>> ??? - A number of friends declarations to allow Shenandoah iterators to >>>> hook up with, >>>> ????? e.g. ClassLoaderData, CodeCache, etc >>>> ??? - Warning and disabling JFR LeakProfiler >>>> ??? - fieldDescriptor.hpp added is_stable() accessor, for use in >>>> Shenandoah C2 optimizations >>>> ??? - Locks initialization in mutexLocker.cpp as usual >>>> ??? - VM operations defines for Shenandoah's VM ops >>>> ??? - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in >>>> Shenandoah's logging >>>> ??? - The usual macros in macro.hpp >>>> >>>> ?*) shared-build >>>> >>>> ??? - Add shenandoah feature, enabled by default, as agreed with >>>> Vladimir K. beforehand >>>> ??? - Some flags for shenandoah-enabled compilation to get >>>> SUPPORT_BARRIER_ON_PRIMITIVES >>>> ????? and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for >>>> Shenandoah's barriers >>>> ??? - --param inline-unit-growth=1000 settings for 2 shenandoah source >>>> files, which is >>>> ????? useful to get the whole marking loop inlined (observed significant >>>> regression if we >>>> ????? don't) >>>> >>>> ?*) shared-tests >>>> >>>> ??? - Test infrastructure to support Shenandoah >>>> ??? - Shenandoah test groups >>>> ??? - Exclude Shenandoah in various tests that can be run with selected GC >>>> ??? - Enable/add configure for Shenandoah for tests that make sense to >>>> run with it >>>> >>>> ?*) shenandoah-tests >>>> >>>> ??? - Shenandoah specific tests, most reside in gc/shenandoah subdirectory >>>> ??? - A couple of tests configurations have been added, e.g. >>>> TestGCBasherWithShenandoah.java >>>> >>>> I intentionally left out shared-compiler for now, because we have some >>>> work left to do >>>> there, but if you click around you'll find the patch anyway, in case you >>>> want to take >>>> a peek at it. >>>> >>>> We have regular builds on: >>>> ? - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x} >>>> ? - {Windows} x {x86_64}, >>>> ? - {MacOS X} x {x86_64} >>>> >>>> This also routinely passes: >>>> ? - the new Shenandoah tests >>>> ? - jcstress with/without aggressive Shenandoah verification >>>> ? - specjvm2008 with/without aggressive Shenandoah verification >>>> >>>> >>>> I'd like to thank my collegues at Red Hat: Christine Flood, she deserves >>>> the credit for being the original inventor of Shenandoah, Aleksey >>>> Shipl?v, Roland Westrelin & Zhengyu Gu for their countless >>>> contributions, everybody else in Red Hat's OpenJDK team for testing, >>>> advice and support, my collegues in Oracle's GC, runtime and compiler >>>> teams for tirelessly helping with and reviewing all the GC interface and >>>> related changes, and of course the many early adopters for reporting >>>> bugs and success stories and feature requests: we wouldn't be here >>>> without any of you! >>>> >>>> Best regards, >>>> Roman >>>>