From david.holmes at oracle.com Tue Sep 1 02:57:47 2015 From: david.holmes at oracle.com (David Holmes) Date: Tue, 1 Sep 2015 12:57:47 +1000 Subject: RFR: 8066821(S) Enhance command line processing to manage deprecating and obsoleting -XX command line arguments In-Reply-To: <55E49CF5.3080200@oracle.com> References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com> <54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com> <55ACBF85.9080404@oracle.com> <55AD2990.8030000@oracle.com> <55ADA880.7090602@oracle.com> <55B1119D.4030802@oracle.com> <55C41C20.90208@oracle.com> <55C83C5C.60702@oracle.com> <55C8D456.9030006@oracle.com> <5A08C1CB-58F9-4193-802F-767EE11F4D2B@oracle.com> <55CA4E0E.2070503@oracle.com> <55DB540A.4010804@oracle.com> <55DCE602.3040801@oracle.com> <55DE9846.8050008@oracle.com> <55DF86C8.3030200@oracle.com> <55E3D443.5020708@oracle.com> <55E49CF5.3080200@oracle.com> Message-ID: <55E5142B.50604@oracle.com> On 1/09/2015 4:29 AM, Derek White wrote: > Hi David, > > Thanks for the comments. > > On 8/31/15 12:12 AM, David Holmes wrote: >> Hi Derek, >> >> Looking good - thanks for the persistence! >> >> One minor comment in arguments.cpp: >> >> 327 { "not deprecated or obsolete", JDK_Version::undefined(), >> JDK_Version::jdk(9), JDK_Version::undefined() }, >> >> That appears to be obsolete. Did you mean: >> >> { "not deprecated or obsolete", JDK_Version::undefined(), >> JDK_Version::undefined(), JDK_Version::jdk(9) }, > > Yes, that was wrong. I fixed this, retested and verified all expected > errors were printed. I also moved the test data to the end of the table > to keep the import stuff first. > > I'm re-running jrprt. Would you like another webrev? Nope that is fine thanks. David > - Derek >> >> Thanks, >> David >> >> >> >> On 28/08/2015 7:53 AM, Derek White wrote: >>> Hi David, >>> >>> Comments inline... >>> >>> On 8/27/15 12:55 AM, David Holmes wrote: >>>> On 26/08/2015 8:02 AM, Derek White wrote: >>>>> Thanks Karen! >>>>> >>>>> If I don't get any other feedback, can I get you to sponsor the change >>>>> for me? After folding in your fixes... >>>> >>>> I need to see those fixes first as I got quite confused trying to >>>> reconcile the code with the documentation. >>> OK, I've integrated Karen's final documentation feedback. >>> >>>> I'm unclear why is_obsolete doesn't care about EXPIRED but >>>> is_deprecated does? >>> Because the caller of is_obsolete doesn't care about EXPIRED but the >>> caller of is_deprecated does :-) >>> >>> In the first case, Arguments::process_argument has already tried >>> processing the argument as a normal, aliased, or deprecated flag, and >>> failed. Right before it starts whining about illegal flags it checks to >>> see if it's obsolete, and if so prints the warning and continues on to >>> the next flag. If the flag isn't obsolete then it must be non-existent. >>> But process_argument treats both cases the same (it's in the definition >>> of EXPIRED). In any case, that was the contract for the original >>> is_newly_obsolete() method. >>> >>> In the "is_deprecated" case, the caller (parse_argument) is in the >>> middle of handling a real-life global flag, but it needs to decide if >>> should print a deprecation warning, or treat it as a normal flag, or >>> note that the flag is now obsolete or expired, in which case it has to >>> bail out of parse_argument. So parse_argument needs to know three >>> states. >>>> Also you often state something like "fits into the range specified" >>>> but it is unclear what range you are referring to. >>> >>> I simplified both the code and documentation you mentioned. The >>> implementations had gotten "overly factored". I hope you find the new >>> code and documentation clearer. >>> >>> I have a JPRT build out running, but are the new webrevs: >>> >>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.09 >>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.8.v.9 >>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.7.v.9 >>> >>> Thanks for your comments! >>> >>> - Derek >>>> >>>> Thanks, >>>> David >>>> >>>>> - Derek >>>>> >>>>> On 8/25/15 2:05 PM, Karen Kinnear wrote: >>>>>> Derek, >>>>>> >>>>>> Many thanks for the latest changes. And thank you for the >>>>>> verify_special_jvm_flags -- that will remind us to >>>>>> clean up the flags when the version number changes. So the expired_in >>>>>> information is for internal use only - we >>>>>> don't print that information for users, we simply tell them "in a >>>>>> future release". >>>>>> >>>>>> Looks good. No need for my further code review. >>>>>> >>>>>> Minor comments: >>>>>> 1. arguments.hpp >>>>>> line 421 "In the case" -> "In this case" >>>>>> >>>>>> 2. arguments.cpp comments >>>>>> >>>>>> I love the high level comments - could you possibly update them? >>>>>> - e.g. they still talk about the deprecated_jvm_flags and >>>>>> obsolete_jvm_flags tables. >>>>>> - e.g. expired and obsolete options should have their global >>>>>> variable definitions and >>>>>> related implementations removed ... >>>>>> >>>>>> comment line 2119 - extra spaces at the front? (or webrev artifact?) >>>>>> >>>>>> thanks very much, >>>>>> Karen >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Aug 24, 2015, at 1:27 PM, Derek White wrote: >>>>>> >>>>>>> RFR: >>>>>>> >>>>>>> This version incorporates Dave and Karen's suggestions to >>>>>>> automatically handle deprecated-> obsolete transitions. This is >>>>>>> implemented with a unified table of deprecated and obsolete options >>>>>>> ("special_jvm_flags"). >>>>>>> >>>>>>> Karen also suggested (offline) that it would be good to verify that >>>>>>> obsolete and expired options no longer have a "globals" variable >>>>>>> defined. This revision adds debug checking of the special_jvm_flags >>>>>>> table to check a bunch of constraints, such as deprecated version >>>>>>> must be < obsolete version < expired version, and checks for >>>>>>> duplicate table entries as well as looking for left-over "globals" >>>>>>> variables. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.08 >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8066821 >>>>>>> >>>>>>> I also have a webrev of webrev.07 vs. webrev.08: >>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.7.v.8 >>>>>>> >>>>>>> Thanks for looking! >>>>>>> >>>>>>> - Derek >>>>>>> >>>>>>> On 8/11/15 3:33 PM, Derek White wrote: >>>>>>>> Hi Karen, >>>>>>>> >>>>>>>> As much as I want to get this RFE over and out, I see your and >>>>>>>> David >>>>>>>> H's point :-) >>>>>>>> >>>>>>>> I'll spin another rev... >>>>>>>> >>>>>>>> - Derek >>>>>>>> >>>>>>>> On 8/10/15 6:07 PM, Karen Kinnear wrote: >>>>>>>>> Derek, >>>>>>>>> >>>>>>>>> Thank you for all of your work on this and responses to >>>>>>>>> suggestions. >>>>>>>>> >>>>>>>>> I had a couple of questions/comments: >>>>>>>>> >>>>>>>>> I appreciate that you have created mechanisms to allow: >>>>>>>>> 1. 2 step obsolete: obsolete_jvm_flags >>>>>>>>> - step 1: obsolete: warn and ignore >>>>>>>>> - step 2: expire >>>>>>>>> - and I expect we will use this for most -XX flags, >>>>>>>>> especially >>>>>>>>> develop flags >>>>>>>>> >>>>>>>>> 2. 2 step deprecate: deprecated_jvm_flags >>>>>>>>> - step 1: deprecate: warn and handle >>>>>>>>> - step 2: expire >>>>>>>>> >>>>>>>>> I see this approach as useful for aliasing and for a few other >>>>>>>>> flags, but not >>>>>>>>> the common path. >>>>>>>>> >>>>>>>>> My concerns: >>>>>>>>> >>>>>>>>> With my understanding of how this is set up today, for the flags >>>>>>>>> that are most exposed to >>>>>>>>> customers, which would want the 3 step deprecation, it would be >>>>>>>>> too >>>>>>>>> easy for an engineer >>>>>>>>> to add the flag to the deprecated_jvm_flags and forget to add >>>>>>>>> it to >>>>>>>>> the obsolete_jvm_flags, >>>>>>>>> as the comments under deprecated_jvm_flags suggest. And we don't >>>>>>>>> want folks to have >>>>>>>>> to go back and do a second step later for flags if they could >>>>>>>>> do it >>>>>>>>> all at once. >>>>>>>>> >>>>>>>>> I think it would be confusing to add a flag to both >>>>>>>>> deprecated_jvm_flags and obsolete_jvm_flags >>>>>>>>> at the same time, since then the expiration release is not clear. >>>>>>>>> >>>>>>>>> 3. For product non-XX, commercial and common -XX flags what I >>>>>>>>> would >>>>>>>>> like to see is >>>>>>>>> a three-step table which I would have called deprecated_jvm_flags, >>>>>>>>> which did >>>>>>>>> - step 1: deprecate: warn and handle >>>>>>>>> - step 2: obsolete: warn and ignore >>>>>>>>> - step 3: expire: unrecognized error >>>>>>>>> >>>>>>>>> Would you be willing to add a table like that which is the >>>>>>>>> recommended approach >>>>>>>>> for external supported flags? >>>>>>>>> >>>>>>>>> So one way to do this would be to change the current >>>>>>>>> deprecated_jvm_flags to have >>>>>>>>> three fields, so that the usual path would be to have three >>>>>>>>> releases listed, and >>>>>>>>> for aliased flags leave the middle field with the undefined >>>>>>>>> version. >>>>>>>>> >>>>>>>>> Another approach would be to have the three fields and have it >>>>>>>>> used by >>>>>>>>> all three approaches, just fill in the steps that are appropriate. >>>>>>>>> >>>>>>>>> minor comment: >>>>>>>>> >>>>>>>>> >>>>>>>>> 1. arguments.hpp >>>>>>>>> Could you possibly change the comments from "and has expired >>>>>>>>> (should be ignored)" to >>>>>>>>> "and has expired (unrecognized error)" >>>>>>>>> >>>>>>>>> unless what you mean really is should be ignored, which is not the >>>>>>>>> same expired. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Karen >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Aug 10, 2015, at 12:41 PM, Derek White wrote: >>>>>>>>> >>>>>>>>>> Thanks David! >>>>>>>>>> >>>>>>>>>> - Derek >>>>>>>>>> >>>>>>>>>> On 8/10/15 1:53 AM, David Holmes wrote: >>>>>>>>>>> Hi Derek, >>>>>>>>>>> >>>>>>>>>>> I don't have any further comments. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> >>>>>>>>>>> On 7/08/2015 12:46 PM, Derek White wrote: >>>>>>>>>>>> Another RFR. >>>>>>>>>>>> >>>>>>>>>>>> I've updated based on David and Kim's last comments. >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.07/ >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8066821 >>>>>>>>>>>> >>>>>>>>>>>> I also have a webrev of webrev.06 vs. webrev.07: >>>>>>>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.6.v.7/ >>>>>>>>>>>> >>>>>>>>>>>> [This webrev is confused about CommandLineOptionTest.java. I >>>>>>>>>>>> double-checked with a recursive diff and the only differences >>>>>>>>>>>> are in >>>>>>>>>>>> arguments.cpp and arguments.hpp.] >>>>>>>>>>>> >>>>>>>>>>>> Thanks for looking! >>>>>>>>>>>> >>>>>>>>>>>> - Derek >>>>>>>>>>>> .... >>>>> >>> > From aph at redhat.com Tue Sep 1 14:45:17 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 1 Sep 2015 15:45:17 +0100 Subject: RFR: 8134869: AARCH64: GHASH intrinsic is not optimal Message-ID: <55E5B9FD.805@redhat.com> I've been looking at the intrinsic we have for GHASH. While it is decent as it goes, its performance is considerably worse than some other implementations of GHASH on the same processor. Thanks are due to Alexander Alexeev who did a fine job implementing the x86 algorithm on AArch64, but the result is not optimal. on AArch64 we have the advantage of a bit-reversal instruction which x86 parts don't have, and this makes it possible to write a fully little-endian implementation of GHASH which is far more idiomatic on AArch64 than the big-endian implementation the x86 version uses. This gets us an overall performance improvement of AES/GCM of 10-20%. I've also taken the opportunity to add a lot of comments. The algorithms used are (fairly) obscure and most open source software implementations don't really explain what they're doing. In particular, the bizarre representation of polynomials in GF(2) (where byte ordering is little endian but bit ordering is big endian) is very confusing and surely deserves a comment or two. http://cr.openjdk.java.net/~aph/8134869-ghash-1/ One other remark: the AES/GCM implementation has a lot of overhead. Some profile data (on x86) looks like this: samples cum. samples % cum. % image name symbol name 479605 479605 36.8408 36.8408 31156.jo aescrypt_encryptBlock 301014 780619 23.1224 59.9632 31156.jo ghash_processBlocks 196563 977182 15.0990 75.0621 31156.jo int com.sun.crypto.provider.GCTR.doFinal(byte[], int, int, byte[], int) 50061 1027243 3.8454 78.9076 31156.jo void TestAESEncode.run() 48159 1075402 3.6993 82.6069 31156.jo void TestAESDecode.run() 18506 1093908 1.4215 84.0284 libjvm.so TypeArrayKlass::allocate_common(int, bool, Thread*) GCTR.doFinal() doesn't need do anything except increment a counter and call aescrypt_encryptBlock, but it still takes 15% of the total runtime. Intrinsifying GCTR.update() would solve this problem. Andrew. From derek.white at oracle.com Tue Sep 1 16:46:33 2015 From: derek.white at oracle.com (Derek White) Date: Tue, 1 Sep 2015 12:46:33 -0400 Subject: RFR: 8066821(S) Enhance command line processing to manage deprecating and obsoleting -XX command line arguments In-Reply-To: <55E5142B.50604@oracle.com> References: <54B44661.8070007@oracle.com> <54B84E91.1050708@oracle.com> <54D15A0B.6030501@oracle.com> <55A93BB3.4040402@oracle.com> <55ACBF85.9080404@oracle.com> <55AD2990.8030000@oracle.com> <55ADA880.7090602@oracle.com> <55B1119D.4030802@oracle.com> <55C41C20.90208@oracle.com> <55C83C5C.60702@oracle.com> <55C8D456.9030006@oracle.com> <5A08C1CB-58F9-4193-802F-767EE11F4D2B@oracle.com> <55CA4E0E.2070503@oracle.com> <55DB540A.4010804@oracle.com> <55DCE602.3040801@oracle.com> <55DE9846.8050008@oracle.com> <55DF86C8.3030200@oracle.com> <55E3D443.5020708@oracle.com> <55E49CF5.3080200@oracle.com> <55E5142B.50604@oracle.com> Message-ID: <55E5D669.1040105@oracle.com> Thanks David! - Derek On 8/31/15 10:57 PM, David Holmes wrote: > On 1/09/2015 4:29 AM, Derek White wrote: >> Hi David, >> >> Thanks for the comments. >> >> On 8/31/15 12:12 AM, David Holmes wrote: >>> Hi Derek, >>> >>> Looking good - thanks for the persistence! >>> >>> One minor comment in arguments.cpp: >>> >>> 327 { "not deprecated or obsolete", JDK_Version::undefined(), >>> JDK_Version::jdk(9), JDK_Version::undefined() }, >>> >>> That appears to be obsolete. Did you mean: >>> >>> { "not deprecated or obsolete", JDK_Version::undefined(), >>> JDK_Version::undefined(), JDK_Version::jdk(9) }, >> >> Yes, that was wrong. I fixed this, retested and verified all expected >> errors were printed. I also moved the test data to the end of the table >> to keep the import stuff first. >> >> I'm re-running jrprt. Would you like another webrev? > > Nope that is fine thanks. > > David > > >> - Derek >>> >>> Thanks, >>> David >>> >>> >>> >>> On 28/08/2015 7:53 AM, Derek White wrote: >>>> Hi David, >>>> >>>> Comments inline... >>>> >>>> On 8/27/15 12:55 AM, David Holmes wrote: >>>>> On 26/08/2015 8:02 AM, Derek White wrote: >>>>>> Thanks Karen! >>>>>> >>>>>> If I don't get any other feedback, can I get you to sponsor the >>>>>> change >>>>>> for me? After folding in your fixes... >>>>> >>>>> I need to see those fixes first as I got quite confused trying to >>>>> reconcile the code with the documentation. >>>> OK, I've integrated Karen's final documentation feedback. >>>> >>>>> I'm unclear why is_obsolete doesn't care about EXPIRED but >>>>> is_deprecated does? >>>> Because the caller of is_obsolete doesn't care about EXPIRED but the >>>> caller of is_deprecated does :-) >>>> >>>> In the first case, Arguments::process_argument has already tried >>>> processing the argument as a normal, aliased, or deprecated flag, and >>>> failed. Right before it starts whining about illegal flags it >>>> checks to >>>> see if it's obsolete, and if so prints the warning and continues on to >>>> the next flag. If the flag isn't obsolete then it must be >>>> non-existent. >>>> But process_argument treats both cases the same (it's in the >>>> definition >>>> of EXPIRED). In any case, that was the contract for the original >>>> is_newly_obsolete() method. >>>> >>>> In the "is_deprecated" case, the caller (parse_argument) is in the >>>> middle of handling a real-life global flag, but it needs to decide if >>>> should print a deprecation warning, or treat it as a normal flag, or >>>> note that the flag is now obsolete or expired, in which case it has to >>>> bail out of parse_argument. So parse_argument needs to know three >>>> states. >>>>> Also you often state something like "fits into the range specified" >>>>> but it is unclear what range you are referring to. >>>> >>>> I simplified both the code and documentation you mentioned. The >>>> implementations had gotten "overly factored". I hope you find the new >>>> code and documentation clearer. >>>> >>>> I have a JPRT build out running, but are the new webrevs: >>>> >>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.09 >>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.8.v.9 >>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.7.v.9 >>>> >>>> Thanks for your comments! >>>> >>>> - Derek >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> - Derek >>>>>> >>>>>> On 8/25/15 2:05 PM, Karen Kinnear wrote: >>>>>>> Derek, >>>>>>> >>>>>>> Many thanks for the latest changes. And thank you for the >>>>>>> verify_special_jvm_flags -- that will remind us to >>>>>>> clean up the flags when the version number changes. So the >>>>>>> expired_in >>>>>>> information is for internal use only - we >>>>>>> don't print that information for users, we simply tell them "in a >>>>>>> future release". >>>>>>> >>>>>>> Looks good. No need for my further code review. >>>>>>> >>>>>>> Minor comments: >>>>>>> 1. arguments.hpp >>>>>>> line 421 "In the case" -> "In this case" >>>>>>> >>>>>>> 2. arguments.cpp comments >>>>>>> >>>>>>> I love the high level comments - could you possibly update them? >>>>>>> - e.g. they still talk about the deprecated_jvm_flags and >>>>>>> obsolete_jvm_flags tables. >>>>>>> - e.g. expired and obsolete options should have their global >>>>>>> variable definitions and >>>>>>> related implementations removed ... >>>>>>> >>>>>>> comment line 2119 - extra spaces at the front? (or webrev >>>>>>> artifact?) >>>>>>> >>>>>>> thanks very much, >>>>>>> Karen >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Aug 24, 2015, at 1:27 PM, Derek White wrote: >>>>>>> >>>>>>>> RFR: >>>>>>>> >>>>>>>> This version incorporates Dave and Karen's suggestions to >>>>>>>> automatically handle deprecated-> obsolete transitions. This is >>>>>>>> implemented with a unified table of deprecated and obsolete >>>>>>>> options >>>>>>>> ("special_jvm_flags"). >>>>>>>> >>>>>>>> Karen also suggested (offline) that it would be good to verify >>>>>>>> that >>>>>>>> obsolete and expired options no longer have a "globals" variable >>>>>>>> defined. This revision adds debug checking of the >>>>>>>> special_jvm_flags >>>>>>>> table to check a bunch of constraints, such as deprecated version >>>>>>>> must be < obsolete version < expired version, and checks for >>>>>>>> duplicate table entries as well as looking for left-over "globals" >>>>>>>> variables. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.08 >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8066821 >>>>>>>> >>>>>>>> I also have a webrev of webrev.07 vs. webrev.08: >>>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.7.v.8 >>>>>>>> >>>>>>>> Thanks for looking! >>>>>>>> >>>>>>>> - Derek >>>>>>>> >>>>>>>> On 8/11/15 3:33 PM, Derek White wrote: >>>>>>>>> Hi Karen, >>>>>>>>> >>>>>>>>> As much as I want to get this RFE over and out, I see your and >>>>>>>>> David >>>>>>>>> H's point :-) >>>>>>>>> >>>>>>>>> I'll spin another rev... >>>>>>>>> >>>>>>>>> - Derek >>>>>>>>> >>>>>>>>> On 8/10/15 6:07 PM, Karen Kinnear wrote: >>>>>>>>>> Derek, >>>>>>>>>> >>>>>>>>>> Thank you for all of your work on this and responses to >>>>>>>>>> suggestions. >>>>>>>>>> >>>>>>>>>> I had a couple of questions/comments: >>>>>>>>>> >>>>>>>>>> I appreciate that you have created mechanisms to allow: >>>>>>>>>> 1. 2 step obsolete: obsolete_jvm_flags >>>>>>>>>> - step 1: obsolete: warn and ignore >>>>>>>>>> - step 2: expire >>>>>>>>>> - and I expect we will use this for most -XX flags, >>>>>>>>>> especially >>>>>>>>>> develop flags >>>>>>>>>> >>>>>>>>>> 2. 2 step deprecate: deprecated_jvm_flags >>>>>>>>>> - step 1: deprecate: warn and handle >>>>>>>>>> - step 2: expire >>>>>>>>>> >>>>>>>>>> I see this approach as useful for aliasing and for a few other >>>>>>>>>> flags, but not >>>>>>>>>> the common path. >>>>>>>>>> >>>>>>>>>> My concerns: >>>>>>>>>> >>>>>>>>>> With my understanding of how this is set up today, for the flags >>>>>>>>>> that are most exposed to >>>>>>>>>> customers, which would want the 3 step deprecation, it would be >>>>>>>>>> too >>>>>>>>>> easy for an engineer >>>>>>>>>> to add the flag to the deprecated_jvm_flags and forget to add >>>>>>>>>> it to >>>>>>>>>> the obsolete_jvm_flags, >>>>>>>>>> as the comments under deprecated_jvm_flags suggest. And we don't >>>>>>>>>> want folks to have >>>>>>>>>> to go back and do a second step later for flags if they could >>>>>>>>>> do it >>>>>>>>>> all at once. >>>>>>>>>> >>>>>>>>>> I think it would be confusing to add a flag to both >>>>>>>>>> deprecated_jvm_flags and obsolete_jvm_flags >>>>>>>>>> at the same time, since then the expiration release is not >>>>>>>>>> clear. >>>>>>>>>> >>>>>>>>>> 3. For product non-XX, commercial and common -XX flags what I >>>>>>>>>> would >>>>>>>>>> like to see is >>>>>>>>>> a three-step table which I would have called >>>>>>>>>> deprecated_jvm_flags, >>>>>>>>>> which did >>>>>>>>>> - step 1: deprecate: warn and handle >>>>>>>>>> - step 2: obsolete: warn and ignore >>>>>>>>>> - step 3: expire: unrecognized error >>>>>>>>>> >>>>>>>>>> Would you be willing to add a table like that which is the >>>>>>>>>> recommended approach >>>>>>>>>> for external supported flags? >>>>>>>>>> >>>>>>>>>> So one way to do this would be to change the current >>>>>>>>>> deprecated_jvm_flags to have >>>>>>>>>> three fields, so that the usual path would be to have three >>>>>>>>>> releases listed, and >>>>>>>>>> for aliased flags leave the middle field with the undefined >>>>>>>>>> version. >>>>>>>>>> >>>>>>>>>> Another approach would be to have the three fields and have it >>>>>>>>>> used by >>>>>>>>>> all three approaches, just fill in the steps that are >>>>>>>>>> appropriate. >>>>>>>>>> >>>>>>>>>> minor comment: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 1. arguments.hpp >>>>>>>>>> Could you possibly change the comments from "and has expired >>>>>>>>>> (should be ignored)" to >>>>>>>>>> "and has expired (unrecognized error)" >>>>>>>>>> >>>>>>>>>> unless what you mean really is should be ignored, which is >>>>>>>>>> not the >>>>>>>>>> same expired. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Karen >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Aug 10, 2015, at 12:41 PM, Derek White wrote: >>>>>>>>>> >>>>>>>>>>> Thanks David! >>>>>>>>>>> >>>>>>>>>>> - Derek >>>>>>>>>>> >>>>>>>>>>> On 8/10/15 1:53 AM, David Holmes wrote: >>>>>>>>>>>> Hi Derek, >>>>>>>>>>>> >>>>>>>>>>>> I don't have any further comments. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> >>>>>>>>>>>> On 7/08/2015 12:46 PM, Derek White wrote: >>>>>>>>>>>>> Another RFR. >>>>>>>>>>>>> >>>>>>>>>>>>> I've updated based on David and Kim's last comments. >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.07/ >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8066821 >>>>>>>>>>>>> >>>>>>>>>>>>> I also have a webrev of webrev.06 vs. webrev.07: >>>>>>>>>>>>> http://cr.openjdk.java.net/~drwhite/8066821/webrev.6.v.7/ >>>>>>>>>>>>> >>>>>>>>>>>>> [This webrev is confused about CommandLineOptionTest.java. I >>>>>>>>>>>>> double-checked with a recursive diff and the only differences >>>>>>>>>>>>> are in >>>>>>>>>>>>> arguments.cpp and arguments.hpp.] >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for looking! >>>>>>>>>>>>> >>>>>>>>>>>>> - Derek >>>>>>>>>>>>> .... >>>>>> >>>> >> From vladimir.kozlov at oracle.com Tue Sep 1 20:45:00 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 Sep 2015 13:45:00 -0700 Subject: RFR: 8134869: AARCH64: GHASH intrinsic is not optimal In-Reply-To: <55E5B9FD.805@redhat.com> References: <55E5B9FD.805@redhat.com> Message-ID: <55E60E4C.4030807@oracle.com> Very nice rewrite. Looks good. Thanks, Vladimir On 9/1/15 7:45 AM, Andrew Haley wrote: > I've been looking at the intrinsic we have for GHASH. While it is > decent as it goes, its performance is considerably worse than some > other implementations of GHASH on the same processor. > > Thanks are due to Alexander Alexeev who did a fine job implementing > the x86 algorithm on AArch64, but the result is not optimal. on > AArch64 we have the advantage of a bit-reversal instruction which x86 > parts don't have, and this makes it possible to write a fully > little-endian implementation of GHASH which is far more idiomatic on > AArch64 than the big-endian implementation the x86 version uses. This > gets us an overall performance improvement of AES/GCM of 10-20%. > > I've also taken the opportunity to add a lot of comments. The > algorithms used are (fairly) obscure and most open source software > implementations don't really explain what they're doing. In > particular, the bizarre representation of polynomials in GF(2) (where > byte ordering is little endian but bit ordering is big endian) is very > confusing and surely deserves a comment or two. > > http://cr.openjdk.java.net/~aph/8134869-ghash-1/ > > One other remark: the AES/GCM implementation has a lot of overhead. > Some profile data (on x86) looks like this: > > samples cum. samples % cum. % image name symbol name > 479605 479605 36.8408 36.8408 31156.jo aescrypt_encryptBlock > 301014 780619 23.1224 59.9632 31156.jo ghash_processBlocks > 196563 977182 15.0990 75.0621 31156.jo int com.sun.crypto.provider.GCTR.doFinal(byte[], int, int, byte[], int) > 50061 1027243 3.8454 78.9076 31156.jo void TestAESEncode.run() > 48159 1075402 3.6993 82.6069 31156.jo void TestAESDecode.run() > 18506 1093908 1.4215 84.0284 libjvm.so TypeArrayKlass::allocate_common(int, bool, Thread*) > > GCTR.doFinal() doesn't need do anything except increment a counter > and call aescrypt_encryptBlock, but it still takes 15% of the total > runtime. Intrinsifying GCTR.update() would solve this problem. > > Andrew. > From vladimir.kozlov at oracle.com Tue Sep 1 23:59:17 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 Sep 2015 16:59:17 -0700 Subject: RFR(S) 8134898: Small fixes found during JVMCI work Message-ID: <55E63BD5.8030802@oracle.com> http://cr.openjdk.java.net/~kvn/8134898/webrev/ https://bugs.openjdk.java.net/browse/JDK-8134898 During review of JVMCI changes we were asked to push not related small changes separately. Removed jre/ from jdk paths in makefiles since jdk9 does not have it anymore. Solaris vm.make and buildtree.make missed changes other unix platforms have. Fix type in doc. Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to do that. It is the only incompatibility in memory layout between fastdebug and debug VM. We maintained such compatibility for long time and it was broken with 7195622 changes. For AOT project it is very important to keep compatibility because we use fastdebug VM for JAOTC tool to compile aot code and we want to be able use debug VM to debug execution of aot code. Note, using debug VM with JAOTC will take 5-10 times longer to compile. Thanks, Vladimir From coleen.phillimore at oracle.com Wed Sep 2 03:55:53 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 01 Sep 2015 23:55:53 -0400 Subject: RFR(S) 8134898: Small fixes found during JVMCI work In-Reply-To: <55E63BD5.8030802@oracle.com> References: <55E63BD5.8030802@oracle.com> Message-ID: <55E67349.10407@oracle.com> On 9/1/15 7:59 PM, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8134898/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8134898 > > During review of JVMCI changes we were asked to push not related small > changes separately. > > Removed jre/ from jdk paths in makefiles since jdk9 does not have it > anymore. > Solaris vm.make and buildtree.make missed changes other unix platforms > have. > Fix type in doc. > Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to > do that. Adding -DCHECK_UNHANDLED_OOPS to the debug build adds a level of indirection to the oops that are somewhat annoying during debugging: eg. 692 void java_lang_Class::set_component_mirror(oop java_class, oop comp_mirror) { 693 assert(_component_mirror_offset != 0, "must be set"); 694 java_class->obj_field_put(_component_mirror_offset, comp_mirror); 695 } (gdb) print java_class $7 = {_o = 0x8cb003b8} (gdb) print java_mirror No symbol "java_mirror" in current context. (gdb) print comp_mirror $8 = {_o = 0x8cb001b0} (gdb) call hsfind(java_class._o) warning: Using non-standard conversion to match function hsfind to supplied arguments "Executing hsfind" 0x000000008cb003b8 is an oop java.lang.Class - klass: 'java/lang/Class' - ---- fields (total size 13 words): ... vs. 693 assert(_component_mirror_offset != 0, "must be set"); (gdb) print java_class $1 = (oop) 0x8cb003b8 (gdb) print comp_mirror $2 = (oop) 0x8cb001b0 (gdb) print hsfind(java_class) warning: Using non-standard conversion to match function hsfind to supplied arguments "Executing hsfind" 0x000000008cb003b8 is an oop java.lang.Class - klass: 'java/lang/Class' - ---- fields (total size 13 words): ... The GC people deal with far more 'oop' objects in their debugging, which is why they asked me originally not to enable CHECK_UNHANDLED_OOPS in the debug build. To resolve the incompatibility, you can simply remove CHECK_UNHANDLED_OOPS around the field in Thread, and put it under #ifndef PRODUCT. Coleen > It is the only incompatibility in memory layout between fastdebug and > debug VM. We maintained such compatibility for long time and it was > broken with 7195622 changes. > For AOT project it is very important to keep compatibility because we > use fastdebug VM for JAOTC tool to compile aot code and we want to be > able use debug VM to debug execution of aot code. > Note, using debug VM with JAOTC will take 5-10 times longer to compile. > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Wed Sep 2 04:19:48 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 Sep 2015 21:19:48 -0700 Subject: RFR(S) 8134898: Small fixes found during JVMCI work In-Reply-To: <55E67349.10407@oracle.com> References: <55E63BD5.8030802@oracle.com> <55E67349.10407@oracle.com> Message-ID: <55E678E4.5090500@oracle.com> > To resolve the incompatibility, you can simply remove > CHECK_UNHANDLED_OOPS around the field in Thread, and put it under > #ifndef PRODUCT. Wonderful! :) I thought it requires #ifdef CHECK_UNHANDLED_OOPS. Please, look on new changes: http://cr.openjdk.java.net/~kvn/8134898/webrev.01/ Thanks, Vladimir On 9/1/15 8:55 PM, Coleen Phillimore wrote: > > > On 9/1/15 7:59 PM, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8134898/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8134898 >> >> During review of JVMCI changes we were asked to push not related small >> changes separately. >> >> Removed jre/ from jdk paths in makefiles since jdk9 does not have it >> anymore. >> Solaris vm.make and buildtree.make missed changes other unix platforms >> have. >> Fix type in doc. >> Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to >> do that. > Adding -DCHECK_UNHANDLED_OOPS to the debug build adds a level of > indirection to the oops that are somewhat annoying during debugging: > > eg. > > 692 void java_lang_Class::set_component_mirror(oop java_class, oop > comp_mirror) { > 693 assert(_component_mirror_offset != 0, "must be set"); > 694 java_class->obj_field_put(_component_mirror_offset, > comp_mirror); > 695 } > > (gdb) print java_class > $7 = {_o = 0x8cb003b8} > (gdb) print java_mirror > No symbol "java_mirror" in current context. > (gdb) print comp_mirror > $8 = {_o = 0x8cb001b0} > (gdb) call hsfind(java_class._o) > warning: Using non-standard conversion to match function hsfind to > supplied arguments > > "Executing hsfind" > 0x000000008cb003b8 is an oop > java.lang.Class > - klass: 'java/lang/Class' > - ---- fields (total size 13 words): > ... > > > vs. > > 693 assert(_component_mirror_offset != 0, "must be set"); > (gdb) print java_class > $1 = (oop) 0x8cb003b8 > (gdb) print comp_mirror > $2 = (oop) 0x8cb001b0 > (gdb) print hsfind(java_class) > warning: Using non-standard conversion to match function hsfind to > supplied arguments > > "Executing hsfind" > 0x000000008cb003b8 is an oop > java.lang.Class > - klass: 'java/lang/Class' > - ---- fields (total size 13 words): > ... > > > The GC people deal with far more 'oop' objects in their debugging, which > is why they asked me originally not to enable CHECK_UNHANDLED_OOPS in > the debug build. > > To resolve the incompatibility, you can simply remove > CHECK_UNHANDLED_OOPS around the field in Thread, and put it under > #ifndef PRODUCT. > > Coleen >> It is the only incompatibility in memory layout between fastdebug and >> debug VM. We maintained such compatibility for long time and it was >> broken with 7195622 changes. >> For AOT project it is very important to keep compatibility because we >> use fastdebug VM for JAOTC tool to compile aot code and we want to be >> able use debug VM to debug execution of aot code. >> Note, using debug VM with JAOTC will take 5-10 times longer to compile. >> >> Thanks, >> Vladimir > From stefan.karlsson at oracle.com Wed Sep 2 07:08:04 2015 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 02 Sep 2015 09:08:04 +0200 Subject: RFR(S) 8134898: Small fixes found during JVMCI work In-Reply-To: <55E67349.10407@oracle.com> References: <55E63BD5.8030802@oracle.com> <55E67349.10407@oracle.com> Message-ID: <55E6A054.5080803@oracle.com> On 2015-09-02 05:55, Coleen Phillimore wrote: > > > On 9/1/15 7:59 PM, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8134898/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8134898 >> >> During review of JVMCI changes we were asked to push not related >> small changes separately. >> >> Removed jre/ from jdk paths in makefiles since jdk9 does not have it >> anymore. >> Solaris vm.make and buildtree.make missed changes other unix >> platforms have. >> Fix type in doc. >> Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to >> do that. > Adding -DCHECK_UNHANDLED_OOPS to the debug build adds a level of > indirection to the oops that are somewhat annoying during debugging: > > eg. > > 692 void java_lang_Class::set_component_mirror(oop java_class, oop > comp_mirror) { > 693 assert(_component_mirror_offset != 0, "must be set"); > 694 java_class->obj_field_put(_component_mirror_offset, > comp_mirror); > 695 } > > (gdb) print java_class > $7 = {_o = 0x8cb003b8} > (gdb) print java_mirror > No symbol "java_mirror" in current context. > (gdb) print comp_mirror > $8 = {_o = 0x8cb001b0} > (gdb) call hsfind(java_class._o) > warning: Using non-standard conversion to match function hsfind to > supplied arguments > > "Executing hsfind" > 0x000000008cb003b8 is an oop > java.lang.Class > - klass: 'java/lang/Class' > - ---- fields (total size 13 words): > ... > > > vs. > > 693 assert(_component_mirror_offset != 0, "must be set"); > (gdb) print java_class > $1 = (oop) 0x8cb003b8 > (gdb) print comp_mirror > $2 = (oop) 0x8cb001b0 > (gdb) print hsfind(java_class) > warning: Using non-standard conversion to match function hsfind to > supplied arguments > > "Executing hsfind" > 0x000000008cb003b8 is an oop > java.lang.Class > - klass: 'java/lang/Class' > - ---- fields (total size 13 words): > ... > > > The GC people deal with far more 'oop' objects in their debugging, > which is why they asked me originally not to enable > CHECK_UNHANDLED_OOPS in the debug build. Yes, it would be a pain to debug. Also, please don't change behaviors like as a part of a bug report with a title that doesn't describe what's being changed. Thanks, StefanK > > To resolve the incompatibility, you can simply remove > CHECK_UNHANDLED_OOPS around the field in Thread, and put it under > #ifndef PRODUCT. > > Coleen >> It is the only incompatibility in memory layout between fastdebug and >> debug VM. We maintained such compatibility for long time and it was >> broken with 7195622 changes. >> For AOT project it is very important to keep compatibility because we >> use fastdebug VM for JAOTC tool to compile aot code and we want to be >> able use debug VM to debug execution of aot code. >> Note, using debug VM with JAOTC will take 5-10 times longer to compile. >> >> Thanks, >> Vladimir > From bertrand.delsart at oracle.com Wed Sep 2 08:39:42 2015 From: bertrand.delsart at oracle.com (Bertrand Delsart) Date: Wed, 2 Sep 2015 10:39:42 +0200 Subject: RFR(S) 8134898: Small fixes found during JVMCI work In-Reply-To: <55E678E4.5090500@oracle.com> References: <55E63BD5.8030802@oracle.com> <55E67349.10407@oracle.com> <55E678E4.5090500@oracle.com> Message-ID: <55E6B5CE.9010901@oracle.com> On 02/09/2015 06:19, Vladimir Kozlov wrote: > > To resolve the incompatibility, you can simply remove > > CHECK_UNHANDLED_OOPS around the field in Thread, and put it under > > #ifndef PRODUCT. > > Wonderful! :) I thought it requires #ifdef CHECK_UNHANDLED_OOPS. > > Please, look on new changes: > > http://cr.openjdk.java.net/~kvn/8134898/webrev.01/ Looks good. Small suggestion (feel free to ignore it). Might be even better to do: // Support for Unhandled Oop detection -#ifdef CHECK_UNHANDLED_OOPS +#if defined(CHECK_UNHANDLED_OOPS) || defined(ASSERT) private: UnhandledOops* _unhandled_oops; +#endif +#ifdef CHECK_UNHANDLED_OOPS public: UnhandledOops* unhandled_oops() { return _unhandled_oops; } // Mark oop safe for gc. It may be stack allocated but won't move. This version: - does not prevent someone from building a PRODUCT VM with CHECK_HANDLED_OOPS if needed - ensures someone does not call unhandled_oops() in debug VMs when CHECK_UNHANDLED_OOPS is not defined Bertrand (not a Reviewer). > > Thanks, > Vladimir > > On 9/1/15 8:55 PM, Coleen Phillimore wrote: >> >> >> On 9/1/15 7:59 PM, Vladimir Kozlov wrote: >>> http://cr.openjdk.java.net/~kvn/8134898/webrev/ >>> https://bugs.openjdk.java.net/browse/JDK-8134898 >>> >>> During review of JVMCI changes we were asked to push not related small >>> changes separately. >>> >>> Removed jre/ from jdk paths in makefiles since jdk9 does not have it >>> anymore. >>> Solaris vm.make and buildtree.make missed changes other unix platforms >>> have. >>> Fix type in doc. >>> Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to >>> do that. >> Adding -DCHECK_UNHANDLED_OOPS to the debug build adds a level of >> indirection to the oops that are somewhat annoying during debugging: >> >> eg. >> >> 692 void java_lang_Class::set_component_mirror(oop java_class, oop >> comp_mirror) { >> 693 assert(_component_mirror_offset != 0, "must be set"); >> 694 java_class->obj_field_put(_component_mirror_offset, >> comp_mirror); >> 695 } >> >> (gdb) print java_class >> $7 = {_o = 0x8cb003b8} >> (gdb) print java_mirror >> No symbol "java_mirror" in current context. >> (gdb) print comp_mirror >> $8 = {_o = 0x8cb001b0} >> (gdb) call hsfind(java_class._o) >> warning: Using non-standard conversion to match function hsfind to >> supplied arguments >> >> "Executing hsfind" >> 0x000000008cb003b8 is an oop >> java.lang.Class >> - klass: 'java/lang/Class' >> - ---- fields (total size 13 words): >> ... >> >> >> vs. >> >> 693 assert(_component_mirror_offset != 0, "must be set"); >> (gdb) print java_class >> $1 = (oop) 0x8cb003b8 >> (gdb) print comp_mirror >> $2 = (oop) 0x8cb001b0 >> (gdb) print hsfind(java_class) >> warning: Using non-standard conversion to match function hsfind to >> supplied arguments >> >> "Executing hsfind" >> 0x000000008cb003b8 is an oop >> java.lang.Class >> - klass: 'java/lang/Class' >> - ---- fields (total size 13 words): >> ... >> >> >> The GC people deal with far more 'oop' objects in their debugging, which >> is why they asked me originally not to enable CHECK_UNHANDLED_OOPS in >> the debug build. >> >> To resolve the incompatibility, you can simply remove >> CHECK_UNHANDLED_OOPS around the field in Thread, and put it under >> #ifndef PRODUCT. >> >> Coleen >>> It is the only incompatibility in memory layout between fastdebug and >>> debug VM. We maintained such compatibility for long time and it was >>> broken with 7195622 changes. >>> For AOT project it is very important to keep compatibility because we >>> use fastdebug VM for JAOTC tool to compile aot code and we want to be >>> able use debug VM to debug execution of aot code. >>> Note, using debug VM with JAOTC will take 5-10 times longer to compile. >>> >>> Thanks, >>> Vladimir >> -- Bertrand Delsart, Grenoble Engineering Center Oracle, 180 av. de l'Europe, ZIRST de Montbonnot 38330 Montbonnot Saint Martin, FRANCE bertrand.delsart at oracle.com Phone : +33 4 76 18 81 23 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From edward.nevill at gmail.com Wed Sep 2 08:52:51 2015 From: edward.nevill at gmail.com (Edward Nevill) Date: Wed, 02 Sep 2015 09:52:51 +0100 Subject: [aarch64-port-dev ] RFR: 8134869: AARCH64: GHASH intrinsic is not optimal In-Reply-To: <55E5B9FD.805@redhat.com> References: <55E5B9FD.805@redhat.com> Message-ID: <1441183971.10331.13.camel@mylittlepony.linaroharston> On Tue, 2015-09-01 at 15:45 +0100, Andrew Haley wrote: > > http://cr.openjdk.java.net/~aph/8134869-ghash-1/ > > One other remark: the AES/GCM implementation has a lot of overhead. > Some profile data (on x86) looks like this: Looks good to me also. I have patched it and done a quick sanity check. I see ~14% additional improvement over Alexander's implementation bringing the overall improvement to slightly over 100%. One minor point. The following seems to have suffered some whitespace mangling. // Table vector lookup -#define INSN(NAME, op) \ - void NAME(FloatRegister Vd, SIMD_Arrangement T, FloatRegister Vn, unsigned registers, FloatRegister Vm) { \ - starti; \ - assert(T == T8B || T == T16B, "invalid arrangement"); \ - assert(0 < registers && registers <= 4, "invalid number of registers"); \ - f(0, 31), f((int)T & 1, 30), f(0b001110000, 29, 21), rf(Vm, 16), f(0, 15); \ - f(registers - 1, 14, 13), f(op, 12),f(0b00, 11, 10), rf(Vn, 5), rf(Vd, 0); \ +#define INSN(NAME, op) \ + void NAME(FloatRegister Vd, SIMD_Arrangement T, FloatRegister Vn, unsigned registers, FloatRegister Vm) { \ + starti; \ + assert(T == T8B || T == T16B, "invalid arrangement"); \ + assert(0 < registers && registers <= 4, "invalid number of registers"); \ + f(0, 31), f((int)T & 1, 30), f(0b001110000, 29, 21), rf(Vm, 16), f(0, 15); \ + f(registers - 1, 14, 13), f(op, 12),f(0b00, 11, 10), rf(Vn, 5), rf(Vd, 0); \ } It looks like the mangling is as a result of replacing spaces with TABs at some point. I cannot see any difference in the above other than whitespace. All the best, Ed. From aph at redhat.com Wed Sep 2 08:55:31 2015 From: aph at redhat.com (Andrew Haley) Date: Wed, 2 Sep 2015 09:55:31 +0100 Subject: [aarch64-port-dev ] RFR: 8134869: AARCH64: GHASH intrinsic is not optimal In-Reply-To: <1441183971.10331.13.camel@mylittlepony.linaroharston> References: <55E5B9FD.805@redhat.com> <1441183971.10331.13.camel@mylittlepony.linaroharston> Message-ID: <55E6B983.1040400@redhat.com> On 09/02/2015 09:52 AM, Edward Nevill wrote: > One minor point. The following seems to have suffered some whitespace mangling. It's whitespace correction: the lines are way too wide, and it's hard to read the rest unless you have a super-wide editor. Andrew. From edward.nevill at gmail.com Wed Sep 2 09:05:31 2015 From: edward.nevill at gmail.com (Edward Nevill) Date: Wed, 02 Sep 2015 10:05:31 +0100 Subject: [aarch64-port-dev ] RFR: 8134869: AARCH64: GHASH intrinsic is not optimal In-Reply-To: <55E6B983.1040400@redhat.com> References: <55E5B9FD.805@redhat.com> <1441183971.10331.13.camel@mylittlepony.linaroharston> <55E6B983.1040400@redhat.com> Message-ID: <1441184731.23750.3.camel@mylittlepony.linaroharston> On Wed, 2015-09-02 at 09:55 +0100, Andrew Haley wrote: > On 09/02/2015 09:52 AM, Edward Nevill wrote: > > One minor point. The following seems to have suffered some whitespace mangling. > > It's whitespace correction: the lines are way too wide, and it's hard to > read the rest unless you have a super-wide editor. OK. But the changeset as posted http://cr.openjdk.java.net/~aph/8134869-ghash-1/ seems to have TABs embedded in it. I thought this was verboten? If I edit the changset in vi with ":se list" I get // Table vector lookup$ -#define INSN(NAME, op) \$ - void NAME(FloatRegister Vd, SIMD_Arrangement T, FloatRegister Vn, unsigned registers, FloatRegister Vm) { \$ - starti; \$ - assert(T == T8B || T == T16B, "invalid arrangement"); \$ - assert(0 < registers && registers <= 4, "invalid number of registers"); \$ - f(0, 31), f((int)T & 1, 30), f(0b001110000, 29, 21), rf(Vm, 16), f(0, 15); \$ - f(registers - 1, 14, 13), f(op, 12),f(0b00, 11, 10), rf(Vn, 5), rf(Vd, 0); \$ +#define INSN(NAME, op)^I^I^I^I^I^I^I\$ + void NAME(FloatRegister Vd, SIMD_Arrangement T, FloatRegister Vn, unsigned registers, FloatRegister Vm) { \$ + starti;^I^I^I^I^I^I^I^I\$ + assert(T == T8B || T == T16B, "invalid arrangement");^I^I\$ + assert(0 < registers && registers <= 4, "invalid number of registers"); \$ + f(0, 31), f((int)T & 1, 30), f(0b001110000, 29, 21), rf(Vm, 16), f(0, 15); \$ + f(registers - 1, 14, 13), f(op, 12),f(0b00, 11, 10), rf(Vn, 5), rf(Vd, 0); \$ All the best, Ed. From aph at redhat.com Wed Sep 2 10:32:09 2015 From: aph at redhat.com (Andrew Haley) Date: Wed, 2 Sep 2015 11:32:09 +0100 Subject: [aarch64-port-dev ] RFR: 8134869: AARCH64: GHASH intrinsic is not optimal In-Reply-To: <1441184731.23750.3.camel@mylittlepony.linaroharston> References: <55E5B9FD.805@redhat.com> <1441183971.10331.13.camel@mylittlepony.linaroharston> <55E6B983.1040400@redhat.com> <1441184731.23750.3.camel@mylittlepony.linaroharston> Message-ID: <55E6D029.7090407@redhat.com> On 09/02/2015 10:05 AM, Edward Nevill wrote: > But the changeset as posted > > http://cr.openjdk.java.net/~aph/8134869-ghash-1/ > > seems to have TABs embedded in it. I thought this was verboten? It is. I did this on a new box, and I have not imported my editor settings; mea culpa. I will untabify before pushing. Thanks, Andrew. From vladimir.kozlov at oracle.com Wed Sep 2 16:19:33 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 2 Sep 2015 09:19:33 -0700 Subject: RFR(S) 8134898: Small fixes found during JVMCI work In-Reply-To: <55E6B5CE.9010901@oracle.com> References: <55E63BD5.8030802@oracle.com> <55E67349.10407@oracle.com> <55E678E4.5090500@oracle.com> <55E6B5CE.9010901@oracle.com> Message-ID: <55E72195.4010005@oracle.com> Thank you, Bertrand Your suggestion is better, I will use it. I will add comment to explain why we need the field when CHECK_UNHANDLED_OOPS is undefined. Thanks, Vladimir On 9/2/15 1:39 AM, Bertrand Delsart wrote: > On 02/09/2015 06:19, Vladimir Kozlov wrote: >> > To resolve the incompatibility, you can simply remove >> > CHECK_UNHANDLED_OOPS around the field in Thread, and put it under >> > #ifndef PRODUCT. >> >> Wonderful! :) I thought it requires #ifdef CHECK_UNHANDLED_OOPS. >> >> Please, look on new changes: >> >> http://cr.openjdk.java.net/~kvn/8134898/webrev.01/ > > Looks good. > > Small suggestion (feel free to ignore it). Might be even better to do: > > // Support for Unhandled Oop detection > -#ifdef CHECK_UNHANDLED_OOPS > +#if defined(CHECK_UNHANDLED_OOPS) || defined(ASSERT) > private: > UnhandledOops* _unhandled_oops; > +#endif > +#ifdef CHECK_UNHANDLED_OOPS > public: > UnhandledOops* unhandled_oops() { return _unhandled_oops; } > // Mark oop safe for gc. It may be stack allocated but won't move. > > This version: > - does not prevent someone from building a PRODUCT VM with CHECK_HANDLED_OOPS if needed > - ensures someone does not call unhandled_oops() in debug VMs when CHECK_UNHANDLED_OOPS is not defined > > Bertrand (not a Reviewer). > >> >> Thanks, >> Vladimir >> >> On 9/1/15 8:55 PM, Coleen Phillimore wrote: >>> >>> >>> On 9/1/15 7:59 PM, Vladimir Kozlov wrote: >>>> http://cr.openjdk.java.net/~kvn/8134898/webrev/ >>>> https://bugs.openjdk.java.net/browse/JDK-8134898 >>>> >>>> During review of JVMCI changes we were asked to push not related small >>>> changes separately. >>>> >>>> Removed jre/ from jdk paths in makefiles since jdk9 does not have it >>>> anymore. >>>> Solaris vm.make and buildtree.make missed changes other unix platforms >>>> have. >>>> Fix type in doc. >>>> Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to >>>> do that. >>> Adding -DCHECK_UNHANDLED_OOPS to the debug build adds a level of >>> indirection to the oops that are somewhat annoying during debugging: >>> >>> eg. >>> >>> 692 void java_lang_Class::set_component_mirror(oop java_class, oop >>> comp_mirror) { >>> 693 assert(_component_mirror_offset != 0, "must be set"); >>> 694 java_class->obj_field_put(_component_mirror_offset, >>> comp_mirror); >>> 695 } >>> >>> (gdb) print java_class >>> $7 = {_o = 0x8cb003b8} >>> (gdb) print java_mirror >>> No symbol "java_mirror" in current context. >>> (gdb) print comp_mirror >>> $8 = {_o = 0x8cb001b0} >>> (gdb) call hsfind(java_class._o) >>> warning: Using non-standard conversion to match function hsfind to >>> supplied arguments >>> >>> "Executing hsfind" >>> 0x000000008cb003b8 is an oop >>> java.lang.Class >>> - klass: 'java/lang/Class' >>> - ---- fields (total size 13 words): >>> ... >>> >>> >>> vs. >>> >>> 693 assert(_component_mirror_offset != 0, "must be set"); >>> (gdb) print java_class >>> $1 = (oop) 0x8cb003b8 >>> (gdb) print comp_mirror >>> $2 = (oop) 0x8cb001b0 >>> (gdb) print hsfind(java_class) >>> warning: Using non-standard conversion to match function hsfind to >>> supplied arguments >>> >>> "Executing hsfind" >>> 0x000000008cb003b8 is an oop >>> java.lang.Class >>> - klass: 'java/lang/Class' >>> - ---- fields (total size 13 words): >>> ... >>> >>> >>> The GC people deal with far more 'oop' objects in their debugging, which >>> is why they asked me originally not to enable CHECK_UNHANDLED_OOPS in >>> the debug build. >>> >>> To resolve the incompatibility, you can simply remove >>> CHECK_UNHANDLED_OOPS around the field in Thread, and put it under >>> #ifndef PRODUCT. >>> >>> Coleen >>>> It is the only incompatibility in memory layout between fastdebug and >>>> debug VM. We maintained such compatibility for long time and it was >>>> broken with 7195622 changes. >>>> For AOT project it is very important to keep compatibility because we >>>> use fastdebug VM for JAOTC tool to compile aot code and we want to be >>>> able use debug VM to debug execution of aot code. >>>> Note, using debug VM with JAOTC will take 5-10 times longer to compile. >>>> >>>> Thanks, >>>> Vladimir >>> > > From sangheon.kim at oracle.com Thu Sep 3 00:33:23 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 2 Sep 2015 17:33:23 -0700 Subject: RFR 8134972: [BACKOUT] GC: implement ranges (optionally constraints) for those flags that have them missing Message-ID: <55E79553.4010609@oracle.com> Hi all, Please review this patch for BACKOUT of command-line validation for GC flags. JDK-8078555 breaks some range test cases. Issue to backout: https://bugs.openjdk.java.net/browse/JDK-8078555 CR: https://bugs.openjdk.java.net/browse/JDK-8134972 Webrev: http://cr.openjdk.java.net/~sangheki/8134972/webrev.00/ Thanks, Sangheon From jesper.wilhelmsson at oracle.com Thu Sep 3 00:44:24 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Thu, 3 Sep 2015 02:44:24 +0200 Subject: RFR 8134972: [BACKOUT] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55E79553.4010609@oracle.com> References: <55E79553.4010609@oracle.com> Message-ID: <55E797E8.1030109@oracle.com> Looks good! /Jesper Den 3/9/15 kl. 02:33, skrev sangheon.kim: > Hi all, > > Please review this patch for BACKOUT of command-line validation for GC flags. > JDK-8078555 breaks some range test cases. > > Issue to backout: https://bugs.openjdk.java.net/browse/JDK-8078555 > > CR: https://bugs.openjdk.java.net/browse/JDK-8134972 > Webrev: http://cr.openjdk.java.net/~sangheki/8134972/webrev.00/ > > Thanks, > Sangheon From tobias.hartmann at oracle.com Thu Sep 3 09:33:49 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 03 Sep 2015 11:33:49 +0200 Subject: [9] RFR(S): 8080999: MemoryPoolMXBean.getUsageThresholdCount() returns incorrect value Message-ID: <55E813FD.8050207@oracle.com> Hi, please review the following patch. https://bugs.openjdk.java.net/browse/JDK-8080999 http://cr.openjdk.java.net/~thartmann/8080999/webrev.00/ Problem: The compiler test [1] sets a maximum usage threshold for a code heap memory pool, allocates enough memory to hit the threshold and checks if the threshold counter was incremented (see 'CodeCacheUtils.hitUsageThreshold' [2]). On success, the allocated memory is freed and we start over again. The counter should only be incremented the first time the threshold is crossed. Subsequent threshold hits are only counted if the usage fell below the threshold value. The test times out waiting for the threshold counter to be incremented. The problem is that there is a race between SensorInfo::set_gauge_sensor_level() which checks if the sensor should be triggered and SensorInfo::clear() which resets the sensor once the usage falls below the threshold value. The race occurs if SensorInfo::process_pending_requests() invokes SensorInfo::clear() because there is a pending request to clear the sensor. In the meantime SensorInfo::set_gauge_sensor_level() may be called because the sensor is triggered again. We now have to remove the pending clear request because the sensor should be activated (see [3]). However, since SensorInfo::clear() does not check the value of '_pending_clear_count' again, it still clears the sensor. The sensor may then be triggered again although the threshold value was only crossed once. As a result, the output of getUsageThresholdCount() is higher than expected. Here is the detailed trace of events (the info in brackets is the output of SensorInfo::print): ... [on count = 1 pending_triggers = 0 pending_clears = 0] WB.freeCodeBlob() LowMemoryDetector::detect_low_memory() SensorInfo::set_gauge_sensor_level() is_below_low -> _pending_clear_count++; [on count = 1 pending_triggers = 0 pending_clears = 1] WB.allocateCodeBlob() LowMemoryDetector::detect_low_memory() SensorInfo::set_gauge_sensor_level() is_over_high -> _pending_trigger_count++; _pending_clear_count = 0; <-- remove pending clear request because the sensor is triggered again [on count = 1 pending_triggers = 1 pending_clears = 0] LowMemoryDetector::process_sensor_changes() SensorInfo::process_pending_requests() <-- still sees the old _pending_clear_count == 1 because there is a race SensorInfo::clear() [off count = 2 pending_triggers = 0 pending_clears = 0] WB.fullGC() LowMemoryDetector::detect_low_memory() SensorInfo::set_gauge_sensor_level() is_over_high -> _pending_trigger_count++; [off count = 2 pending_triggers = 1 pending_clears = 0] At this point the threshold count should be 2 because the threshold was only hit once. However, we have a pending trigger request. Now there are tow possibilities: 1) We check the counter before the pending trigger is processed: We continue because as expected the count is 2 but we fail later because the overall count does not match the expected value (RuntimeException: Unexpected threshold usage count (assert failed: 11 == 10)). 2) We check after the pending trigger is processed: The counter is 3 and we time out waiting for it to be 2. Solution: I added asserts to the code to make sure that such race conditions are detected. I changed the implementation of SensorInfo::clear() to acquire the Service_lock and bail out if _pending_clear_count was reset in the meantime (i.e., if we lost a race to SensorInfo::set_gauge_sensor_level()). I also added a missing 335 _sensor_count += count; to SensorInfo::clear() because it may trigger counter increments as well. Testing: - 15k runs of failing testcase - JPRT Thanks, Tobias [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/UsageThresholdExceededTest.java [2] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/CodeCacheUtils.java#l50 [3] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/src/share/vm/services/lowMemoryDetector.cpp#l222 From david.holmes at oracle.com Thu Sep 3 11:22:08 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 3 Sep 2015 21:22:08 +1000 Subject: RFR(S) 8134898: Small fixes found during JVMCI work In-Reply-To: <55E63BD5.8030802@oracle.com> References: <55E63BD5.8030802@oracle.com> Message-ID: <55E82D60.1080108@oracle.com> On 2/09/2015 9:59 AM, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8134898/webrev/ > https://bugs.openjdk.java.net/browse/JDK-8134898 > > During review of JVMCI changes we were asked to push not related small > changes separately. > > Removed jre/ from jdk paths in makefiles since jdk9 does not have it > anymore. JDK9 also does not have rt.jar anymore so much of what is in this area of make/Makefile seems unused and unworking. > Solaris vm.make and buildtree.make missed changes other unix platforms > have. make/solaris/makefiles/vm.make Does Shark even exist on Solaris? Has this been tested? Thanks, David > Fix type in doc. > Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to do > that. > It is the only incompatibility in memory layout between fastdebug and > debug VM. We maintained such compatibility for long time and it was > broken with 7195622 changes. > For AOT project it is very important to keep compatibility because we > use fastdebug VM for JAOTC tool to compile aot code and we want to be > able use debug VM to debug execution of aot code. > Note, using debug VM with JAOTC will take 5-10 times longer to compile. > > Thanks, > Vladimir From vladimir.kozlov at oracle.com Thu Sep 3 16:20:31 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 3 Sep 2015 09:20:31 -0700 Subject: RFR(S) 8134898: Small fixes found during JVMCI work In-Reply-To: <55E82D60.1080108@oracle.com> References: <55E63BD5.8030802@oracle.com> <55E82D60.1080108@oracle.com> Message-ID: <55E8734F.4090205@oracle.com> On 9/3/15 4:22 AM, David Holmes wrote: > On 2/09/2015 9:59 AM, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8134898/webrev/ >> https://bugs.openjdk.java.net/browse/JDK-8134898 >> >> During review of JVMCI changes we were asked to push not related small >> changes separately. >> >> Removed jre/ from jdk paths in makefiles since jdk9 does not have it >> anymore. > > JDK9 also does not have rt.jar anymore so much of what is in this area of make/Makefile seems unused and unworking. > >> Solaris vm.make and buildtree.make missed changes other unix platforms >> have. > > make/solaris/makefiles/vm.make > > Does Shark even exist on Solaris? Has this been tested? I don't know. I made this change for consistency of makefiles on all unix platforms. Main change in the file was fix for Src_Files_EXCLUDE because exclusion of jvmci files did not work. Thanks, Vladimir > > Thanks, > David > > >> Fix type in doc. >> Added -DCHECK_UNHANDLED_OOPS to debug build. I very strongly want to do >> that. >> It is the only incompatibility in memory layout between fastdebug and >> debug VM. We maintained such compatibility for long time and it was >> broken with 7195622 changes. >> For AOT project it is very important to keep compatibility because we >> use fastdebug VM for JAOTC tool to compile aot code and we want to be >> able use debug VM to debug execution of aot code. >> Note, using debug VM with JAOTC will take 5-10 times longer to compile. >> >> Thanks, >> Vladimir From nobeh5 at gmail.com Sat Sep 5 12:07:47 2015 From: nobeh5 at gmail.com (Behrooz Nobakht) Date: Sat, 5 Sep 2015 14:07:47 +0200 Subject: Trigger OOM Killer with a minimal example? Message-ID: Hi, I have been trying to create a minimal example/setting to be able to trigger the Unix OOM Killer. Here is the general scenario that I follow: 1. Setup up a machine with `X` MB of memory 2. Configure the machine to have `Y` MB of swap space 3. Run the Java example 4. Observe in `dmesg` the killing happens For (1), I use a small virtual machine on VB using Ubuntu 14.04.03-server with X ~= 1000MB memory ```3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux``` For (2), I use a small script to make the necessary swap space with Y = 64MB https://gist.github.com/nobeh/8924c2f5292d54b3fba9#file-create-swap-sh For (3), I use a small Java example at: https://gist.github.com/nobeh/8924c2f5292d54b3fba9#file-killer-java and this is how I run the program (to preserve our live environment settings): https://gist.github.com/nobeh/8924c2f5292d54b3fba9#file-run-sh I have three "types" of observations: Type1: OOM Killer triggered Sometimes, the example indeed leads to triggering OOM Killer with the following messages in `dmesg`: ``` [ 8072.902446] Out of memory: Kill process 2619 (java) score 865 or sacrifice child [ 8072.904516] Killed process 2619 (java) total-vm:2214952kB, anon-rss:885596kB, file-rss:0kB ``` Type2: VM failure with an hs_ error log Other times the run leads to the following error message: ``` Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000f4400000, 111149056, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 111149056 bytes for committing reserved memory. # An error report file with more information is saved as: # /home/behrooz/hs_err_pid3030.log ``` I can provide the logs if necessary. Type3: OutOfMemoryError: Java heap space And, sometimes, the program actually exits with an OutOfMemoryError ``` [ 1825] *** ERROR *** java.lang.OutOfMemoryError: Java heap space at Killer$KillerArrays.(Killer.java:21) at Killer.main(Killer.java:51) ``` Having said the above, I understand that "consistently" triggering OOM Killer can be hard considering OS configurations and runtime state. If it is possible to do this, what am I missing? How can I "consistently" make the program killed by OOM Killer? Thanks, Behrooz From marcus.larsson at oracle.com Mon Sep 7 13:33:04 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 7 Sep 2015 15:33:04 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging Message-ID: <55ED9210.6030904@oracle.com> Hi, Please review the following patch adding the unified logging framework to hotspot. JEP: https://bugs.openjdk.java.net/browse/JDK-8046148 Webrev: http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ See the JEP description for a general overview of the new feature. Below are some notes on the implementation. The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. A simple jtreg test is included in this patch. Additional tests will be added at a later stage. Thanks, Marcus From kirk.pepperdine at gmail.com Mon Sep 7 15:53:30 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Mon, 7 Sep 2015 17:53:30 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> Hi all, I thought that the idea behind tagging was to eliminate levels. Kind regards, Kirk Pepperdine > On Sep 7, 2015, at 3:33 PM, Marcus Larsson wrote: > > Hi, > > Please review the following patch adding the unified logging framework to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). > > The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. > > A simple jtreg test is included in this patch. Additional tests will be added at a later stage. > > Thanks, > Marcus From kirk.pepperdine at gmail.com Mon Sep 7 16:00:10 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Mon, 7 Sep 2015 18:00:10 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <48B932BA-4C3B-4B30-8941-1382EA1D590E@kodewerk.com> Hi, I reviewed the JEP, something that I?d not done in quite some time. JEP is much stronger but unfortunately it still contains levels. Peter Lawyer?s Chronicle (github) is a great example on how to provide hi volume journaling/logging with very little impact on performance. I think there are some great lessons to be learned by looking at it. Kind regards, Kirk Pepperdine > On Sep 7, 2015, at 3:33 PM, Marcus Larsson wrote: > > Hi, > > Please review the following patch adding the unified logging framework to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). > > The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. > > A simple jtreg test is included in this patch. Additional tests will be added at a later stage. > > Thanks, > Marcus From cnewland at chrisnewland.com Mon Sep 7 16:05:30 2015 From: cnewland at chrisnewland.com (Chris Newland) Date: Mon, 7 Sep 2015 17:05:30 +0100 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging Message-ID: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> Hi Marcus, Thanks for making this visible. Could you let me know how this will affect diagnostic VM logging such as LogCompilation and PrintAssembly? Once implemented, will the current output formats still be available? Disclosure: I'm the author of a log processing tool (JITWatch [1]) that consumes LogCompilation, PrintAssembly, and TraceClassLoading. I've had a cursory grep through the webrev and it doesn't seem to affect these but I'll apply the changes tonight and have a proper look. Thanks, Chris [1] https://github.com/AdoptOpenJDK/jitwatch On Mon, September 7, 2015 14:33, Marcus Larsson wrote: > Hi, > > > Please review the following patch adding the unified logging framework > to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > > See the JEP description for a general overview of the new feature. Below > are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the > unified logging framework. The main entry point is log.hpp, which contains > the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template > arguments to the Log class. Every combination of tags used in a log call > has a corresponding LogTagSet instance, which keeps a track of all the > outputs it should write the log message to (and their levels). Having tags > as template arguments allows mapping directly from a set of tags to the > LogTagSet instance, which means that the overhead for disabled > logging should be low. Currently each log message can be tagged with up to > 5 tags, but this can be increased if ever required (and with C++11's > variadic templates the limit can be removed completely). > > The LogConfiguration class keeps track of configured outputs (stdout, > stderr, and possible file outputs). Configuration is done either by command > line arguments (-Xlog) or by DCMD. Both methods will in turn use the > LogConfiguration class to perform the parsing & configuration. This > configuration includes iterating over all LogTagSet instances and updating > them accordingly. The LogTagLevelExpression class is used to represent the > selection of tags and levels for a given configuration request (the > "what"-expression). > > > The LogDecorators class contains a selection of decorators. Instances of > this class is kept in LogTagSet to track what decorators to use (this is > the union of all decorators used by its outputs). Each log call will > create a LogDecorations instance (note: different classes), which will > contain the actual decoration strings for the log message. These > decorations are used for each output the tagset is set to log on, and are > then discarded. > > The LogPrefix class allows messages of specific sets of tags to be > prefixed. The prefix should supply a printf-style format with argument. > (This allows GC logging to prefix messages of certain tagsets with > GCId.) Prefixes are implemented using template specializations based on > the specified tags, with the general/unspecialized case giving an empty > prefix. > > The LogOutput class defines the interface for all types of log outputs. > LogFileStreamOutput corresponds to FILE* stream based log outputs. > LogFileOutput builds on this and adds the file management and log > rotation support. > > A simple jtreg test is included in this patch. Additional tests will be > added at a later stage. > > Thanks, > Marcus > > From marcus.larsson at oracle.com Tue Sep 8 09:08:54 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Tue, 8 Sep 2015 11:08:54 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> Message-ID: <55EEA5A6.8020602@oracle.com> Hi, On 2015-09-07 17:53, Kirk Pepperdine wrote: > Hi all, > > I thought that the idea behind tagging was to eliminate levels. I believe the major concern with the first proposal was that not every log message could be classified in a strictly hierarchical manner. Some messages belong to multiple components, which is a use case the hierarchy of loggers doesn't support. This is why the tag based solution was proposed instead. Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now. Thanks, Marcus > > Kind regards, > Kirk Pepperdine > >> On Sep 7, 2015, at 3:33 PM, Marcus Larsson wrote: >> >> Hi, >> >> Please review the following patch adding the unified logging framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will be added at a later stage. >> >> Thanks, >> Marcus From kirk.pepperdine at gmail.com Tue Sep 8 10:45:42 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 8 Sep 2015 12:45:42 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EEA5A6.8020602@oracle.com> References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> Message-ID: Hi, Indeed, not all messages are hierarchical. This is one of the reasons I originally questioned the use of log levels. For example, debugging and informational logs may come from the same source and I certainly wouldn?t want to be forced mix them to satisfy a forced structure. Is the intention you can use either or? I ask because I?m not sure that mixing them with tags will alleviate the problems I commonly see when tuning applications in the field. > > Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now. I would like to believe that this work is important enough to get right and that it?s never too late for fundamental changes up until it is integrated. I would like to believe that there are back out strategies in the cases where implementations show unintended consequences. Kind regards, Kirk Pepperdine From marcus.larsson at oracle.com Tue Sep 8 11:20:33 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Tue, 8 Sep 2015 13:20:33 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> Message-ID: <55EEC481.7030605@oracle.com> Hi Chris, On 2015-09-07 18:05, Chris Newland wrote: > Hi Marcus, > > Thanks for making this visible. > > Could you let me know how this will affect diagnostic VM logging such as > LogCompilation and PrintAssembly? The current patch doesn't modify any existing logging in the JVM, it only contains the new framework. However, follow-up JEPs and RFEs will convert existing logging to use the new framework, at which point you will see changes in how the logging works. Given that the actual log messages are left untouched during migration, and the tty-print calls are just converted into log calls tagged appropriately, these changes should not significantly affect you. It is very dependent on the tags and level that are chosen to replace the previous flag-based configuration. There might be cases where additional log messages are merged into the same tag and level category as some previously flag-controlled output. For example, PrintAssembly-output might be merged with logging from some other logging flag. This really depends on the work to migrate to the new framework. The method to enable logging will be changed from the individual flags to the unified log configuration scheme (-Xlog:... or by diagnostic commands, see the JEP for more info). Unless configured otherwise, the new log framework prefixes the log messages with decorators like uptime, level and tags (these are the default decorators). > > Once implemented, will the current output formats still be available? Currently there is no specific support for XML output in the unified logging framework. Output can be directed either to stdout, stderr or plain text files. I'm not entirely sure how the migration of the XML logging will be done. Perhaps it will be preceded by an RFE to add better support for XML outputs in the unified logging framework, or it might just be treated as a regular text file. Thanks, Marcus > > Disclosure: I'm the author of a log processing tool (JITWatch [1]) that > consumes LogCompilation, PrintAssembly, and TraceClassLoading. > > I've had a cursory grep through the webrev and it doesn't seem to affect > these but I'll apply the changes tonight and have a proper look. > > Thanks, > > Chris > > [1] https://github.com/AdoptOpenJDK/jitwatch > > On Mon, September 7, 2015 14:33, Marcus Larsson wrote: >> Hi, >> >> >> Please review the following patch adding the unified logging framework >> to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> >> See the JEP description for a general overview of the new feature. Below >> are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which contains >> the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log call >> has a corresponding LogTagSet instance, which keeps a track of all the >> outputs it should write the log message to (and their levels). Having tags >> as template arguments allows mapping directly from a set of tags to the >> LogTagSet instance, which means that the overhead for disabled >> logging should be low. Currently each log message can be tagged with up to >> 5 tags, but this can be increased if ever required (and with C++11's >> variadic templates the limit can be removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by command >> line arguments (-Xlog) or by DCMD. Both methods will in turn use the >> LogConfiguration class to perform the parsing & configuration. This >> configuration includes iterating over all LogTagSet instances and updating >> them accordingly. The LogTagLevelExpression class is used to represent the >> selection of tags and levels for a given configuration request (the >> "what"-expression). >> >> >> The LogDecorators class contains a selection of decorators. Instances of >> this class is kept in LogTagSet to track what decorators to use (this is >> the union of all decorators used by its outputs). Each log call will >> create a LogDecorations instance (note: different classes), which will >> contain the actual decoration strings for the log message. These >> decorations are used for each output the tagset is set to log on, and are >> then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with argument. >> (This allows GC logging to prefix messages of certain tagsets with >> GCId.) Prefixes are implemented using template specializations based on >> the specified tags, with the general/unspecialized case giving an empty >> prefix. >> >> The LogOutput class defines the interface for all types of log outputs. >> LogFileStreamOutput corresponds to FILE* stream based log outputs. >> LogFileOutput builds on this and adds the file management and log >> rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will be >> added at a later stage. >> >> Thanks, >> Marcus >> >> > From bengt.rutisson at oracle.com Tue Sep 8 11:35:12 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 08 Sep 2015 13:35:12 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place Message-ID: <55EEC7F0.1060701@oracle.com> Hi everyone, This is mostly a GC related patch, but it adds a field to the Thread class, so I'm sending this out on the broader hotspot-dev list. Could I have a couple of reviews for this patch? http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8134953 Currently the GC ID, that is used for event tracing and logging, is stored in the GCTracer object. That has caused some minor problems since there are multiple GCTracers active at the same time. The correct GC IDs need to be passed around in many places. For some upcoming GC logging changes I would like to have a more consistent way of finding the currently active GC ID. I've played around with a couple of different solutions, but eventually I found that the simplest solution is to store the current GC ID in the thread. That way there is a single way to look it up in all places. It is also fairly straight forward to set it up. I've reworked the GCId class a bit to support this and I've introduced a GCIdMark class that is a scoped object that helps to set up and tear down a current GC ID. By moving the GC ID out from the GCTracer class I got rid of a few minor issues as well. One is that we no longer need to keep track of the G1 "aborted concurrent GC ID". That was necessary before since we did logging *after* we had told the corresponding GCTracer that the concurrent cycle was aborted and it had thrown its GC ID away. Now the GC ID can stay in scope until we have completed all logging. Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC used to have to create a new GC ID since their logging was done before we had the correct GCTracer set up. Now the GC ID can be available early enough for this logging to be using the same (and correct) GC ID as the rest of the GC. Same for HeapDumpAfterFullGC and PrintClassHistogramAfterFullGC. I've added an uint to the Thread class to keep track of the currently active GC ID. In the current code there are actually only NamedThreads that need the GC ID. So, I'm not sure where the best place is for this field is. But it seems like the Thread class contains most of the data, even data that is only used by some subclasses, so I opted for putting the field in Thread as opposed to in NamedThread. This opens up for possible future extensions when Java threads may do part of the GC work. However, I'm open to moving the field down to NamedThread if that is seen as better. In the GCTracer class there were many asserts that were checking that the GC ID was properly set up. Mostly these assert verify that start/end is called correctly and that the other methods are called inside of the start/end scope. Now that the GC ID is moved out of the GCTracer class these asserts had to be dealt with. I figured there were three ways to handle it; just remove them, replace them with check that the GC ID from the current thread is correct, or implement a new status field to keep track of GC start/end state. Personally I'm not sure these asserts are valuable enough to spend time on, so I went for the first approach: removing them. I don't think making them use the thread GC ID will be appropriate. But if anyone feels strongly about these asserts I can implement a separate start/end state. There are a few "Tasks" in the GC code that are executed by worker threads. To make the worker threads use the correct GC ID I make sure that the Task constructors copy the GC ID from the initiating thread into a local variable. When the task is executed in its worker thread it sets up the GC ID based on the local variable. The proposed change does not alter any logging (except for the small bug fix for HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. This means that no existing tests need to be updated. In particular the test/gc/logging/TestGCId.java test passes even after these changes. A big thanks to Per, who pre-reviewed these changes and came with some really good feedback. Thanks, Bengt From claes.redestad at oracle.com Tue Sep 8 11:42:52 2015 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 08 Sep 2015 13:42:52 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> Message-ID: <55EEC9BC.3030005@oracle.com> On 2015-09-08 12:45, Kirk Pepperdine wrote: > Hi, > > Indeed, not all messages are hierarchical. This is one of the reasons I originally questioned the use of log levels. For example, debugging and informational logs may come from the same source and I certainly wouldn?t want to be forced mix them to satisfy a forced structure. Is the intention you can use either or? I ask because I?m not sure that mixing them with tags will alleviate the problems I commonly see when tuning applications in the field. > >> Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now. > I would like to believe that this work is important enough to get right and that it?s never too late for fundamental changes up until it is integrated. I would like to believe that there are back out strategies in the cases where implementations show unintended consequences. Can you give an example of what you believe will go wrong with allowing a mix of levels and tags and how removing levels from the JEP would improve things? Having high-level tags which interact with the standard "levels" seem like a natural fit to me, while the framework proposed in the JEP make it possible for each VM team to support ever more detailed tags to each subsystem where that might be appropriate. For better or worse it adds a degree of freedom. Is your concern that we need more granular tags going forward to help zooming in on specific issues, and that levels for such niche tags might seem/feel superfluous? Thanks! /Claes > > Kind regards, > Kirk Pepperdine > From kirk.pepperdine at gmail.com Tue Sep 8 11:55:13 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 8 Sep 2015 13:55:13 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EEC481.7030605@oracle.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> Message-ID: <83E03D9A-E125-41CD-B863-60B14DD1AAE2@gmail.com> Hi, > >> >> Once implemented, will the current output formats still be available? > > Currently there is no specific support for XML output in the unified logging framework. Output can be directed either to stdout, stderr or plain text files. I'm not entirely sure how the migration of the XML logging will be done. Perhaps it will be preceded by an RFE to add better support for XML outputs in the unified logging framework, or it might just be treated as a regular text file. My question is, can output from different tags be directed to different files? Regards, Kirk From marcus.larsson at oracle.com Tue Sep 8 12:16:23 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Tue, 8 Sep 2015 14:16:23 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <83E03D9A-E125-41CD-B863-60B14DD1AAE2@gmail.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <83E03D9A-E125-41CD-B863-60B14DD1AAE2@gmail.com> Message-ID: <55EED197.1040500@oracle.com> Hi, On 2015-09-08 13:55, Kirk Pepperdine wrote: > Hi, >>> Once implemented, will the current output formats still be available? >> Currently there is no specific support for XML output in the unified logging framework. Output can be directed either to stdout, stderr or plain text files. I'm not entirely sure how the migration of the XML logging will be done. Perhaps it will be preceded by an RFE to add better support for XML outputs in the unified logging framework, or it might just be treated as a regular text file. > My question is, can output from different tags be directed to different files? Yes. The log configuration allows you to specify exactly which tags on what level should go to which output. This is all described in the JEP. Marcus > > Regards, > Kirk > From kirk.pepperdine at gmail.com Tue Sep 8 12:47:07 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 8 Sep 2015 14:47:07 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EEC9BC.3030005@oracle.com> References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> Message-ID: > On Sep 8, 2015, at 1:42 PM, Claes Redestad wrote: > > > > On 2015-09-08 12:45, Kirk Pepperdine wrote: >> Hi, >> >> Indeed, not all messages are hierarchical. This is one of the reasons I originally questioned the use of log levels. For example, debugging and informational logs may come from the same source and I certainly wouldn?t want to be forced mix them to satisfy a forced structure. Is the intention you can use either or? I ask because I?m not sure that mixing them with tags will alleviate the problems I commonly see when tuning applications in the field. >> >>> Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now. >> I would like to believe that this work is important enough to get right and that it?s never too late for fundamental changes up until it is integrated. I would like to believe that there are back out strategies in the cases where implementations show unintended consequences. > > Can you give an example of what you believe will go wrong with allowing a mix of levels and tags and how removing levels from the JEP would improve things? I?m slowly working through the patch but I?ve not had time to get through all of it yet so some things mentioned here might be in the source already? but here are some comments from experiences. As you have discovered, you can?t define a tag system using levels. That said, if you have tags you can define levels but you don?t need to explicitly support them, it just come for free out of the box. So I would argue both the spec and implementation would be simplified by moving to a tag system. More over, it doesn?t preclude someone defining a hierarchical system if them so need to have one. Next, I think there are two points of view that you have to consider. Usability for the producers and usability for the consumers. From a consumer POV what I would like to see is a logging system that promotes stability in the output. Using broad ill defined categories as they are currently defined in the JDK have historically lead to unstable output. This in turn breaks down stream tooling that maybe used to help triage system out in the wild. How does this happen? Well what gets put into a level is completely arbitrary. What goes into warning, fine, finest.. who knows? More to the point, the producers often have to make a quick decision and in their cubicles they often don?t see the entire downstream tooling chains that they are affecting when they do this even when the information being added it very valuable. I?m going to suggest a system, maybe not *the* system or even the *best* system. It?s just a simple system that people have been using in pub/sub messaging systems for years. I think the technique applies because logging is a form of journalling which is just another form of messaging (hence I suggested people looking at Peter Lawyer?s Chronicle which is designed to be used in HFT applications). We can also use ObjectName (jmx) to define tags. If you have a tag, let say, gc.agetable, it?s pretty clear what you should get. Also that output is very likely to remain stable over time. In fact, my experience with PrintTenuringDistrubtion is just that. The output has remained stable for years. I cannot say that about any overly broad category of GC logging such as PrintGCDetails (which I would equate to a level). If I have another tag say, gc.collection.scavenge I should get information about scavenges. Again purpose is very clear from both a producer/consumer POV. I can take age data or scavenge or both or gc.collection.* to get all sub categories for gc.collection. This looks like levels and indeed could be used to implement levels but it?s based on categories (tags). At any rate, the taxonomy is clearer and should offer guidelines that makes it easier for the developer to push the information out on the right channel while offering better stability needed for downstream tooling. > > Having high-level tags which interact with the standard "levels" seem like a natural fit to me, while the framework proposed in the JEP make it possible for each VM team to support ever more detailed tags to each subsystem where that might be appropriate. For better or worse it adds a degree of freedom. Is your concern that we need more granular tags going forward to help zooming in on specific issues, and that levels for such niche tags might seem/feel superfluous? If the system has been built with tags on levels then I would suggest the system has been built upside down or inside out, however you want to describe it.. in the same way that JDK logging was build upside down (first problem, super class knows about subclasses and that is a primary source of classloader leaks and that is just the first issue). Kind regards, Kirk From kirk.pepperdine at gmail.com Tue Sep 8 12:48:59 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 8 Sep 2015 14:48:59 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EED197.1040500@oracle.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <83E03D9A-E125-41CD-B863-60B14DD1AAE2@gmail.com> <55EED197.1040500@oracle.com> Message-ID: <361822B8-3DDE-42B6-88DA-EE7499A6945A@gmail.com> > On Sep 8, 2015, at 2:16 PM, Marcus Larsson wrote: > > Hi, > > On 2015-09-08 13:55, Kirk Pepperdine wrote: >> Hi, >>>> Once implemented, will the current output formats still be available? >>> Currently there is no specific support for XML output in the unified logging framework. Output can be directed either to stdout, stderr or plain text files. I'm not entirely sure how the migration of the XML logging will be done. Perhaps it will be preceded by an RFE to add better support for XML outputs in the unified logging framework, or it might just be treated as a regular text file. >> My question is, can output from different tags be directed to different files? > > Yes. The log configuration allows you to specify exactly which tags on what level should go to which output. This is all described in the JEP. Ok, thanks.. I didn?t get that sense from the JEP. Then I (with all due respect) would suggest that this framework is being build upside down (as per the email I just previously sent). Kind regards, Kirk From per.liden at oracle.com Tue Sep 8 13:23:30 2015 From: per.liden at oracle.com (Per Liden) Date: Tue, 8 Sep 2015 15:23:30 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <55EEC7F0.1060701@oracle.com> References: <55EEC7F0.1060701@oracle.com> Message-ID: <55EEE152.3030404@oracle.com> Hi Bengt, On 2015-09-08 13:35, Bengt Rutisson wrote: > > Hi everyone, > > This is mostly a GC related patch, but it adds a field to the Thread > class, so I'm sending this out on the broader hotspot-dev list. > > Could I have a couple of reviews for this patch? > > http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8134953 > Looks good. I think this is a nice simplification, especially for G1's concurrent parts. Just two comments: genCollectedHeap.cpp: --------------------- - GenCollectedHeap::do_collection() has two GCIdMarks, in different scopes. Do we really want that second mark? gcId.cpp: -------- - I think you might have left currentNamedthread() in there. You probably just want to use Thread::current() instead. cheers, /Per > Currently the GC ID, that is used for event tracing and logging, is > stored in the GCTracer object. That has caused some minor problems since > there are multiple GCTracers active at the same time. The correct GC IDs > need to be passed around in many places. > > For some upcoming GC logging changes I would like to have a more > consistent way of finding the currently active GC ID. I've played around > with a couple of different solutions, but eventually I found that the > simplest solution is to store the current GC ID in the thread. That way > there is a single way to look it up in all places. It is also fairly > straight forward to set it up. > > I've reworked the GCId class a bit to support this and I've introduced a > GCIdMark class that is a scoped object that helps to set up and tear > down a current GC ID. By moving the GC ID out from the GCTracer class I > got rid of a few minor issues as well. One is that we no longer need to > keep track of the G1 "aborted concurrent GC ID". That was necessary > before since we did logging *after* we had told the corresponding > GCTracer that the concurrent cycle was aborted and it had thrown its GC > ID away. Now the GC ID can stay in scope until we have completed all > logging. > > Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC used > to have to create a new GC ID since their logging was done before we had > the correct GCTracer set up. Now the GC ID can be available early enough > for this logging to be using the same (and correct) GC ID as the rest of > the GC. Same for HeapDumpAfterFullGC and PrintClassHistogramAfterFullGC. > > I've added an uint to the Thread class to keep track of the currently > active GC ID. In the current code there are actually only NamedThreads > that need the GC ID. So, I'm not sure where the best place is for this > field is. But it seems like the Thread class contains most of the data, > even data that is only used by some subclasses, so I opted for putting > the field in Thread as opposed to in NamedThread. This opens up for > possible future extensions when Java threads may do part of the GC work. > However, I'm open to moving the field down to NamedThread if that is > seen as better. > > In the GCTracer class there were many asserts that were checking that > the GC ID was properly set up. Mostly these assert verify that start/end > is called correctly and that the other methods are called inside of the > start/end scope. Now that the GC ID is moved out of the GCTracer class > these asserts had to be dealt with. I figured there were three ways to > handle it; just remove them, replace them with check that the GC ID from > the current thread is correct, or implement a new status field to keep > track of GC start/end state. Personally I'm not sure these asserts are > valuable enough to spend time on, so I went for the first approach: > removing them. I don't think making them use the thread GC ID will be > appropriate. But if anyone feels strongly about these asserts I can > implement a separate start/end state. > > There are a few "Tasks" in the GC code that are executed by worker > threads. To make the worker threads use the correct GC ID I make sure > that the Task constructors copy the GC ID from the initiating thread > into a local variable. When the task is executed in its worker thread it > sets up the GC ID based on the local variable. > > The proposed change does not alter any logging (except for the small bug > fix for > HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. This > means that no existing tests need to be updated. In particular the > test/gc/logging/TestGCId.java test passes even after these changes. > > > A big thanks to Per, who pre-reviewed these changes and came with some > really good feedback. > > Thanks, > Bengt From kirill.zhaldybin at oracle.com Tue Sep 8 16:37:40 2015 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Tue, 08 Sep 2015 19:37:40 +0300 Subject: RFR(XS): 8132980: Improve stability of whitebox methods getCodeBlob and freeCodeBlob Message-ID: <55EF0ED4.2030301@oracle.com> Hi! Could you please review the following patch for WB methods? I added checks for negative size and null address values (where applicable) for getCodeBlob, allocateCodeBlob and freeCodeBlob, added regression test. CR: https://bugs.openjdk.java.net/browse/JDK-8132980 Webrev: hotspot: http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132980/webrev.00/ Thanks, Kirill From volker.simonis at gmail.com Tue Sep 8 16:56:46 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 8 Sep 2015 18:56:46 +0200 Subject: RFR(XXS): 8135179: Fix conversion warning after 8067341 Message-ID: Hi, can somebody please review and sponsor the following trivial fix: http://cr.openjdk.java.net/~simonis/webrevs/2015/8135179/ https://bugs.openjdk.java.net/browse/JDK-8135179 Change "8067341: Modify PLAB sizing algorithm to waste less" introduced the following code: size_t const cur_plab_sz = (double)total_waste_allowed / G1LastPLABAverageOccupancy; which triggers a conversion warning with older versions of GCC and potentially other compilers as well: hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp: In member function 'virtual void G1EvacStats::adjust_desired_plab_sz()': hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp:96: warning: converting to 'size_t' from 'double' make[4]: *** [g1EvacStats.o] Error 1 The warning can be easily fixed as follows: size_t const cur_plab_sz = (sizte_t)((double)total_waste_allowed / G1LastPLABAverageOccupancy); Thank you and best regards, Volker PS: as a side note I want to mention that we are currently disabling '-Wconversion' for GCC 4.3 and later because it produces too many warnings which are by default treated as errors. All the warnings produced by '-Wconversion' are about narrowing conversions which may result in a potential data loss. It may be a good idea to fix all these implicit narrowing conversions and re-enable -Wconversion warning by default. But unfortunately there's quite some code which has to be changed in order to meet the requirements of '-Wconversion'. A quick research showed that there are currently 137 different implicit narrowing conversions spread over 1785 different code locations. For more details see "8135181: Re-enable '-Wconversion' for GCC 4.3 and later" (https://bugs.openjdk.java.net/browse/JDK-8135181) which I've opened to track the issue. What's the general opinion? Would it be worth wile to fix all these conversion warnings by inserting implicit casts? From ioi.lam at oracle.com Tue Sep 8 20:26:58 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 08 Sep 2015 13:26:58 -0700 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <55EF4492.6000904@oracle.com> Hi Marcus, I am eagerly waiting for this to implement a new logging feature in the VM. How much testing has been done on this patch? I can see only two locations where any logs are bring generated: + log_error(logging)("%s", errbuf); + log_error(logging)("Could not open log file '%s' (%s).\n", _file_name, strerror(errno)); Do you have a follow-on patch that's already in work to test the various functionalists of the Unified JVM Logging framework? I am asking so that I can know how much surprise to expect when I start using it. Thanks - Ioi On 9/7/15 6:33 AM, Marcus Larsson wrote: > Hi, > > Please review the following patch adding the unified logging framework > to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. > Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the > unified logging framework. The main entry point is log.hpp, which > contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template > arguments to the Log class. Every combination of tags used in a log > call has a corresponding LogTagSet instance, which keeps a track of > all the outputs it should write the log message to (and their levels). > Having tags as template arguments allows mapping directly from a set > of tags to the LogTagSet instance, which means that the overhead for > disabled logging should be low. Currently each log message can be > tagged with up to 5 tags, but this can be increased if ever required > (and with C++11's variadic templates the limit can be removed > completely). > > The LogConfiguration class keeps track of configured outputs (stdout, > stderr, and possible file outputs). Configuration is done either by > command line arguments (-Xlog) or by DCMD. Both methods will in turn > use the LogConfiguration class to perform the parsing & configuration. > This configuration includes iterating over all LogTagSet instances and > updating them accordingly. The LogTagLevelExpression class is used to > represent the selection of tags and levels for a given configuration > request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances > of this class is kept in LogTagSet to track what decorators to use > (this is the union of all decorators used by its outputs). Each log > call will create a LogDecorations instance (note: different classes), > which will contain the actual decoration strings for the log message. > These decorations are used for each output the tagset is set to log > on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be > prefixed. The prefix should supply a printf-style format with > argument. (This allows GC logging to prefix messages of certain > tagsets with GCId.) Prefixes are implemented using template > specializations based on the specified tags, with the > general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log > outputs. LogFileStreamOutput corresponds to FILE* stream based log > outputs. LogFileOutput builds on this and adds the file management and > log rotation support. > > A simple jtreg test is included in this patch. Additional tests will > be added at a later stage. > > Thanks, > Marcus From ioi.lam at oracle.com Tue Sep 8 20:53:27 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 08 Sep 2015 13:53:27 -0700 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EF4492.6000904@oracle.com> References: <55ED9210.6030904@oracle.com> <55EF4492.6000904@oracle.com> Message-ID: <55EF4AC7.40809@oracle.com> Also, is it possible to combine the output of two tags into a single file, like this? -Xlog:gc:file=out.txt -Xlog:comp:file=out.txt this way, I can have both GC and Compiler logs that's in an (approximate) temporal order. The alternatives are: - write both to stdout -- but I need to filter out junk that I don't want - write to different files -- but I lose the relative ordering between the two sets of logs From the current patch, it seems like if I assign the same file name to two tags, fopen() will be called twice and the result will be platform dependent. - Ioi On 9/8/15 1:26 PM, Ioi Lam wrote: > Hi Marcus, > > I am eagerly waiting for this to implement a new logging feature in > the VM. How much testing has been done on this patch? I can see only > two locations where any logs are bring generated: > > + log_error(logging)("%s", errbuf); > + log_error(logging)("Could not open log file '%s' (%s).\n", > _file_name, strerror(errno)); > > Do you have a follow-on patch that's already in work to test the > various functionalists of the Unified JVM Logging framework? I am > asking so that I can know how much surprise to expect when I start > using it. > > Thanks > - Ioi > > > On 9/7/15 6:33 AM, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch adding the unified logging >> framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. >> Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which >> contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log >> call has a corresponding LogTagSet instance, which keeps a track of >> all the outputs it should write the log message to (and their >> levels). Having tags as template arguments allows mapping directly >> from a set of tags to the LogTagSet instance, which means that the >> overhead for disabled logging should be low. Currently each log >> message can be tagged with up to 5 tags, but this can be increased if >> ever required (and with C++11's variadic templates the limit can be >> removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by >> command line arguments (-Xlog) or by DCMD. Both methods will in turn >> use the LogConfiguration class to perform the parsing & >> configuration. This configuration includes iterating over all >> LogTagSet instances and updating them accordingly. The >> LogTagLevelExpression class is used to represent the selection of >> tags and levels for a given configuration request (the >> "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances >> of this class is kept in LogTagSet to track what decorators to use >> (this is the union of all decorators used by its outputs). Each log >> call will create a LogDecorations instance (note: different classes), >> which will contain the actual decoration strings for the log message. >> These decorations are used for each output the tagset is set to log >> on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with >> argument. (This allows GC logging to prefix messages of certain >> tagsets with GCId.) Prefixes are implemented using template >> specializations based on the specified tags, with the >> general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log >> outputs. LogFileStreamOutput corresponds to FILE* stream based log >> outputs. LogFileOutput builds on this and adds the file management >> and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will >> be added at a later stage. >> >> Thanks, >> Marcus > From marcus.larsson at oracle.com Wed Sep 9 07:33:06 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 9 Sep 2015 09:33:06 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EF4AC7.40809@oracle.com> References: <55ED9210.6030904@oracle.com> <55EF4492.6000904@oracle.com> <55EF4AC7.40809@oracle.com> Message-ID: <55EFE0B2.2060100@oracle.com> Hi, On 2015-09-08 22:53, Ioi Lam wrote: > Also, is it possible to combine the output of two tags into a single > file, like this? > > -Xlog:gc:file=out.txt -Xlog:comp:file=out.txt > > this way, I can have both GC and Compiler logs that's in an > (approximate) temporal order. The alternatives are: > > - write both to stdout -- but I need to filter out junk that I don't want > - write to different files -- but I lose the relative ordering between > the two sets of logs > > From the current patch, it seems like if I assign the same file name > to two tags, fopen() will be called twice and the result will be > platform dependent. > > - Ioi This is indeed possible! The syntax for configuring this would be -Xlog:gc,comp:file=out.txt. Configuring logging for the same output multiple times only overrides/replaces the previous configuration. The second fopen will not happen, because the log configuration will notice the output already exists and simply reconfigure the existing output. Thanks, Marcus > > On 9/8/15 1:26 PM, Ioi Lam wrote: >> Hi Marcus, >> >> I am eagerly waiting for this to implement a new logging feature in >> the VM. How much testing has been done on this patch? I can see only >> two locations where any logs are bring generated: >> >> + log_error(logging)("%s", errbuf); >> + log_error(logging)("Could not open log file '%s' (%s).\n", >> _file_name, strerror(errno)); >> >> Do you have a follow-on patch that's already in work to test the >> various functionalists of the Unified JVM Logging framework? I am >> asking so that I can know how much surprise to expect when I start >> using it. >> >> Thanks >> - Ioi >> >> >> On 9/7/15 6:33 AM, Marcus Larsson wrote: >>> Hi, >>> >>> Please review the following patch adding the unified logging >>> framework to hotspot. >>> >>> JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>> >>> See the JEP description for a general overview of the new feature. >>> Below are some notes on the implementation. >>> >>> The patch adds the new 'share/vm/logging' subdirectory containing >>> the unified logging framework. The main entry point is log.hpp, >>> which contains the necessary macros and definitions to use the >>> framework. >>> >>> Log tags are defined/listed in logTag.hpp, and are passed as >>> template arguments to the Log class. Every combination of tags used >>> in a log call has a corresponding LogTagSet instance, which keeps a >>> track of all the outputs it should write the log message to (and >>> their levels). Having tags as template arguments allows mapping >>> directly from a set of tags to the LogTagSet instance, which means >>> that the overhead for disabled logging should be low. Currently each >>> log message can be tagged with up to 5 tags, but this can be >>> increased if ever required (and with C++11's variadic templates the >>> limit can be removed completely). >>> >>> The LogConfiguration class keeps track of configured outputs >>> (stdout, stderr, and possible file outputs). Configuration is done >>> either by command line arguments (-Xlog) or by DCMD. Both methods >>> will in turn use the LogConfiguration class to perform the parsing & >>> configuration. This configuration includes iterating over all >>> LogTagSet instances and updating them accordingly. The >>> LogTagLevelExpression class is used to represent the selection of >>> tags and levels for a given configuration request (the >>> "what"-expression). >>> >>> The LogDecorators class contains a selection of decorators. >>> Instances of this class is kept in LogTagSet to track what >>> decorators to use (this is the union of all decorators used by its >>> outputs). Each log call will create a LogDecorations instance (note: >>> different classes), which will contain the actual decoration strings >>> for the log message. These decorations are used for each output the >>> tagset is set to log on, and are then discarded. >>> >>> The LogPrefix class allows messages of specific sets of tags to be >>> prefixed. The prefix should supply a printf-style format with >>> argument. (This allows GC logging to prefix messages of certain >>> tagsets with GCId.) Prefixes are implemented using template >>> specializations based on the specified tags, with the >>> general/unspecialized case giving an empty prefix. >>> >>> The LogOutput class defines the interface for all types of log >>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>> outputs. LogFileOutput builds on this and adds the file management >>> and log rotation support. >>> >>> A simple jtreg test is included in this patch. Additional tests will >>> be added at a later stage. >>> >>> Thanks, >>> Marcus >> > From cnewland at chrisnewland.com Wed Sep 9 07:48:49 2015 From: cnewland at chrisnewland.com (Chris Newland) Date: Wed, 9 Sep 2015 08:48:49 +0100 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EEC481.7030605@oracle.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> Message-ID: <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> Hi Marcus, Thanks for your answers. The nested XML structure of LogCompilation supports the description of arbitrarily complicated JIT parsing chains. The JIT compiler can log the start of a compilation with an opening XML tag and then hand off to the various subsystems (inlining, escape analysis, reg alloc etc.) which output child XML nodes and once they return the parent tag is closed. Do you have any thoughts on how this could be acheived in a non-XML format? Kind regards, Chris >> >> Once implemented, will the current output formats still be available? >> > > Currently there is no specific support for XML output in the unified > logging framework. Output can be directed either to stdout, stderr or plain > text files. I'm not entirely sure how the migration of the XML logging > will be done. Perhaps it will be preceded by an RFE to add better support > for XML outputs in the unified logging framework, or it might just be > treated as a regular text file. > > Thanks, > Marcus > From staffan.larsen at oracle.com Wed Sep 9 08:13:57 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 9 Sep 2015 10:13:57 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <938EE5E3-A1FD-47B0-B6D6-7D6D7EEB7044@oracle.com> Marcus, This looks good and I?m happy to see this finally be integrated! /Staffan > On 7 sep 2015, at 15:33, Marcus Larsson wrote: > > Hi, > > Please review the following patch adding the unified logging framework to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). > > The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. > > A simple jtreg test is included in this patch. Additional tests will be added at a later stage. > > Thanks, > Marcus From marcus.larsson at oracle.com Wed Sep 9 08:24:14 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 9 Sep 2015 10:24:14 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <938EE5E3-A1FD-47B0-B6D6-7D6D7EEB7044@oracle.com> References: <55ED9210.6030904@oracle.com> <938EE5E3-A1FD-47B0-B6D6-7D6D7EEB7044@oracle.com> Message-ID: <55EFECAE.2000908@oracle.com> Thank you for reviewing Staffan! Marcus On 2015-09-09 10:13, Staffan Larsen wrote: > Marcus, > > This looks good and I?m happy to see this finally be integrated! > > /Staffan > >> On 7 sep 2015, at 15:33, Marcus Larsson wrote: >> >> Hi, >> >> Please review the following patch adding the unified logging framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will be added at a later stage. >> >> Thanks, >> Marcus From marcus.larsson at oracle.com Wed Sep 9 08:27:31 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 9 Sep 2015 10:27:31 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> Message-ID: <55EFED73.6070307@oracle.com> Hi, On 2015-09-08 14:47, Kirk Pepperdine wrote: >> On Sep 8, 2015, at 1:42 PM, Claes Redestad wrote: >> >> >> >> On 2015-09-08 12:45, Kirk Pepperdine wrote: >>> Hi, >>> >>> Indeed, not all messages are hierarchical. This is one of the reasons I originally questioned the use of log levels. For example, debugging and informational logs may come from the same source and I certainly wouldn?t want to be forced mix them to satisfy a forced structure. Is the intention you can use either or? I ask because I?m not sure that mixing them with tags will alleviate the problems I commonly see when tuning applications in the field. >>> >>>> Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now. >>> I would like to believe that this work is important enough to get right and that it?s never too late for fundamental changes up until it is integrated. I would like to believe that there are back out strategies in the cases where implementations show unintended consequences. >> Can you give an example of what you believe will go wrong with allowing a mix of levels and tags and how removing levels from the JEP would improve things? > I?m slowly working through the patch but I?ve not had time to get through all of it yet so some things mentioned here might be in the source already? but here are some comments from experiences. > > As you have discovered, you can?t define a tag system using levels. That said, if you have tags you can define levels but you don?t need to explicitly support them, it just come for free out of the box. So I would argue both the spec and implementation would be simplified by moving to a tag system. More over, it doesn?t preclude someone defining a hierarchical system if them so need to have one. While implementing levels using tags might simplify the implementation, I think it would introduce more complexity for the log producers, since there would be a much wider choice between "levels" in contrast to the current six levels. It should be easy to specify how much logging you want, so I don't think it would be a good idea to omit levels completely either. For example, some consumers of the logging might only be interested in a certain level of log messages, but from any component or from a broad range of components. Others might want every piece of logging belonging to a (very) specific component. > > Next, I think there are two points of view that you have to consider. Usability for the producers and usability for the consumers. From a consumer POV what I would like to see is a logging system that promotes stability in the output. Using broad ill defined categories as they are currently defined in the JDK have historically lead to unstable output. This in turn breaks down stream tooling that maybe used to help triage system out in the wild. How does this happen? Well what gets put into a level is completely arbitrary. What goes into warning, fine, finest.. who knows? More to the point, the producers often have to make a quick decision and in their cubicles they often don?t see the entire downstream tooling chains that they are affecting when they do this even when the information being added it very valuable. I?m going to suggest a system, maybe not *the* system or even the *best* system. It?s just a simple system that people have been using in pub/sub messaging systems for years. I think the technique applies because logging is a form of journalling which is just another form of messaging (hence I suggested people looking at Peter Lawyer?s Chronicle which is designed to be used in HFT applications). We can also use ObjectName (jmx) to define tags. > > If you have a tag, let say, gc.agetable, it?s pretty clear what you should get. Also that output is very likely to remain stable over time. In fact, my experience with PrintTenuringDistrubtion is just that. The output has remained stable for years. I cannot say that about any overly broad category of GC logging such as PrintGCDetails (which I would equate to a level). If I have another tag say, gc.collection.scavenge I should get information about scavenges. Again purpose is very clear from both a producer/consumer POV. I can take age data or scavenge or both or gc.collection.* to get all sub categories for gc.collection. This looks like levels and indeed could be used to implement levels but it?s based on categories (tags). At any rate, the taxonomy is clearer and should offer guidelines that makes it easier for the developer to push the information out on the right channel while offering better stability needed for downstream tooling. The problem of classifying log messages exists whether you use real levels or not; "Should this log message be tagged with verbose?" vs "Should this log message be on trace level?". > > >> Having high-level tags which interact with the standard "levels" seem like a natural fit to me, while the framework proposed in the JEP make it possible for each VM team to support ever more detailed tags to each subsystem where that might be appropriate. For better or worse it adds a degree of freedom. Is your concern that we need more granular tags going forward to help zooming in on specific issues, and that levels for such niche tags might seem/feel superfluous? > If the system has been built with tags on levels then I would suggest the system has been built upside down or inside out, however you want to describe it.. in the same way that JDK logging was build upside down (first problem, super class knows about subclasses and that is a primary source of classloader leaks and that is just the first issue). > > Kind regards, > Kirk > I would say the system has been built with levels on tags. The tags define *what* you want to log, and the levels define *how much* information you want. The levels are sort of a filter on the tags you select. Thanks, Marcus From marcus.larsson at oracle.com Wed Sep 9 08:36:35 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 9 Sep 2015 10:36:35 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> Message-ID: <55EFEF93.4050402@oracle.com> Hi, On 2015-09-09 09:48, Chris Newland wrote: > Hi Marcus, > > Thanks for your answers. > > The nested XML structure of LogCompilation supports the description of > arbitrarily complicated JIT parsing chains. > > The JIT compiler can log the start of a compilation with an opening XML > tag and then hand off to the various subsystems (inlining, escape > analysis, reg alloc etc.) which output child XML nodes and once they > return the parent tag is closed. > > Do you have any thoughts on how this could be acheived in a non-XML format? > > Kind regards, > > Chris I'm not suggesting we go away from the XML format, I just mean that the current XML-producing code could be modified to use the unified logging as its output instead of a file. UL could be happily unaware of the fact that XML is being written to some log file, and there won't be any need to implement XML support in the UL framework itself. Thanks, Marcus > >>> Once implemented, will the current output formats still be available? >>> >> Currently there is no specific support for XML output in the unified >> logging framework. Output can be directed either to stdout, stderr or plain >> text files. I'm not entirely sure how the migration of the XML logging >> will be done. Perhaps it will be preceded by an RFE to add better support >> for XML outputs in the unified logging framework, or it might just be >> treated as a regular text file. >> >> Thanks, >> Marcus >> From kirk at kodewerk.com Wed Sep 9 09:36:25 2015 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 9 Sep 2015 11:36:25 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EFEF93.4050402@oracle.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> <55EFEF93.4050402@oracle.com> Message-ID: <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> Hi Chris, My understanding is that current log formats won?t change unless the authors of the log messages decide to change it them selves. I?ve already started making changes to Censum?s parsers in preparation for the impending mess that will come as users start to sort out how to best use it. I?m happy to share some thoughts with you if you like. I?m not so happy with the mixing of levels and tags and decorators and the other things in there. High performance journaling requires that you get out of the way and stay out of the way. Adding all this extra formatting and bits definitely puts you in the way. However, having a unified logging framework is long overdue so.. we can go back and forth on the details of the implementation but the general direction is good. Regards, Kirk > On Sep 9, 2015, at 10:36 AM, Marcus Larsson wrote: > > Hi, > > On 2015-09-09 09:48, Chris Newland wrote: >> Hi Marcus, >> >> Thanks for your answers. >> >> The nested XML structure of LogCompilation supports the description of >> arbitrarily complicated JIT parsing chains. >> >> The JIT compiler can log the start of a compilation with an opening XML >> tag and then hand off to the various subsystems (inlining, escape >> analysis, reg alloc etc.) which output child XML nodes and once they >> return the parent tag is closed. >> >> Do you have any thoughts on how this could be acheived in a non-XML format? >> >> Kind regards, >> >> Chris > > I'm not suggesting we go away from the XML format, I just mean that the current XML-producing code could be modified to use the unified logging as its output instead of a file. UL could be happily unaware of the fact that XML is being written to some log file, and there won't be any need to implement XML support in the UL framework itself. > > Thanks, > Marcus > >> >>>> Once implemented, will the current output formats still be available? >>>> >>> Currently there is no specific support for XML output in the unified >>> logging framework. Output can be directed either to stdout, stderr or plain >>> text files. I'm not entirely sure how the migration of the XML logging >>> will be done. Perhaps it will be preceded by an RFE to add better support >>> for XML outputs in the unified logging framework, or it might just be >>> treated as a regular text file. >>> >>> Thanks, >>> Marcus >>> > From staffan.larsen at oracle.com Wed Sep 9 10:53:37 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Wed, 9 Sep 2015 12:53:37 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> <55EFEF93.4050402@oracle.com> <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> Message-ID: > On 9 sep 2015, at 11:36, Kirk Pepperdine wrote: > > Hi Chris, > > My understanding is that current log formats won?t change unless the authors of the log messages decide to change it them selves. I?ve already started making changes to Censum?s parsers in preparation for the impending mess that will come as users start to sort out how to best use it. I?m happy to share some thoughts with you if you like. > > I?m not so happy with the mixing of levels and tags and decorators and the other things in there. High performance journaling requires that you get out of the way and stay out of the way. I don?t think the target has ever been a ?high performance journaling? system. We have JFR for that use case. /Staffan > Adding all this extra formatting and bits definitely puts you in the way. However, having a unified logging framework is long overdue so.. we can go back and forth on the details of the implementation but the general direction is good. > > Regards, > Kirk > >> On Sep 9, 2015, at 10:36 AM, Marcus Larsson wrote: >> >> Hi, >> >> On 2015-09-09 09:48, Chris Newland wrote: >>> Hi Marcus, >>> >>> Thanks for your answers. >>> >>> The nested XML structure of LogCompilation supports the description of >>> arbitrarily complicated JIT parsing chains. >>> >>> The JIT compiler can log the start of a compilation with an opening XML >>> tag and then hand off to the various subsystems (inlining, escape >>> analysis, reg alloc etc.) which output child XML nodes and once they >>> return the parent tag is closed. >>> >>> Do you have any thoughts on how this could be acheived in a non-XML format? >>> >>> Kind regards, >>> >>> Chris >> >> I'm not suggesting we go away from the XML format, I just mean that the current XML-producing code could be modified to use the unified logging as its output instead of a file. UL could be happily unaware of the fact that XML is being written to some log file, and there won't be any need to implement XML support in the UL framework itself. >> >> Thanks, >> Marcus >> >>> >>>>> Once implemented, will the current output formats still be available? >>>>> >>>> Currently there is no specific support for XML output in the unified >>>> logging framework. Output can be directed either to stdout, stderr or plain >>>> text files. I'm not entirely sure how the migration of the XML logging >>>> will be done. Perhaps it will be preceded by an RFE to add better support >>>> for XML outputs in the unified logging framework, or it might just be >>>> treated as a regular text file. >>>> >>>> Thanks, >>>> Marcus >>>> >> > From cnewland at chrisnewland.com Wed Sep 9 11:22:28 2015 From: cnewland at chrisnewland.com (Chris Newland) Date: Wed, 9 Sep 2015 12:22:28 +0100 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> <55EFEF93.4050402@oracle.com> <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> Message-ID: <587fd8e63e19f2fb4239fd779132a6ad.squirrel@excalibur.xssl.net> Thanks for the offer Kirk, I'd definitely be interested in hearing your thoughts. Marcus, my headache with the existing logging is different VM sources colliding, resulting in split lines that I can't deterministically reassemble. I believe this is caused by no locking around the tty. Will this problem be solved by unified logging when there is a single destination? Thanks, Chris On Wed, September 9, 2015 10:36, Kirk Pepperdine wrote: > Hi Chris, > > > My understanding is that current log formats won?t change unless the > authors of the log messages decide to change it them selves. I?ve already > started making changes to Censum?s parsers in preparation for the > impending mess that will come as users start to sort out how to best use > it. I?m happy to share some thoughts with you if you like. > > I?m not so happy with the mixing of levels and tags and decorators and > the other things in there. High performance journaling requires that you > get out of the way and stay out of the way. Adding all this extra > formatting and bits definitely puts you in the way. However, having a > unified logging framework is long overdue so.. we can go back and forth > on the details of the implementation but the general direction is good. > > Regards, > Kirk > > >> On Sep 9, 2015, at 10:36 AM, Marcus Larsson >> wrote: >> >> >> Hi, >> >> >> On 2015-09-09 09:48, Chris Newland wrote: >> >>> Hi Marcus, >>> >>> >>> Thanks for your answers. >>> >>> >>> The nested XML structure of LogCompilation supports the description >>> of arbitrarily complicated JIT parsing chains. >>> >>> The JIT compiler can log the start of a compilation with an opening >>> XML >>> tag and then hand off to the various subsystems (inlining, escape >>> analysis, reg alloc etc.) which output child XML nodes and once they >>> return the parent tag is closed. >>> >>> Do you have any thoughts on how this could be acheived in a non-XML >>> format? >>> >>> Kind regards, >>> >>> >>> Chris >>> >> >> I'm not suggesting we go away from the XML format, I just mean that the >> current XML-producing code could be modified to use the unified logging >> as its output instead of a file. UL could be happily unaware of the >> fact that XML is being written to some log file, and there won't be any >> need to implement XML support in the UL framework itself. >> >> Thanks, >> Marcus >> >> >>> >>>>> Once implemented, will the current output formats still be >>>>> available? >>>>> >>>> Currently there is no specific support for XML output in the >>>> unified logging framework. Output can be directed either to stdout, >>>> stderr or plain text files. I'm not entirely sure how the migration >>>> of the XML logging will be done. Perhaps it will be preceded by an >>>> RFE to add better support >>>> for XML outputs in the unified logging framework, or it might just >>>> be treated as a regular text file. >>>> >>>> Thanks, >>>> Marcus >>>> >>>> >> > > From marcus.larsson at oracle.com Wed Sep 9 11:26:31 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 9 Sep 2015 13:26:31 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> <55EFED73.6070307@oracle.com> <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> Message-ID: <55F01767.2080307@oracle.com> On 2015-09-09 11:31, Kirk Pepperdine wrote: >> On Sep 9, 2015, at 10:27 AM, Marcus Larsson wrote: >> >> Hi, >> >> On 2015-09-08 14:47, Kirk Pepperdine wrote: >>>> On Sep 8, 2015, at 1:42 PM, Claes Redestad wrote: >>>> >>>> >>>> >>>> On 2015-09-08 12:45, Kirk Pepperdine wrote: >>>>> Hi, >>>>> >>>>> Indeed, not all messages are hierarchical. This is one of the reasons I originally questioned the use of log levels. For example, debugging and informational logs may come from the same source and I certainly wouldn?t want to be forced mix them to satisfy a forced structure. Is the intention you can use either or? I ask because I?m not sure that mixing them with tags will alleviate the problems I commonly see when tuning applications in the field. >>>>> >>>>>> Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now. >>>>> I would like to believe that this work is important enough to get right and that it?s never too late for fundamental changes up until it is integrated. I would like to believe that there are back out strategies in the cases where implementations show unintended consequences. >>>> Can you give an example of what you believe will go wrong with allowing a mix of levels and tags and how removing levels from the JEP would improve things? >>> I?m slowly working through the patch but I?ve not had time to get through all of it yet so some things mentioned here might be in the source already? but here are some comments from experiences. >>> >>> As you have discovered, you can?t define a tag system using levels. That said, if you have tags you can define levels but you don?t need to explicitly support them, it just come for free out of the box. So I would argue both the spec and implementation would be simplified by moving to a tag system. More over, it doesn?t preclude someone defining a hierarchical system if them so need to have one. >> While implementing levels using tags might simplify the implementation, I think it would introduce more complexity for the log producers, since there would be a much wider choice between "levels" in contrast to the current six levels. > This actually doesn?t introduce a complexity, it simplifies things. Again, I think it would be advantageous to look at messaging implementations such as Chronicle. At least to me, a choice between one of six possible alternatives is easier than a choice of any number of alternatives from an unlimited set. >> It should be easy to specify how much logging you want, so I don't think it would be a good idea to omit levels completely either. > tags do not preclude levels. If people want levels they can easily be created with tags. Again Neal Ford has a number of great talks on taxonomy and in my opinion he gets it spot on when he talks about the problems with level based systems. > >> For example, some consumers of the logging might only be interested in a certain level of log messages, but from any component or from a broad range of components. Others might want every piece of logging belonging to a (very) specific component. > Again, tags do a much better job at this. >>> Next, I think there are two points of view that you have to consider. Usability for the producers and usability for the consumers. From a consumer POV what I would like to see is a logging system that promotes stability in the output. Using broad ill defined categories as they are currently defined in the JDK have historically lead to unstable output. This in turn breaks down stream tooling that maybe used to help triage system out in the wild. How does this happen? Well what gets put into a level is completely arbitrary. What goes into warning, fine, finest.. who knows? More to the point, the producers often have to make a quick decision and in their cubicles they often don?t see the entire downstream tooling chains that they are affecting when they do this even when the information being added it very valuable. I?m going to suggest a system, maybe not *the* system or even the *best* system. It?s just a simple system that people have been using in pub/sub messaging systems for years. I think the technique applies because logging is a form of journalling which is just another form of messaging (hence I suggested people looking at Peter Lawyer?s Chronicle which is designed to be used in HFT applications). We can also use ObjectName (jmx) to define tags. >>> >>> If you have a tag, let say, gc.agetable, it?s pretty clear what you should get. Also that output is very likely to remain stable over time. In fact, my experience with PrintTenuringDistrubtion is just that. The output has remained stable for years. I cannot say that about any overly broad category of GC logging such as PrintGCDetails (which I would equate to a level). If I have another tag say, gc.collection.scavenge I should get information about scavenges. Again purpose is very clear from both a producer/consumer POV. I can take age data or scavenge or both or gc.collection.* to get all sub categories for gc.collection. This looks like levels and indeed could be used to implement levels but it?s based on categories (tags). At any rate, the taxonomy is clearer and should offer guidelines that makes it easier for the developer to push the information out on the right channel while offering better stability needed for downstream tooling. >> The problem of classifying log messages exists whether you use real levels or not; "Should this log message be tagged with verbose?" vs "Should this log message be on trace level??. > I think the problem can be diminished in complexity when you include the vocabulary of the domain in your tag systems. If you think of forcing predefined terms then yes, you are correct. With the current proposal one can easily direct VM wide warnings and errors to a log file, the necessary configuration would be -Xlog:all=warning:out.log. Removing the concept of levels and only using domain specific tags to classify log messages would make it pretty complicated to achieve the same effect. The consumer would have to know about all the different components and their domain specific warning/error-equivalent tags in order to get the same output, or am I missing something? >>> >>>> Having high-level tags which interact with the standard "levels" seem like a natural fit to me, while the framework proposed in the JEP make it possible for each VM team to support ever more detailed tags to each subsystem where that might be appropriate. For better or worse it adds a degree of freedom. Is your concern that we need more granular tags going forward to help zooming in on specific issues, and that levels for such niche tags might seem/feel superfluous? >>> If the system has been built with tags on levels then I would suggest the system has been built upside down or inside out, however you want to describe it.. in the same way that JDK logging was build upside down (first problem, super class knows about subclasses and that is a primary source of classloader leaks and that is just the first issue). >>> >>> Kind regards, >>> Kirk >>> >> I would say the system has been built with levels on tags. The tags define *what* you want to log, and the levels define *how much* information you want. The levels are sort of a filter on the tags you select. > So what you are saying is that as a producer I can completely ignore levels if I so choose to do so. Each message must be logged to a level, so it can't be ignored really. I just don't see what you mean with the inside-out argument. Whether you view it as levels on tags or tags on levels is up to you. Thanks, Marcus > > I do have to apologize as I?ve not been able to get through the entire implementation yet but I hope to shortly. > > Regards, > Kirk > From bengt.rutisson at oracle.com Wed Sep 9 11:38:59 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 09 Sep 2015 13:38:59 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <55EEE152.3030404@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> Message-ID: <55F01A53.1070905@oracle.com> Hi Per, Thanks for looking at this! On 2015-09-08 15:23, Per Liden wrote: > Hi Bengt, > > On 2015-09-08 13:35, Bengt Rutisson wrote: >> >> Hi everyone, >> >> This is mostly a GC related patch, but it adds a field to the Thread >> class, so I'm sending this out on the broader hotspot-dev list. >> >> Could I have a couple of reviews for this patch? >> >> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8134953 >> > > Looks good. I think this is a nice simplification, especially for G1's > concurrent parts. Just two comments: > > genCollectedHeap.cpp: > --------------------- > - GenCollectedHeap::do_collection() has two GCIdMarks, in different > scopes. Do we really want that second mark? We potentially do two GCs in GenCollectedHeap::do_collection(). First a young GC and then potentially a full GC. These two should have different GC IDs. So, yes, we need two GCIdMarks in this method. The scoping could be better, but I think that is a refactoring that should be done separately from my current patch. I'll talk to Jesper about it since he has been cleaning up this code lately. > > > gcId.cpp: > -------- > - I think you might have left currentNamedthread() in there. You > probably just want to use Thread::current() instead. Yes, good catch. Thanks. However, after thinking some more about the implications of moving the GC ID out from the GCTracer I figured it would be possible to just store the previous GC ID in the GCIdMark and then restore it in the destructor of that class. That way we don't have to store anything in the Thread class but can still have both a concurrent GC ID and a STW GC ID active at the same time. This also removes the need to copy the GC ID to the worker tasks. Here's an updated webrev with a solution that does not add anything to the Thread class: http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ And here's a diff compared to the last version: http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ Unfotunately the webrev tool had some bad luck when creating the diff webrev so at least the g1CollectedHeap.cpp seem to contain the complete changes rather than just the diff compared to the 01 version. The rest of the diff looks correct as far as I can tell. Thanks, Bengt > > > cheers, > /Per > > >> Currently the GC ID, that is used for event tracing and logging, is >> stored in the GCTracer object. That has caused some minor problems since >> there are multiple GCTracers active at the same time. The correct GC IDs >> need to be passed around in many places. >> >> For some upcoming GC logging changes I would like to have a more >> consistent way of finding the currently active GC ID. I've played around >> with a couple of different solutions, but eventually I found that the >> simplest solution is to store the current GC ID in the thread. That way >> there is a single way to look it up in all places. It is also fairly >> straight forward to set it up. >> >> I've reworked the GCId class a bit to support this and I've introduced a >> GCIdMark class that is a scoped object that helps to set up and tear >> down a current GC ID. By moving the GC ID out from the GCTracer class I >> got rid of a few minor issues as well. One is that we no longer need to >> keep track of the G1 "aborted concurrent GC ID". That was necessary >> before since we did logging *after* we had told the corresponding >> GCTracer that the concurrent cycle was aborted and it had thrown its GC >> ID away. Now the GC ID can stay in scope until we have completed all >> logging. >> >> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC used >> to have to create a new GC ID since their logging was done before we had >> the correct GCTracer set up. Now the GC ID can be available early enough >> for this logging to be using the same (and correct) GC ID as the rest of >> the GC. Same for HeapDumpAfterFullGC and PrintClassHistogramAfterFullGC. >> >> I've added an uint to the Thread class to keep track of the currently >> active GC ID. In the current code there are actually only NamedThreads >> that need the GC ID. So, I'm not sure where the best place is for this >> field is. But it seems like the Thread class contains most of the data, >> even data that is only used by some subclasses, so I opted for putting >> the field in Thread as opposed to in NamedThread. This opens up for >> possible future extensions when Java threads may do part of the GC work. >> However, I'm open to moving the field down to NamedThread if that is >> seen as better. >> >> In the GCTracer class there were many asserts that were checking that >> the GC ID was properly set up. Mostly these assert verify that start/end >> is called correctly and that the other methods are called inside of the >> start/end scope. Now that the GC ID is moved out of the GCTracer class >> these asserts had to be dealt with. I figured there were three ways to >> handle it; just remove them, replace them with check that the GC ID from >> the current thread is correct, or implement a new status field to keep >> track of GC start/end state. Personally I'm not sure these asserts are >> valuable enough to spend time on, so I went for the first approach: >> removing them. I don't think making them use the thread GC ID will be >> appropriate. But if anyone feels strongly about these asserts I can >> implement a separate start/end state. >> >> There are a few "Tasks" in the GC code that are executed by worker >> threads. To make the worker threads use the correct GC ID I make sure >> that the Task constructors copy the GC ID from the initiating thread >> into a local variable. When the task is executed in its worker thread it >> sets up the GC ID based on the local variable. >> >> The proposed change does not alter any logging (except for the small bug >> fix for >> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. This >> means that no existing tests need to be updated. In particular the >> test/gc/logging/TestGCId.java test passes even after these changes. >> >> >> A big thanks to Per, who pre-reviewed these changes and came with some >> really good feedback. >> >> Thanks, >> Bengt From marcus.larsson at oracle.com Wed Sep 9 14:05:00 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 9 Sep 2015 16:05:00 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <587fd8e63e19f2fb4239fd779132a6ad.squirrel@excalibur.xssl.net> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> <55EFEF93.4050402@oracle.com> <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> <587fd8e63e19f2fb4239fd779132a6ad.squirrel@excalibur.xssl.net> Message-ID: <55F03C8C.4030606@oracle.com> On 2015-09-09 13:22, Chris Newland wrote: > Thanks for the offer Kirk, I'd definitely be interested in hearing your > thoughts. > > Marcus, my headache with the existing logging is different VM sources > colliding, resulting in split lines that I can't deterministically > reassemble. I believe this is caused by no locking around the tty. > > Will this problem be solved by unified logging when there is a single > destination? The UL framework only supports writing of complete log lines. Some of the existing code prints parts of its log lines in chunks directly to the tty, which means that such logging can be interleaved by other writes to the tty causing split lines. This code will have to be reworked during the migration to the UL framework so that only complete lines are logged. Once this is done, there should be no split lines in the log anymore, so in a sense the problem will be resolved with UL. :) Thanks, Marcus > > Thanks, > > Chris > > On Wed, September 9, 2015 10:36, Kirk Pepperdine wrote: >> Hi Chris, >> >> >> My understanding is that current log formats won?t change unless the >> authors of the log messages decide to change it them selves. I?ve already >> started making changes to Censum?s parsers in preparation for the >> impending mess that will come as users start to sort out how to best use >> it. I?m happy to share some thoughts with you if you like. >> >> I?m not so happy with the mixing of levels and tags and decorators and >> the other things in there. High performance journaling requires that you >> get out of the way and stay out of the way. Adding all this extra >> formatting and bits definitely puts you in the way. However, having a >> unified logging framework is long overdue so.. we can go back and forth >> on the details of the implementation but the general direction is good. >> >> Regards, >> Kirk >> >> >>> On Sep 9, 2015, at 10:36 AM, Marcus Larsson >>> wrote: >>> >>> >>> Hi, >>> >>> >>> On 2015-09-09 09:48, Chris Newland wrote: >>> >>>> Hi Marcus, >>>> >>>> >>>> Thanks for your answers. >>>> >>>> >>>> The nested XML structure of LogCompilation supports the description >>>> of arbitrarily complicated JIT parsing chains. >>>> >>>> The JIT compiler can log the start of a compilation with an opening >>>> XML >>>> tag and then hand off to the various subsystems (inlining, escape >>>> analysis, reg alloc etc.) which output child XML nodes and once they >>>> return the parent tag is closed. >>>> >>>> Do you have any thoughts on how this could be acheived in a non-XML >>>> format? >>>> >>>> Kind regards, >>>> >>>> >>>> Chris >>>> >>> I'm not suggesting we go away from the XML format, I just mean that the >>> current XML-producing code could be modified to use the unified logging >>> as its output instead of a file. UL could be happily unaware of the >>> fact that XML is being written to some log file, and there won't be any >>> need to implement XML support in the UL framework itself. >>> >>> Thanks, >>> Marcus >>> >>> >>>>>> Once implemented, will the current output formats still be >>>>>> available? >>>>>> >>>>> Currently there is no specific support for XML output in the >>>>> unified logging framework. Output can be directed either to stdout, >>>>> stderr or plain text files. I'm not entirely sure how the migration >>>>> of the XML logging will be done. Perhaps it will be preceded by an >>>>> RFE to add better support >>>>> for XML outputs in the unified logging framework, or it might just >>>>> be treated as a regular text file. >>>>> >>>>> Thanks, >>>>> Marcus >>>>> >>>>> >> > From bengt.rutisson at oracle.com Wed Sep 9 14:40:29 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 9 Sep 2015 16:40:29 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <55F01A53.1070905@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> Message-ID: <55F044DD.4080301@oracle.com> Hi again, I found an issue with how G1 young collections kick off concurrent marks. There is a short window where we might have two active GC IDs at the same time. I've fixed this and updated the webrevs. In case anyone had already started looking at the webrevs you need to do a refresh now. The fix is in G1CollectedHeap::do_collection_pause_at_safepoint(). Thanks, Bengt On 09/09/15 13:38, Bengt Rutisson wrote: > > Hi Per, > > Thanks for looking at this! > > On 2015-09-08 15:23, Per Liden wrote: >> Hi Bengt, >> >> On 2015-09-08 13:35, Bengt Rutisson wrote: >>> >>> Hi everyone, >>> >>> This is mostly a GC related patch, but it adds a field to the Thread >>> class, so I'm sending this out on the broader hotspot-dev list. >>> >>> Could I have a couple of reviews for this patch? >>> >>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>> >> >> Looks good. I think this is a nice simplification, especially for >> G1's concurrent parts. Just two comments: >> >> genCollectedHeap.cpp: >> --------------------- >> - GenCollectedHeap::do_collection() has two GCIdMarks, in different >> scopes. Do we really want that second mark? > > > We potentially do two GCs in GenCollectedHeap::do_collection(). First > a young GC and then potentially a full GC. These two should have > different GC IDs. So, yes, we need two GCIdMarks in this method. The > scoping could be better, but I think that is a refactoring that should > be done separately from my current patch. I'll talk to Jesper about it > since he has been cleaning up this code lately. > >> >> >> gcId.cpp: >> -------- >> - I think you might have left currentNamedthread() in there. You >> probably just want to use Thread::current() instead. > > Yes, good catch. Thanks. > > However, after thinking some more about the implications of moving the > GC ID out from the GCTracer I figured it would be possible to just > store the previous GC ID in the GCIdMark and then restore it in the > destructor of that class. That way we don't have to store anything in > the Thread class but can still have both a concurrent GC ID and a STW > GC ID active at the same time. This also removes the need to copy the > GC ID to the worker tasks. > > Here's an updated webrev with a solution that does not add anything to > the Thread class: > > http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ > > And here's a diff compared to the last version: > > http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ > > Unfotunately the webrev tool had some bad luck when creating the diff > webrev so at least the g1CollectedHeap.cpp seem to contain the > complete changes rather than just the diff compared to the 01 version. > The rest of the diff looks correct as far as I can tell. > > Thanks, > Bengt >> >> >> cheers, >> /Per >> >> >>> Currently the GC ID, that is used for event tracing and logging, is >>> stored in the GCTracer object. That has caused some minor problems >>> since >>> there are multiple GCTracers active at the same time. The correct GC >>> IDs >>> need to be passed around in many places. >>> >>> For some upcoming GC logging changes I would like to have a more >>> consistent way of finding the currently active GC ID. I've played >>> around >>> with a couple of different solutions, but eventually I found that the >>> simplest solution is to store the current GC ID in the thread. That way >>> there is a single way to look it up in all places. It is also fairly >>> straight forward to set it up. >>> >>> I've reworked the GCId class a bit to support this and I've >>> introduced a >>> GCIdMark class that is a scoped object that helps to set up and tear >>> down a current GC ID. By moving the GC ID out from the GCTracer class I >>> got rid of a few minor issues as well. One is that we no longer need to >>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>> before since we did logging *after* we had told the corresponding >>> GCTracer that the concurrent cycle was aborted and it had thrown its GC >>> ID away. Now the GC ID can stay in scope until we have completed all >>> logging. >>> >>> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC used >>> to have to create a new GC ID since their logging was done before we >>> had >>> the correct GCTracer set up. Now the GC ID can be available early >>> enough >>> for this logging to be using the same (and correct) GC ID as the >>> rest of >>> the GC. Same for HeapDumpAfterFullGC and >>> PrintClassHistogramAfterFullGC. >>> >>> I've added an uint to the Thread class to keep track of the currently >>> active GC ID. In the current code there are actually only NamedThreads >>> that need the GC ID. So, I'm not sure where the best place is for this >>> field is. But it seems like the Thread class contains most of the data, >>> even data that is only used by some subclasses, so I opted for putting >>> the field in Thread as opposed to in NamedThread. This opens up for >>> possible future extensions when Java threads may do part of the GC >>> work. >>> However, I'm open to moving the field down to NamedThread if that is >>> seen as better. >>> >>> In the GCTracer class there were many asserts that were checking that >>> the GC ID was properly set up. Mostly these assert verify that >>> start/end >>> is called correctly and that the other methods are called inside of the >>> start/end scope. Now that the GC ID is moved out of the GCTracer class >>> these asserts had to be dealt with. I figured there were three ways to >>> handle it; just remove them, replace them with check that the GC ID >>> from >>> the current thread is correct, or implement a new status field to keep >>> track of GC start/end state. Personally I'm not sure these asserts are >>> valuable enough to spend time on, so I went for the first approach: >>> removing them. I don't think making them use the thread GC ID will be >>> appropriate. But if anyone feels strongly about these asserts I can >>> implement a separate start/end state. >>> >>> There are a few "Tasks" in the GC code that are executed by worker >>> threads. To make the worker threads use the correct GC ID I make sure >>> that the Task constructors copy the GC ID from the initiating thread >>> into a local variable. When the task is executed in its worker >>> thread it >>> sets up the GC ID based on the local variable. >>> >>> The proposed change does not alter any logging (except for the small >>> bug >>> fix for >>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>> This >>> means that no existing tests need to be updated. In particular the >>> test/gc/logging/TestGCId.java test passes even after these changes. >>> >>> >>> A big thanks to Per, who pre-reviewed these changes and came with some >>> really good feedback. >>> >>> Thanks, >>> Bengt > From kirk at kodewerk.com Wed Sep 9 14:43:03 2015 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 9 Sep 2015 16:43:03 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> <55EFEF93.4050402@oracle.com> <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> Message-ID: <1AA299C5-A194-4140-8272-293765FFE0EE@kodewerk.com> >> >> I?m not so happy with the mixing of levels and tags and decorators and the other things in there. High performance journaling requires that you get out of the way and stay out of the way. > > I don?t think the target has ever been a ?high performance journaling? system. We have JFR for that use case. JFR is a commercial product so I?m not sure how it fits into decisions about an source project. My point wasn?t that the JVM provide high performance journaling, my point was that journaling, messaging and logging are all different flavors of the same activity and that there lessons that can be learned and applied from the other like activities. Further more, GC logging is currently part of the STW cost so having something that gets out of the way will be a huge benefit across the board. I know that the write is detached from the producer but even so, slow logging can cause things to back up. My extreme example is an application where 4.3 seconds of a 4.5 second time budget was spent in Log4J?s async adaptor! I have plenty other examples where logging is the prime consumer of the latency budget in an application at moderate rates of logging and that happens irregardless of the logging framework in use. Why I suggested to take a look at Chronicle is because it can log/message at very high rates, much much higher than any of the other logging frameworks without inflicting significant latency of the logging thread. I?m not suggesting that high performance logging is a hard requirement but if you can get there with the same effort why would you not want to go down that path? Machine go fast, you just need to get out of the way. Kind regards, Kirk From rachel.protacio at oracle.com Wed Sep 9 15:37:12 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Wed, 9 Sep 2015 11:37:12 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <55F05228.7080905@oracle.com> Hi! For runtime logging, there are many locations where it is necessary to have a block led by a conditional checking whether the debug/info/trace logging is enabled with the appropriate tag. Would it be possible to add macros to logging/log.hpp such as #define log_debug_is_enabled(...) log_is_enabled(Debug, __VA_ARGS__) #define log_trace_is_enabled(...) log_is_enabled(Trace, __VA_ARGS__) etc.? If they are added to your framework patch, that will help Max and me as we write our logging code. Additionally, the print format for the logging was not immediately intuitive. For example, as I'm working on the safepoint logging, I might have an output line like [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) Are the spaces after debug intended to line up the tags with each other (since the longest one is warning, it seems to fit)? If so, is that meant for readability? And what happens if the number of seconds goes into double digits - won't the alignment be off? Thanks, Rachel On 9/7/2015 9:33 AM, Marcus Larsson wrote: > Hi, > > Please review the following patch adding the unified logging framework > to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. > Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the > unified logging framework. The main entry point is log.hpp, which > contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template > arguments to the Log class. Every combination of tags used in a log > call has a corresponding LogTagSet instance, which keeps a track of > all the outputs it should write the log message to (and their levels). > Having tags as template arguments allows mapping directly from a set > of tags to the LogTagSet instance, which means that the overhead for > disabled logging should be low. Currently each log message can be > tagged with up to 5 tags, but this can be increased if ever required > (and with C++11's variadic templates the limit can be removed > completely). > > The LogConfiguration class keeps track of configured outputs (stdout, > stderr, and possible file outputs). Configuration is done either by > command line arguments (-Xlog) or by DCMD. Both methods will in turn > use the LogConfiguration class to perform the parsing & configuration. > This configuration includes iterating over all LogTagSet instances and > updating them accordingly. The LogTagLevelExpression class is used to > represent the selection of tags and levels for a given configuration > request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances > of this class is kept in LogTagSet to track what decorators to use > (this is the union of all decorators used by its outputs). Each log > call will create a LogDecorations instance (note: different classes), > which will contain the actual decoration strings for the log message. > These decorations are used for each output the tagset is set to log > on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be > prefixed. The prefix should supply a printf-style format with > argument. (This allows GC logging to prefix messages of certain > tagsets with GCId.) Prefixes are implemented using template > specializations based on the specified tags, with the > general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log > outputs. LogFileStreamOutput corresponds to FILE* stream based log > outputs. LogFileOutput builds on this and adds the file management and > log rotation support. > > A simple jtreg test is included in this patch. Additional tests will > be added at a later stage. > > Thanks, > Marcus From doko at ubuntu.com Wed Sep 9 16:19:04 2015 From: doko at ubuntu.com (Matthias Klose) Date: Wed, 9 Sep 2015 18:19:04 +0200 Subject: [PATCH] fix zero builds for "unknown" architectures Message-ID: <55F05BF8.3090502@ubuntu.com> seen with jdk9 / tag jdk9-b80. zero builds which don't match one of the hotspot architectures fail to build in src/os/linux/vm/os_linux.cpp, because there is no default / or else clause: const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model name") IA64_ONLY("") SPARC_ONLY("cpu") ARM32_ONLY("Processor") PPC_ONLY("Processor") AARCH64_ONLY("Processor"); and: strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") IA64_ONLY("IA64") SPARC_ONLY("sparcv9") ARM32_ONLY("ARM") PPC_ONLY("PPC64") AARCH64_ONLY("AArch64"), length); attached are two alternate patches how to fix this, either by not using the *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. Two other issues: - The zero builds only define ARM, not ARM32, so the clause should be ARM_ONLY (or at least an ARM_ONLY added). - The cpuinfo string seems to be wrong for AMD64_ONLY. Verified that zero builds without errors with on of these patches. Attaching both patches. Matthias -------------- next part -------------- --- src/hotspot/src/os/linux/vm/os_linux.cpp +++ src/hotspot/src/os/linux/vm/os_linux.cpp @@ -2211,9 +2211,13 @@ } } -const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model name") - IA64_ONLY("") SPARC_ONLY("cpu") - ARM32_ONLY("Processor") PPC_ONLY("Processor") AARCH64_ONLY("Processor"); +#if defined(AMD64) || defined(IA32) || defined(X32) +const char* search_string = "model name"; +#elif defined(SPARC) +const char* search_string = "cpu"; +#else +const char* search_string = "Processor"; +#endif // Parses the cpuinfo file for string representing the model name. void os::get_summary_cpu_info(char* cpuinfo, size_t length) { @@ -2248,9 +2252,26 @@ } // cpuinfo not found or parsing failed, just print generic string. The entire // /proc/cpuinfo file will be printed later in the file (or enough of it for x86) - strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") - IA64_ONLY("IA64") SPARC_ONLY("sparcv9") - ARM32_ONLY("ARM") PPC_ONLY("PPC64") AARCH64_ONLY("AArch64"), length); + +#if defined(AMD64) + strncpy(cpuinfo, "x86_64", length); +#elif defined(IA32) + strncpy(cpuinfo, "x86_32", length); +#elif defined(IA64) + strncpy(cpuinfo, "IA64", length); +#elif defined(SPARC) + strncpy(cpuinfo, "sparcv9", length); +#elif defined(ARM) + strncpy(cpuinfo, "ARM", length); +#elif defined(AARCH64) + strncpy(cpuinfo, "AArch64", length); +#elif defined(PPC) + strncpy(cpuinfo, "PPC64", length); +#elif defined(ZERO_LIBARCH) + strncpy(cpuinfo, ZERO_LIBARCH, length); +#else + strncpy(cpuinfo, "unknown", length); +#endif } void os::print_siginfo(outputStream* st, void* siginfo) { -------------- next part -------------- --- a/hotspot/src/share/vm/utilities/macros.hpp +++ b/hotspot/src/share/vm/utilities/macros.hpp @@ -425,6 +425,12 @@ #define NOT_AARCH64(code) code #endif +#if !defined(X86) && !defined(IA64) && !defined(SPARC) && !defined(PPC) && !defined(ARM) && !defined(AARCH64) +#define UNKNOWN_ARCH_ONLY(code) code +#else +#define UNKNOWN_ARCH_ONLY(code) +#endif + #ifdef JAVASE_EMBEDDED #define EMBEDDED_ONLY(code) code #define NOT_EMBEDDED(code) --- a/hotspot/src/os/linux/vm/os_linux.cpp +++ b/hotspot/src/os/linux/vm/os_linux.cpp @@ -2213,7 +2213,8 @@ const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model name") IA64_ONLY("") SPARC_ONLY("cpu") - ARM32_ONLY("Processor") PPC_ONLY("Processor") AARCH64_ONLY("Processor"); + ARM_ONLY("Processor") PPC_ONLY("Processor") AARCH64_ONLY("Processor") + UNKNOWN_ARCH_ONLY("Processor"); // Parses the cpuinfo file for string representing the model name. void os::get_summary_cpu_info(char* cpuinfo, size_t length) { @@ -2248,9 +2249,15 @@ } // cpuinfo not found or parsing failed, just print generic string. The entire // /proc/cpuinfo file will be printed later in the file (or enough of it for x86) - strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") + strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_64") IA64_ONLY("IA64") SPARC_ONLY("sparcv9") - ARM32_ONLY("ARM") PPC_ONLY("PPC64") AARCH64_ONLY("AArch64"), length); + ARM_ONLY("ARM") PPC_ONLY("PPC64") AARCH64_ONLY("AArch64") +#ifdef ZERO_LIBARCH + ZERO_LIBARCH +#else + UNKNOWN_ARCH_ONLY("unknown") +#endif + , length); } void os::print_siginfo(outputStream* st, void* siginfo) { From stanislav.smirnov at oracle.com Wed Sep 9 16:42:34 2015 From: stanislav.smirnov at oracle.com (Stas Smirnov) Date: Wed, 9 Sep 2015 19:42:34 +0300 Subject: RFR(XS): 8132980: Improve stability of whitebox methods getCodeBlob and freeCodeBlob In-Reply-To: <55EF0ED4.2030301@oracle.com> References: <55EF0ED4.2030301@oracle.com> Message-ID: <55F0617A.60003@oracle.com> Hi, changes looks good, one small remark, in one case when "addr == 0" you just make a return and in another case you THROW_MSG_NULL is there any specific reason for a different way of handling the same condition? On 08.09.2015 19:37, Kirill Zhaldybin wrote: > Hi! > > Could you please review the following patch for WB methods? > > I added checks for negative size and null address values (where > applicable) for getCodeBlob, allocateCodeBlob and freeCodeBlob, added > regression test. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8132980 > Webrev: > hotspot: > http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132980/webrev.00/ > > Thanks, > Kirill From coleen.phillimore at oracle.com Wed Sep 9 16:56:29 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 09 Sep 2015 12:56:29 -0400 Subject: [PATCH] fix zero builds for "unknown" architectures In-Reply-To: <55F05BF8.3090502@ubuntu.com> References: <55F05BF8.3090502@ubuntu.com> Message-ID: <55F064BD.6020603@oracle.com> I think I like patch #1 better. I will sponsor it and make sure it works on my machine. Thank you for fixing this on the other platforms. Coleen On 9/9/15 12:19 PM, Matthias Klose wrote: > seen with jdk9 / tag jdk9-b80. zero builds which don't match one of the hotspot > architectures fail to build in src/os/linux/vm/os_linux.cpp, because there is no > default / or else clause: > > const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model name") > IA64_ONLY("") SPARC_ONLY("cpu") > ARM32_ONLY("Processor") PPC_ONLY("Processor") > AARCH64_ONLY("Processor"); > > and: > > strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") > IA64_ONLY("IA64") SPARC_ONLY("sparcv9") > ARM32_ONLY("ARM") PPC_ONLY("PPC64") AARCH64_ONLY("AArch64"), > length); > > attached are two alternate patches how to fix this, either by not using the > *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. > > Two other issues: > - The zero builds only define ARM, not ARM32, so the clause > should be ARM_ONLY (or at least an ARM_ONLY added). > - The cpuinfo string seems to be wrong for AMD64_ONLY. > > Verified that zero builds without errors with on of these patches. > > Attaching both patches. > > Matthias > From kirill.zhaldybin at oracle.com Wed Sep 9 17:25:17 2015 From: kirill.zhaldybin at oracle.com (Kirill Zhaldybin) Date: Wed, 09 Sep 2015 20:25:17 +0300 Subject: RFR(XS): 8132980: Improve stability of whitebox methods getCodeBlob and freeCodeBlob In-Reply-To: <55F0617A.60003@oracle.com> References: <55EF0ED4.2030301@oracle.com> <55F0617A.60003@oracle.com> Message-ID: <55F06B7D.2020309@oracle.com> Stanislav, "addr == 0" in freeCodeBlob is just ignored since it's traditional for C/C++ to allow free(0) but when we are trying to read something from addr == 0 in getCodeBlob it looks like error (and crash JVM actually). Thank you. Regards. Kirill On 09.09.2015 19:42, Stas Smirnov wrote: > Hi, > > changes looks good, one small remark, in one case when "addr == 0" you > just make a return and in another case you THROW_MSG_NULL is there any > specific reason for a different way of handling the same condition? > > On 08.09.2015 19:37, Kirill Zhaldybin wrote: >> Hi! >> >> Could you please review the following patch for WB methods? >> >> I added checks for negative size and null address values (where >> applicable) for getCodeBlob, allocateCodeBlob and freeCodeBlob, added >> regression test. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8132980 >> Webrev: >> hotspot: >> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132980/webrev.00/ >> >> Thanks, >> Kirill > From jon.masamitsu at oracle.com Wed Sep 9 23:00:16 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 9 Sep 2015 16:00:16 -0700 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <55F044DD.4080301@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> Message-ID: <55F0BA00.5000704@oracle.com> Bengt, When a CMS concurrent collection hands off to a STW foreground collection, (done in acquire_control_and_collect()), I expected a new GCId would be used. Did I miss it? That STW collection does not go through do_collection(). Jon On 9/9/2015 7:40 AM, Bengt Rutisson wrote: > > Hi again, > > I found an issue with how G1 young collections kick off concurrent > marks. There is a short window where we might have two active GC IDs > at the same time. I've fixed this and updated the webrevs. In case > anyone had already started looking at the webrevs you need to do a > refresh now. The fix is in > G1CollectedHeap::do_collection_pause_at_safepoint(). > > Thanks, > Bengt > > On 09/09/15 13:38, Bengt Rutisson wrote: >> >> Hi Per, >> >> Thanks for looking at this! >> >> On 2015-09-08 15:23, Per Liden wrote: >>> Hi Bengt, >>> >>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>> >>>> Hi everyone, >>>> >>>> This is mostly a GC related patch, but it adds a field to the Thread >>>> class, so I'm sending this out on the broader hotspot-dev list. >>>> >>>> Could I have a couple of reviews for this patch? >>>> >>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>> >>> >>> Looks good. I think this is a nice simplification, especially for >>> G1's concurrent parts. Just two comments: >>> >>> genCollectedHeap.cpp: >>> --------------------- >>> - GenCollectedHeap::do_collection() has two GCIdMarks, in different >>> scopes. Do we really want that second mark? >> >> >> We potentially do two GCs in GenCollectedHeap::do_collection(). First >> a young GC and then potentially a full GC. These two should have >> different GC IDs. So, yes, we need two GCIdMarks in this method. The >> scoping could be better, but I think that is a refactoring that >> should be done separately from my current patch. I'll talk to Jesper >> about it since he has been cleaning up this code lately. >> >>> >>> >>> gcId.cpp: >>> -------- >>> - I think you might have left currentNamedthread() in there. You >>> probably just want to use Thread::current() instead. >> >> Yes, good catch. Thanks. >> >> However, after thinking some more about the implications of moving >> the GC ID out from the GCTracer I figured it would be possible to >> just store the previous GC ID in the GCIdMark and then restore it in >> the destructor of that class. That way we don't have to store >> anything in the Thread class but can still have both a concurrent GC >> ID and a STW GC ID active at the same time. This also removes the >> need to copy the GC ID to the worker tasks. >> >> Here's an updated webrev with a solution that does not add anything >> to the Thread class: >> >> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >> >> And here's a diff compared to the last version: >> >> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >> >> Unfotunately the webrev tool had some bad luck when creating the diff >> webrev so at least the g1CollectedHeap.cpp seem to contain the >> complete changes rather than just the diff compared to the 01 >> version. The rest of the diff looks correct as far as I can tell. >> >> Thanks, >> Bengt >>> >>> >>> cheers, >>> /Per >>> >>> >>>> Currently the GC ID, that is used for event tracing and logging, is >>>> stored in the GCTracer object. That has caused some minor problems >>>> since >>>> there are multiple GCTracers active at the same time. The correct >>>> GC IDs >>>> need to be passed around in many places. >>>> >>>> For some upcoming GC logging changes I would like to have a more >>>> consistent way of finding the currently active GC ID. I've played >>>> around >>>> with a couple of different solutions, but eventually I found that the >>>> simplest solution is to store the current GC ID in the thread. That >>>> way >>>> there is a single way to look it up in all places. It is also fairly >>>> straight forward to set it up. >>>> >>>> I've reworked the GCId class a bit to support this and I've >>>> introduced a >>>> GCIdMark class that is a scoped object that helps to set up and tear >>>> down a current GC ID. By moving the GC ID out from the GCTracer >>>> class I >>>> got rid of a few minor issues as well. One is that we no longer >>>> need to >>>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>>> before since we did logging *after* we had told the corresponding >>>> GCTracer that the concurrent cycle was aborted and it had thrown >>>> its GC >>>> ID away. Now the GC ID can stay in scope until we have completed all >>>> logging. >>>> >>>> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC used >>>> to have to create a new GC ID since their logging was done before >>>> we had >>>> the correct GCTracer set up. Now the GC ID can be available early >>>> enough >>>> for this logging to be using the same (and correct) GC ID as the >>>> rest of >>>> the GC. Same for HeapDumpAfterFullGC and >>>> PrintClassHistogramAfterFullGC. >>>> >>>> I've added an uint to the Thread class to keep track of the currently >>>> active GC ID. In the current code there are actually only NamedThreads >>>> that need the GC ID. So, I'm not sure where the best place is for this >>>> field is. But it seems like the Thread class contains most of the >>>> data, >>>> even data that is only used by some subclasses, so I opted for putting >>>> the field in Thread as opposed to in NamedThread. This opens up for >>>> possible future extensions when Java threads may do part of the GC >>>> work. >>>> However, I'm open to moving the field down to NamedThread if that is >>>> seen as better. >>>> >>>> In the GCTracer class there were many asserts that were checking that >>>> the GC ID was properly set up. Mostly these assert verify that >>>> start/end >>>> is called correctly and that the other methods are called inside of >>>> the >>>> start/end scope. Now that the GC ID is moved out of the GCTracer class >>>> these asserts had to be dealt with. I figured there were three ways to >>>> handle it; just remove them, replace them with check that the GC ID >>>> from >>>> the current thread is correct, or implement a new status field to keep >>>> track of GC start/end state. Personally I'm not sure these asserts are >>>> valuable enough to spend time on, so I went for the first approach: >>>> removing them. I don't think making them use the thread GC ID will be >>>> appropriate. But if anyone feels strongly about these asserts I can >>>> implement a separate start/end state. >>>> >>>> There are a few "Tasks" in the GC code that are executed by worker >>>> threads. To make the worker threads use the correct GC ID I make sure >>>> that the Task constructors copy the GC ID from the initiating thread >>>> into a local variable. When the task is executed in its worker >>>> thread it >>>> sets up the GC ID based on the local variable. >>>> >>>> The proposed change does not alter any logging (except for the >>>> small bug >>>> fix for >>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>> This >>>> means that no existing tests need to be updated. In particular the >>>> test/gc/logging/TestGCId.java test passes even after these changes. >>>> >>>> >>>> A big thanks to Per, who pre-reviewed these changes and came with some >>>> really good feedback. >>>> >>>> Thanks, >>>> Bengt >> > From kirk at jclarity.com Wed Sep 9 09:31:25 2015 From: kirk at jclarity.com (Kirk Pepperdine) Date: Wed, 9 Sep 2015 11:31:25 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EFED73.6070307@oracle.com> References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> <55EFED73.6070307@oracle.com> Message-ID: <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> > On Sep 9, 2015, at 10:27 AM, Marcus Larsson wrote: > > Hi, > > On 2015-09-08 14:47, Kirk Pepperdine wrote: >>> On Sep 8, 2015, at 1:42 PM, Claes Redestad wrote: >>> >>> >>> >>> On 2015-09-08 12:45, Kirk Pepperdine wrote: >>>> Hi, >>>> >>>> Indeed, not all messages are hierarchical. This is one of the reasons I originally questioned the use of log levels. For example, debugging and informational logs may come from the same source and I certainly wouldn?t want to be forced mix them to satisfy a forced structure. Is the intention you can use either or? I ask because I?m not sure that mixing them with tags will alleviate the problems I commonly see when tuning applications in the field. >>>> >>>>> Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now. >>>> I would like to believe that this work is important enough to get right and that it?s never too late for fundamental changes up until it is integrated. I would like to believe that there are back out strategies in the cases where implementations show unintended consequences. >>> Can you give an example of what you believe will go wrong with allowing a mix of levels and tags and how removing levels from the JEP would improve things? >> I?m slowly working through the patch but I?ve not had time to get through all of it yet so some things mentioned here might be in the source already? but here are some comments from experiences. >> >> As you have discovered, you can?t define a tag system using levels. That said, if you have tags you can define levels but you don?t need to explicitly support them, it just come for free out of the box. So I would argue both the spec and implementation would be simplified by moving to a tag system. More over, it doesn?t preclude someone defining a hierarchical system if them so need to have one. > > While implementing levels using tags might simplify the implementation, I think it would introduce more complexity for the log producers, since there would be a much wider choice between "levels" in contrast to the current six levels. This actually doesn?t introduce a complexity, it simplifies things. Again, I think it would be advantageous to look at messaging implementations such as Chronicle. > It should be easy to specify how much logging you want, so I don't think it would be a good idea to omit levels completely either. tags do not preclude levels. If people want levels they can easily be created with tags. Again Neal Ford has a number of great talks on taxonomy and in my opinion he gets it spot on when he talks about the problems with level based systems. > For example, some consumers of the logging might only be interested in a certain level of log messages, but from any component or from a broad range of components. Others might want every piece of logging belonging to a (very) specific component. Again, tags do a much better job at this. > >> >> Next, I think there are two points of view that you have to consider. Usability for the producers and usability for the consumers. From a consumer POV what I would like to see is a logging system that promotes stability in the output. Using broad ill defined categories as they are currently defined in the JDK have historically lead to unstable output. This in turn breaks down stream tooling that maybe used to help triage system out in the wild. How does this happen? Well what gets put into a level is completely arbitrary. What goes into warning, fine, finest.. who knows? More to the point, the producers often have to make a quick decision and in their cubicles they often don?t see the entire downstream tooling chains that they are affecting when they do this even when the information being added it very valuable. I?m going to suggest a system, maybe not *the* system or even the *best* system. It?s just a simple system that people have been using in pub/sub messaging systems for years. I think the technique applies because logging is a form of journalling which is just another form of messaging (hence I suggested people looking at Peter Lawyer?s Chronicle which is designed to be used in HFT applications). We can also use ObjectName (jmx) to define tags. >> >> If you have a tag, let say, gc.agetable, it?s pretty clear what you should get. Also that output is very likely to remain stable over time. In fact, my experience with PrintTenuringDistrubtion is just that. The output has remained stable for years. I cannot say that about any overly broad category of GC logging such as PrintGCDetails (which I would equate to a level). If I have another tag say, gc.collection.scavenge I should get information about scavenges. Again purpose is very clear from both a producer/consumer POV. I can take age data or scavenge or both or gc.collection.* to get all sub categories for gc.collection. This looks like levels and indeed could be used to implement levels but it?s based on categories (tags). At any rate, the taxonomy is clearer and should offer guidelines that makes it easier for the developer to push the information out on the right channel while offering better stability needed for downstream tooling. > > The problem of classifying log messages exists whether you use real levels or not; "Should this log message be tagged with verbose?" vs "Should this log message be on trace level??. I think the problem can be diminished in complexity when you include the vocabulary of the domain in your tag systems. If you think of forcing predefined terms then yes, you are correct. > >> >> >>> Having high-level tags which interact with the standard "levels" seem like a natural fit to me, while the framework proposed in the JEP make it possible for each VM team to support ever more detailed tags to each subsystem where that might be appropriate. For better or worse it adds a degree of freedom. Is your concern that we need more granular tags going forward to help zooming in on specific issues, and that levels for such niche tags might seem/feel superfluous? >> If the system has been built with tags on levels then I would suggest the system has been built upside down or inside out, however you want to describe it.. in the same way that JDK logging was build upside down (first problem, super class knows about subclasses and that is a primary source of classloader leaks and that is just the first issue). >> >> Kind regards, >> Kirk >> > > I would say the system has been built with levels on tags. The tags define *what* you want to log, and the levels define *how much* information you want. The levels are sort of a filter on the tags you select. So what you are saying is that as a producer I can completely ignore levels if I so choose to do so. I do have to apologize as I?ve not been able to get through the entire implementation yet but I hope to shortly. Regards, Kirk From kirk at jclarity.com Wed Sep 9 15:13:14 2015 From: kirk at jclarity.com (Kirk Pepperdine) Date: Wed, 9 Sep 2015 17:13:14 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F01767.2080307@oracle.com> References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> <55EFED73.6070307@oracle.com> <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> <55F01767.2080307@oracle.com> Message-ID: <9278DAFD-791D-4D7E-8F35-2DB77B20A306@jclarity.com> Hi, > > At least to me, a choice between one of six possible alternatives is easier than a choice of any number of alternatives from an unlimited set. That pub/sub works with an unlimited set of subjects this doesn?t seem to represent much of a problem. That hash tags in a tweet come from an unlimited set seems to work and in fact imagine twitter trying to predefine every single hash tag. Again I urge you to look at Neal Ford?s talks on taxonomy. > >>> It should be easy to specify how much logging you want, so I don't think it would be a good idea to omit levels completely either. >> tags do not preclude levels. If people want levels they can easily be created with tags. Again Neal Ford has a number of great talks on taxonomy and in my opinion he gets it spot on when he talks about the problems with level based systems. >> >>> For example, some consumers of the logging might only be interested in a certain level of log messages, but from any component or from a broad range of components. Others might want every piece of logging belonging to a (very) specific component. >> Again, tags do a much better job at this. >>>> Next, I think there are two points of view that you have to consider. Usability for the producers and usability for the consumers. From a consumer POV what I would like to see is a logging system that promotes stability in the output. Using broad ill defined categories as they are currently defined in the JDK have historically lead to unstable output. This in turn breaks down stream tooling that maybe used to help triage system out in the wild. How does this happen? Well what gets put into a level is completely arbitrary. What goes into warning, fine, finest.. who knows? More to the point, the producers often have to make a quick decision and in their cubicles they often don?t see the entire downstream tooling chains that they are affecting when they do this even when the information being added it very valuable. I?m going to suggest a system, maybe not *the* system or even the *best* system. It?s just a simple system that people have been using in pub/sub messaging systems for years. I think the technique applies because logging is a form of journalling which is just another form of messaging (hence I suggested people looking at Peter Lawyer?s Chronicle which is designed to be used in HFT applications). We can also use ObjectName (jmx) to define tags. >>>> >>>> If you have a tag, let say, gc.agetable, it?s pretty clear what you should get. Also that output is very likely to remain stable over time. In fact, my experience with PrintTenuringDistrubtion is just that. The output has remained stable for years. I cannot say that about any overly broad category of GC logging such as PrintGCDetails (which I would equate to a level). If I have another tag say, gc.collection.scavenge I should get information about scavenges. Again purpose is very clear from both a producer/consumer POV. I can take age data or scavenge or both or gc.collection.* to get all sub categories for gc.collection. This looks like levels and indeed could be used to implement levels but it?s based on categories (tags). At any rate, the taxonomy is clearer and should offer guidelines that makes it easier for the developer to push the information out on the right channel while offering better stability needed for downstream tooling. >>> The problem of classifying log messages exists whether you use real levels or not; "Should this log message be tagged with verbose?" vs "Should this log message be on trace level??. >> I think the problem can be diminished in complexity when you include the vocabulary of the domain in your tag systems. If you think of forcing predefined terms then yes, you are correct. > > With the current proposal one can easily direct VM wide warnings and errors to a log file, the necessary configuration would be -Xlog:all=warning:out.log. Removing the concept of levels and only using domain specific tags to classify log messages would make it pretty complicated to achieve the same effect. The consumer would have to know about all the different components and their domain specific warning/error-equivalent tags in order to get the same output, or am I missing something? Well, it depends upon how you implement it. JMX has a discovery capability built into ObjectName. I can connect to MBeans that I?ve discovered. Other messaging services also have similar types of discovery services. > >>>> >>>>> Having high-level tags which interact with the standard "levels" seem like a natural fit to me, while the framework proposed in the JEP make it possible for each VM team to support ever more detailed tags to each subsystem where that might be appropriate. For better or worse it adds a degree of freedom. Is your concern that we need more granular tags going forward to help zooming in on specific issues, and that levels for such niche tags might seem/feel superfluous? >>>> If the system has been built with tags on levels then I would suggest the system has been built upside down or inside out, however you want to describe it.. in the same way that JDK logging was build upside down (first problem, super class knows about subclasses and that is a primary source of classloader leaks and that is just the first issue). >>>> >>>> Kind regards, >>>> Kirk >>>> >>> I would say the system has been built with levels on tags. The tags define *what* you want to log, and the levels define *how much* information you want. The levels are sort of a filter on the tags you select. >> So what you are saying is that as a producer I can completely ignore levels if I so choose to do so. > > Each message must be logged to a level, so it can't be ignored really. I just don't see what you mean with the inside-out argument. Whether you view it as levels on tags or tags on levels is up to you. I think you are assuming a hierarchical implementation. I need to finish working through the implementation to answer the questions I have about how pervasive levels are in the framework. If one is forced to specify a level then my opinion stands. Kind regards, Kirk From david.holmes at oracle.com Wed Sep 9 23:59:17 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 10 Sep 2015 09:59:17 +1000 Subject: [PATCH] fix zero builds for "unknown" architectures In-Reply-To: <55F064BD.6020603@oracle.com> References: <55F05BF8.3090502@ubuntu.com> <55F064BD.6020603@oracle.com> Message-ID: <55F0C7D5.2090909@oracle.com> On 10/09/2015 2:56 AM, Coleen Phillimore wrote: > > I think I like patch #1 better. I will sponsor it and make sure it > works on my machine. Thank you for fixing this on the other platforms. I think there should be a way to detect an unknown arch via the build mechanics. I would think an unknown arch would also impact includes of _ headers ... unless the "arch" is Zero in that case? In which case maybe we should be checking for Zero here? Also need to check the ARM versus ARM32 - as with PPC/PPC32/PPC64 I think plain ARM means 32-bit or 64-bit, but Aarch64 covers 64-bit. David > Coleen > > On 9/9/15 12:19 PM, Matthias Klose wrote: >> seen with jdk9 / tag jdk9-b80. zero builds which don't match one of >> the hotspot >> architectures fail to build in src/os/linux/vm/os_linux.cpp, because >> there is no >> default / or else clause: >> >> const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model >> name") >> IA64_ONLY("") SPARC_ONLY("cpu") >> ARM32_ONLY("Processor") >> PPC_ONLY("Processor") >> AARCH64_ONLY("Processor"); >> >> and: >> >> strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") >> IA64_ONLY("IA64") SPARC_ONLY("sparcv9") >> ARM32_ONLY("ARM") PPC_ONLY("PPC64") >> AARCH64_ONLY("AArch64"), >> length); >> >> attached are two alternate patches how to fix this, either by not >> using the >> *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. >> >> Two other issues: >> - The zero builds only define ARM, not ARM32, so the clause >> should be ARM_ONLY (or at least an ARM_ONLY added). >> - The cpuinfo string seems to be wrong for AMD64_ONLY. >> >> Verified that zero builds without errors with on of these patches. >> >> Attaching both patches. >> >> Matthias >> > From vitalyd at gmail.com Thu Sep 10 00:47:17 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 9 Sep 2015 20:47:17 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> <55EFED73.6070307@oracle.com> <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> Message-ID: Kirk, It sounds like you're proposing a hierarchical naming scheme for log statements; if anyone is familiar with the Graphite tool, it does something similar for metric names. You can then wildcard certain parts of the path when using its query tools. Is that what you have in mind? That's flexible but can lead to other issues. Every log line will need to know which path it's using, which can lead to an explosion of these paths as JVM devs will need to know what tags already exist for the (sub)component they're logging from. It's much easier to quickly decide what level is appropriate. Consumers will still need to know what tags exist to get the right level of logging; wildcarding parts may be too much, fully specifying all desired tags can become unwieldy. Before going further, I'll stop to confirm that this is your gist. As for Chronicle, that seems like a separate issue altogether which is the mechanics of the logging. I do agree that logging needs to be as efficient as possible as it's done synchronously. This also implies the filtering scheme has to be efficient when the JVM decides whether the log should be emitted or not. sent from my phone On Sep 9, 2015 7:31 PM, "Kirk Pepperdine" wrote: > > > On Sep 9, 2015, at 10:27 AM, Marcus Larsson > wrote: > > > > Hi, > > > > On 2015-09-08 14:47, Kirk Pepperdine wrote: > >>> On Sep 8, 2015, at 1:42 PM, Claes Redestad > wrote: > >>> > >>> > >>> > >>> On 2015-09-08 12:45, Kirk Pepperdine wrote: > >>>> Hi, > >>>> > >>>> Indeed, not all messages are hierarchical. This is one of the reasons > I originally questioned the use of log levels. For example, debugging and > informational logs may come from the same source and I certainly wouldn?t > want to be forced mix them to satisfy a forced structure. Is the intention > you can use either or? I ask because I?m not sure that mixing them with > tags will alleviate the problems I commonly see when tuning applications in > the field. > >>>> > >>>>> Log levels have always been part of the plan as far as I can tell. > Both the first and second proposal include the levels in their > descriptions. In any case, the JEP is due for integration and it is too > late for such a fundamental change to the feature now. > >>>> I would like to believe that this work is important enough to get > right and that it?s never too late for fundamental changes up until it is > integrated. I would like to believe that there are back out strategies in > the cases where implementations show unintended consequences. > >>> Can you give an example of what you believe will go wrong with > allowing a mix of levels and tags and how removing levels from the JEP > would improve things? > >> I?m slowly working through the patch but I?ve not had time to get > through all of it yet so some things mentioned here might be in the source > already? but here are some comments from experiences. > >> > >> As you have discovered, you can?t define a tag system using levels. > That said, if you have tags you can define levels but you don?t need to > explicitly support them, it just come for free out of the box. So I would > argue both the spec and implementation would be simplified by moving to a > tag system. More over, it doesn?t preclude someone defining a hierarchical > system if them so need to have one. > > > > While implementing levels using tags might simplify the implementation, > I think it would introduce more complexity for the log producers, since > there would be a much wider choice between "levels" in contrast to the > current six levels. > > This actually doesn?t introduce a complexity, it simplifies things. Again, > I think it would be advantageous to look at messaging implementations such > as Chronicle. > > It should be easy to specify how much logging you want, so I don't think > it would be a good idea to omit levels completely either. > > tags do not preclude levels. If people want levels they can easily be > created with tags. Again Neal Ford has a number of great talks on taxonomy > and in my opinion he gets it spot on when he talks about the problems with > level based systems. > > > For example, some consumers of the logging might only be interested in a > certain level of log messages, but from any component or from a broad range > of components. Others might want every piece of logging belonging to a > (very) specific component. > > Again, tags do a much better job at this. > > > >> > >> Next, I think there are two points of view that you have to consider. > Usability for the producers and usability for the consumers. From a > consumer POV what I would like to see is a logging system that promotes > stability in the output. Using broad ill defined categories as they are > currently defined in the JDK have historically lead to unstable output. > This in turn breaks down stream tooling that maybe used to help triage > system out in the wild. How does this happen? Well what gets put into a > level is completely arbitrary. What goes into warning, fine, finest.. who > knows? More to the point, the producers often have to make a quick decision > and in their cubicles they often don?t see the entire downstream tooling > chains that they are affecting when they do this even when the information > being added it very valuable. I?m going to suggest a system, maybe not > *the* system or even the *best* system. It?s just a simple system that > people have been using in pub/sub messaging systems for years. I think the > technique applies because logging is a form of journalling which is just > another form of messaging (hence I suggested people looking at Peter > Lawyer?s Chronicle which is designed to be used in HFT applications). We > can also use ObjectName (jmx) to define tags. > >> > >> If you have a tag, let say, gc.agetable, it?s pretty clear what you > should get. Also that output is very likely to remain stable over time. In > fact, my experience with PrintTenuringDistrubtion is just that. The output > has remained stable for years. I cannot say that about any overly broad > category of GC logging such as PrintGCDetails (which I would equate to a > level). If I have another tag say, gc.collection.scavenge I should get > information about scavenges. Again purpose is very clear from both a > producer/consumer POV. I can take age data or scavenge or both or > gc.collection.* to get all sub categories for gc.collection. This looks > like levels and indeed could be used to implement levels but it?s based on > categories (tags). At any rate, the taxonomy is clearer and should offer > guidelines that makes it easier for the developer to push the information > out on the right channel while offering better stability needed for > downstream tooling. > > > > The problem of classifying log messages exists whether you use real > levels or not; "Should this log message be tagged with verbose?" vs "Should > this log message be on trace level??. > > I think the problem can be diminished in complexity when you include the > vocabulary of the domain in your tag systems. If you think of forcing > predefined terms then yes, you are correct. > > > >> > >> > >>> Having high-level tags which interact with the standard "levels" seem > like a natural fit to me, while the framework proposed in the JEP make it > possible for each VM team to support ever more detailed tags to each > subsystem where that might be appropriate. For better or worse it adds a > degree of freedom. Is your concern that we need more granular tags going > forward to help zooming in on specific issues, and that levels for such > niche tags might seem/feel superfluous? > >> If the system has been built with tags on levels then I would suggest > the system has been built upside down or inside out, however you want to > describe it.. in the same way that JDK logging was build upside down (first > problem, super class knows about subclasses and that is a primary source of > classloader leaks and that is just the first issue). > >> > >> Kind regards, > >> Kirk > >> > > > > I would say the system has been built with levels on tags. The tags > define *what* you want to log, and the levels define *how much* information > you want. The levels are sort of a filter on the tags you select. > > So what you are saying is that as a producer I can completely ignore > levels if I so choose to do so. > > I do have to apologize as I?ve not been able to get through the entire > implementation yet but I hope to shortly. > > Regards, > Kirk > > From john.r.rose at oracle.com Thu Sep 10 00:48:36 2015 From: john.r.rose at oracle.com (John Rose) Date: Wed, 9 Sep 2015 17:48:36 -0700 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F03C8C.4030606@oracle.com> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> <55EFEF93.4050402@oracle.com> <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> <587fd8e63e19f2fb4239fd779132a6ad.squirrel@excalibur.xssl.net> <55F03C8C.4030606@oracle.com> Message-ID: On Sep 9, 2015, at 7:05 AM, Marcus Larsson wrote: > > The UL framework only supports writing of complete log lines. Some of the existing code prints parts of its log lines in chunks directly to the tty, which means that such logging can be interleaved by other writes to the tty causing split lines. This code will have to be reworked during the migration to the UL framework so that only complete lines are logged. Once this is done, there should be no split lines in the log anymore, so in a sense the problem will be resolved with UL. :) And, "in a sense", we can transmit existing XML-style logs by dissolving them into disconnected sequences of lines. But we can all agree that it's a pretty weak "sense", since it makes the real logs difficult or impossible to reassemble. Here's my take on the interesting relations between the new design of decorated lines and the existing need (in the compilation logs) for hierarchical markup. https://bugs.openjdk.java.net/browse/JDK-8046148#comment-13568278 However much we are charmed by the simplicity of decorated single lines, simply wishing it so will not turn them into an adequate carrier channel for the compilation logs, without some "stretch goal" work, in the direction of jumbo (multi-line) log entries. ? John From david.holmes at oracle.com Thu Sep 10 00:54:19 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 10 Sep 2015 10:54:19 +1000 Subject: [PATCH] fix zero builds for "unknown" architectures In-Reply-To: <55F0C7D5.2090909@oracle.com> References: <55F05BF8.3090502@ubuntu.com> <55F064BD.6020603@oracle.com> <55F0C7D5.2090909@oracle.com> Message-ID: <55F0D4BB.1000302@oracle.com> Is this considered too hackish (assuming it works): const char* search_string = IA32_ONLY("model name"; // ) \ AMD64_ONLY("model name"; //) \ IA64_ONLY("";//) \ SPARC_ONLY("cpu"; //) \ ARM32_ONLY("Processor"; //) \ PPC_ONLY("Processor"; //) \ AARCH64_ONLY("Processor"; //) \ "Processor"; The idea being that if any _ONLY is defined the "default" becomes part of a comment. The strncpy case could also add a const char* cpu_string = ... IA32_ONLY(...) ... Though looking at all the Elf stuff earlier in the file I have to wonder whether we can consolidate some of this arch related stuff somehow. David On 10/09/2015 9:59 AM, David Holmes wrote: > On 10/09/2015 2:56 AM, Coleen Phillimore wrote: >> >> I think I like patch #1 better. I will sponsor it and make sure it >> works on my machine. Thank you for fixing this on the other platforms. > > I think there should be a way to detect an unknown arch via the build > mechanics. I would think an unknown arch would also impact includes of > _ headers ... unless the "arch" is Zero in that case? In which > case maybe we should be checking for Zero here? > > Also need to check the ARM versus ARM32 - as with PPC/PPC32/PPC64 I > think plain ARM means 32-bit or 64-bit, but Aarch64 covers 64-bit. > > David > >> Coleen >> >> On 9/9/15 12:19 PM, Matthias Klose wrote: >>> seen with jdk9 / tag jdk9-b80. zero builds which don't match one of >>> the hotspot >>> architectures fail to build in src/os/linux/vm/os_linux.cpp, because >>> there is no >>> default / or else clause: >>> >>> const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model >>> name") >>> IA64_ONLY("") SPARC_ONLY("cpu") >>> ARM32_ONLY("Processor") >>> PPC_ONLY("Processor") >>> AARCH64_ONLY("Processor"); >>> >>> and: >>> >>> strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") >>> IA64_ONLY("IA64") SPARC_ONLY("sparcv9") >>> ARM32_ONLY("ARM") PPC_ONLY("PPC64") >>> AARCH64_ONLY("AArch64"), >>> length); >>> >>> attached are two alternate patches how to fix this, either by not >>> using the >>> *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. >>> >>> Two other issues: >>> - The zero builds only define ARM, not ARM32, so the clause >>> should be ARM_ONLY (or at least an ARM_ONLY added). >>> - The cpuinfo string seems to be wrong for AMD64_ONLY. >>> >>> Verified that zero builds without errors with on of these patches. >>> >>> Attaching both patches. >>> >>> Matthias >>> >> From coleen.phillimore at oracle.com Thu Sep 10 01:42:14 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 09 Sep 2015 21:42:14 -0400 Subject: [PATCH] fix zero builds for "unknown" architectures In-Reply-To: <55F0C7D5.2090909@oracle.com> References: <55F05BF8.3090502@ubuntu.com> <55F064BD.6020603@oracle.com> <55F0C7D5.2090909@oracle.com> Message-ID: <55F0DFF6.9010602@oracle.com> On 9/9/15 7:59 PM, David Holmes wrote: > On 10/09/2015 2:56 AM, Coleen Phillimore wrote: >> >> I think I like patch #1 better. I will sponsor it and make sure it >> works on my machine. Thank you for fixing this on the other platforms. > > I think there should be a way to detect an unknown arch via the build > mechanics. I would think an unknown arch would also impact includes of > _ headers ... unless the "arch" is Zero in that case? In > which case maybe we should be checking for Zero here? > > Also need to check the ARM versus ARM32 - as with PPC/PPC32/PPC64 I > think plain ARM means 32-bit or 64-bit, but Aarch64 covers 64-bit. I'm not sure what this comment means. The code does check ARM, but it doesn't have specific PPC checks for 32 and 64 bit. I think the PPC people verified this change works for them. +#if defined(AMD64) + strncpy(cpuinfo, "x86_64", length); +#elif defined(IA32) + strncpy(cpuinfo, "x86_32", length); +#elif defined(IA64) + strncpy(cpuinfo, "IA64", length); +#elif defined(SPARC) + strncpy(cpuinfo, "sparcv9", length); +#elif defined(ARM) + strncpy(cpuinfo, "ARM", length); +#elif defined(AARCH64) + strncpy(cpuinfo, "AArch64", length); +#elif defined(PPC) + strncpy(cpuinfo, "PPC64", length); +#elif defined(ZERO_LIBARCH) + strncpy(cpuinfo, ZERO_LIBARCH, length); +#else + strncpy(cpuinfo, "unknown", length); +#endif And your other mail suggests a default case, which both of the parts of Matthias's patch has. I was in the process of checking this in under the small change only needs one reviewer rule. Coleen > > David > >> Coleen >> >> On 9/9/15 12:19 PM, Matthias Klose wrote: >>> seen with jdk9 / tag jdk9-b80. zero builds which don't match one of >>> the hotspot >>> architectures fail to build in src/os/linux/vm/os_linux.cpp, because >>> there is no >>> default / or else clause: >>> >>> const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model >>> name") >>> IA64_ONLY("") SPARC_ONLY("cpu") >>> ARM32_ONLY("Processor") >>> PPC_ONLY("Processor") >>> AARCH64_ONLY("Processor"); >>> >>> and: >>> >>> strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") >>> IA64_ONLY("IA64") SPARC_ONLY("sparcv9") >>> ARM32_ONLY("ARM") PPC_ONLY("PPC64") >>> AARCH64_ONLY("AArch64"), >>> length); >>> >>> attached are two alternate patches how to fix this, either by not >>> using the >>> *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. >>> >>> Two other issues: >>> - The zero builds only define ARM, not ARM32, so the clause >>> should be ARM_ONLY (or at least an ARM_ONLY added). >>> - The cpuinfo string seems to be wrong for AMD64_ONLY. >>> >>> Verified that zero builds without errors with on of these patches. >>> >>> Attaching both patches. >>> >>> Matthias >>> >> From coleen.phillimore at oracle.com Thu Sep 10 03:12:53 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 09 Sep 2015 23:12:53 -0400 Subject: [PATCH] fix zero builds for "unknown" architectures In-Reply-To: <55F0DFF6.9010602@oracle.com> References: <55F05BF8.3090502@ubuntu.com> <55F064BD.6020603@oracle.com> <55F0C7D5.2090909@oracle.com> <55F0DFF6.9010602@oracle.com> Message-ID: <55F0F535.4040309@oracle.com> David pointed out to me offline that the order of ARM conditionals are incorrect, because AARCH64 is a special ARM so I've reversed them: +#if defined(AMD64) + strncpy(cpuinfo, "x86_64", length); +#elif defined(IA32) + strncpy(cpuinfo, "x86_32", length); +#elif defined(IA64) + strncpy(cpuinfo, "IA64", length); +#elif defined(SPARC) + strncpy(cpuinfo, "sparcv9", length); +#elif defined(AARCH64) + strncpy(cpuinfo, "AArch64", length); +#elif defined(ARM) + strncpy(cpuinfo, "ARM", length); +#elif defined(PPC) + strncpy(cpuinfo, "PPC64", length); +#elif defined(ZERO_LIBARCH) + strncpy(cpuinfo, ZERO_LIBARCH, length); +#else + strncpy(cpuinfo, "unknown", length); +#endif Does this look right? Thanks, Coleen On 9/9/15 9:42 PM, Coleen Phillimore wrote: > > > On 9/9/15 7:59 PM, David Holmes wrote: >> On 10/09/2015 2:56 AM, Coleen Phillimore wrote: >>> >>> I think I like patch #1 better. I will sponsor it and make sure it >>> works on my machine. Thank you for fixing this on the other >>> platforms. >> >> I think there should be a way to detect an unknown arch via the build >> mechanics. I would think an unknown arch would also impact includes >> of _ headers ... unless the "arch" is Zero in that case? In >> which case maybe we should be checking for Zero here? >> >> Also need to check the ARM versus ARM32 - as with PPC/PPC32/PPC64 I >> think plain ARM means 32-bit or 64-bit, but Aarch64 covers 64-bit. > > I'm not sure what this comment means. The code does check ARM, but > it doesn't have specific PPC checks for 32 and 64 bit. I think the > PPC people verified this change works for them. > > +#if defined(AMD64) > + strncpy(cpuinfo, "x86_64", length); > +#elif defined(IA32) > + strncpy(cpuinfo, "x86_32", length); > +#elif defined(IA64) > + strncpy(cpuinfo, "IA64", length); > +#elif defined(SPARC) > + strncpy(cpuinfo, "sparcv9", length); > +#elif defined(ARM) > + strncpy(cpuinfo, "ARM", length); > +#elif defined(AARCH64) > + strncpy(cpuinfo, "AArch64", length); > +#elif defined(PPC) > + strncpy(cpuinfo, "PPC64", length); > +#elif defined(ZERO_LIBARCH) > + strncpy(cpuinfo, ZERO_LIBARCH, length); > +#else > + strncpy(cpuinfo, "unknown", length); > +#endif > > And your other mail suggests a default case, which both of the parts > of Matthias's patch has. > > I was in the process of checking this in under the small change only > needs one reviewer rule. > > Coleen >> >> David >> >>> Coleen >>> >>> On 9/9/15 12:19 PM, Matthias Klose wrote: >>>> seen with jdk9 / tag jdk9-b80. zero builds which don't match one of >>>> the hotspot >>>> architectures fail to build in src/os/linux/vm/os_linux.cpp, because >>>> there is no >>>> default / or else clause: >>>> >>>> const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model >>>> name") >>>> IA64_ONLY("") SPARC_ONLY("cpu") >>>> ARM32_ONLY("Processor") >>>> PPC_ONLY("Processor") >>>> AARCH64_ONLY("Processor"); >>>> >>>> and: >>>> >>>> strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") >>>> IA64_ONLY("IA64") SPARC_ONLY("sparcv9") >>>> ARM32_ONLY("ARM") PPC_ONLY("PPC64") >>>> AARCH64_ONLY("AArch64"), >>>> length); >>>> >>>> attached are two alternate patches how to fix this, either by not >>>> using the >>>> *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. >>>> >>>> Two other issues: >>>> - The zero builds only define ARM, not ARM32, so the clause >>>> should be ARM_ONLY (or at least an ARM_ONLY added). >>>> - The cpuinfo string seems to be wrong for AMD64_ONLY. >>>> >>>> Verified that zero builds without errors with on of these patches. >>>> >>>> Attaching both patches. >>>> >>>> Matthias >>>> >>> > From david.holmes at oracle.com Thu Sep 10 03:44:21 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 10 Sep 2015 13:44:21 +1000 Subject: [PATCH] fix zero builds for "unknown" architectures In-Reply-To: <55F0F535.4040309@oracle.com> References: <55F05BF8.3090502@ubuntu.com> <55F064BD.6020603@oracle.com> <55F0C7D5.2090909@oracle.com> <55F0DFF6.9010602@oracle.com> <55F0F535.4040309@oracle.com> Message-ID: <55F0FC95.9050709@oracle.com> Hi Coleen, This looks right to me now. For the record though there should be a RFR email with the bug id etc. Thanks, David On 10/09/2015 1:12 PM, Coleen Phillimore wrote: > > David pointed out to me offline that the order of ARM conditionals are > incorrect, because AARCH64 is a special ARM so I've reversed them: > > +#if defined(AMD64) > + strncpy(cpuinfo, "x86_64", length); > +#elif defined(IA32) > + strncpy(cpuinfo, "x86_32", length); > +#elif defined(IA64) > + strncpy(cpuinfo, "IA64", length); > +#elif defined(SPARC) > + strncpy(cpuinfo, "sparcv9", length); > +#elif defined(AARCH64) > + strncpy(cpuinfo, "AArch64", length); > +#elif defined(ARM) > + strncpy(cpuinfo, "ARM", length); > +#elif defined(PPC) > + strncpy(cpuinfo, "PPC64", length); > +#elif defined(ZERO_LIBARCH) > + strncpy(cpuinfo, ZERO_LIBARCH, length); > +#else > + strncpy(cpuinfo, "unknown", length); > +#endif > > Does this look right? > > Thanks, > Coleen > > On 9/9/15 9:42 PM, Coleen Phillimore wrote: >> >> >> On 9/9/15 7:59 PM, David Holmes wrote: >>> On 10/09/2015 2:56 AM, Coleen Phillimore wrote: >>>> >>>> I think I like patch #1 better. I will sponsor it and make sure it >>>> works on my machine. Thank you for fixing this on the other >>>> platforms. >>> >>> I think there should be a way to detect an unknown arch via the build >>> mechanics. I would think an unknown arch would also impact includes >>> of _ headers ... unless the "arch" is Zero in that case? In >>> which case maybe we should be checking for Zero here? >>> >>> Also need to check the ARM versus ARM32 - as with PPC/PPC32/PPC64 I >>> think plain ARM means 32-bit or 64-bit, but Aarch64 covers 64-bit. >> >> I'm not sure what this comment means. The code does check ARM, but >> it doesn't have specific PPC checks for 32 and 64 bit. I think the >> PPC people verified this change works for them. >> >> +#if defined(AMD64) >> + strncpy(cpuinfo, "x86_64", length); >> +#elif defined(IA32) >> + strncpy(cpuinfo, "x86_32", length); >> +#elif defined(IA64) >> + strncpy(cpuinfo, "IA64", length); >> +#elif defined(SPARC) >> + strncpy(cpuinfo, "sparcv9", length); >> +#elif defined(ARM) >> + strncpy(cpuinfo, "ARM", length); >> +#elif defined(AARCH64) >> + strncpy(cpuinfo, "AArch64", length); >> +#elif defined(PPC) >> + strncpy(cpuinfo, "PPC64", length); >> +#elif defined(ZERO_LIBARCH) >> + strncpy(cpuinfo, ZERO_LIBARCH, length); >> +#else >> + strncpy(cpuinfo, "unknown", length); >> +#endif >> >> And your other mail suggests a default case, which both of the parts >> of Matthias's patch has. >> >> I was in the process of checking this in under the small change only >> needs one reviewer rule. >> >> Coleen >>> >>> David >>> >>>> Coleen >>>> >>>> On 9/9/15 12:19 PM, Matthias Klose wrote: >>>>> seen with jdk9 / tag jdk9-b80. zero builds which don't match one of >>>>> the hotspot >>>>> architectures fail to build in src/os/linux/vm/os_linux.cpp, because >>>>> there is no >>>>> default / or else clause: >>>>> >>>>> const char* search_string = IA32_ONLY("model name") AMD64_ONLY("model >>>>> name") >>>>> IA64_ONLY("") SPARC_ONLY("cpu") >>>>> ARM32_ONLY("Processor") >>>>> PPC_ONLY("Processor") >>>>> AARCH64_ONLY("Processor"); >>>>> >>>>> and: >>>>> >>>>> strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") >>>>> IA64_ONLY("IA64") SPARC_ONLY("sparcv9") >>>>> ARM32_ONLY("ARM") PPC_ONLY("PPC64") >>>>> AARCH64_ONLY("AArch64"), >>>>> length); >>>>> >>>>> attached are two alternate patches how to fix this, either by not >>>>> using the >>>>> *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. >>>>> >>>>> Two other issues: >>>>> - The zero builds only define ARM, not ARM32, so the clause >>>>> should be ARM_ONLY (or at least an ARM_ONLY added). >>>>> - The cpuinfo string seems to be wrong for AMD64_ONLY. >>>>> >>>>> Verified that zero builds without errors with on of these patches. >>>>> >>>>> Attaching both patches. >>>>> >>>>> Matthias >>>>> >>>> >> > From coleen.phillimore at oracle.com Thu Sep 10 03:54:44 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 09 Sep 2015 23:54:44 -0400 Subject: [PATCH] fix zero builds for "unknown" architectures In-Reply-To: <55F0FC95.9050709@oracle.com> References: <55F05BF8.3090502@ubuntu.com> <55F064BD.6020603@oracle.com> <55F0C7D5.2090909@oracle.com> <55F0DFF6.9010602@oracle.com> <55F0F535.4040309@oracle.com> <55F0FC95.9050709@oracle.com> Message-ID: <55F0FF04.9030300@oracle.com> On 9/9/15 11:44 PM, David Holmes wrote: > Hi Coleen, > > This looks right to me now. For the record though there should be a > RFR email with the bug id etc. Thanks David. For the record, I'll send an RFR. The bug was created after Matthias sent the patch. Coleen > > Thanks, > David > > On 10/09/2015 1:12 PM, Coleen Phillimore wrote: >> >> David pointed out to me offline that the order of ARM conditionals are >> incorrect, because AARCH64 is a special ARM so I've reversed them: >> >> +#if defined(AMD64) >> + strncpy(cpuinfo, "x86_64", length); >> +#elif defined(IA32) >> + strncpy(cpuinfo, "x86_32", length); >> +#elif defined(IA64) >> + strncpy(cpuinfo, "IA64", length); >> +#elif defined(SPARC) >> + strncpy(cpuinfo, "sparcv9", length); >> +#elif defined(AARCH64) >> + strncpy(cpuinfo, "AArch64", length); >> +#elif defined(ARM) >> + strncpy(cpuinfo, "ARM", length); >> +#elif defined(PPC) >> + strncpy(cpuinfo, "PPC64", length); >> +#elif defined(ZERO_LIBARCH) >> + strncpy(cpuinfo, ZERO_LIBARCH, length); >> +#else >> + strncpy(cpuinfo, "unknown", length); >> +#endif >> >> Does this look right? >> >> Thanks, >> Coleen >> >> On 9/9/15 9:42 PM, Coleen Phillimore wrote: >>> >>> >>> On 9/9/15 7:59 PM, David Holmes wrote: >>>> On 10/09/2015 2:56 AM, Coleen Phillimore wrote: >>>>> >>>>> I think I like patch #1 better. I will sponsor it and make sure it >>>>> works on my machine. Thank you for fixing this on the other >>>>> platforms. >>>> >>>> I think there should be a way to detect an unknown arch via the build >>>> mechanics. I would think an unknown arch would also impact includes >>>> of _ headers ... unless the "arch" is Zero in that case? In >>>> which case maybe we should be checking for Zero here? >>>> >>>> Also need to check the ARM versus ARM32 - as with PPC/PPC32/PPC64 I >>>> think plain ARM means 32-bit or 64-bit, but Aarch64 covers 64-bit. >>> >>> I'm not sure what this comment means. The code does check ARM, but >>> it doesn't have specific PPC checks for 32 and 64 bit. I think the >>> PPC people verified this change works for them. >>> >>> +#if defined(AMD64) >>> + strncpy(cpuinfo, "x86_64", length); >>> +#elif defined(IA32) >>> + strncpy(cpuinfo, "x86_32", length); >>> +#elif defined(IA64) >>> + strncpy(cpuinfo, "IA64", length); >>> +#elif defined(SPARC) >>> + strncpy(cpuinfo, "sparcv9", length); >>> +#elif defined(ARM) >>> + strncpy(cpuinfo, "ARM", length); >>> +#elif defined(AARCH64) >>> + strncpy(cpuinfo, "AArch64", length); >>> +#elif defined(PPC) >>> + strncpy(cpuinfo, "PPC64", length); >>> +#elif defined(ZERO_LIBARCH) >>> + strncpy(cpuinfo, ZERO_LIBARCH, length); >>> +#else >>> + strncpy(cpuinfo, "unknown", length); >>> +#endif >>> >>> And your other mail suggests a default case, which both of the parts >>> of Matthias's patch has. >>> >>> I was in the process of checking this in under the small change only >>> needs one reviewer rule. >>> >>> Coleen >>>> >>>> David >>>> >>>>> Coleen >>>>> >>>>> On 9/9/15 12:19 PM, Matthias Klose wrote: >>>>>> seen with jdk9 / tag jdk9-b80. zero builds which don't match one of >>>>>> the hotspot >>>>>> architectures fail to build in src/os/linux/vm/os_linux.cpp, because >>>>>> there is no >>>>>> default / or else clause: >>>>>> >>>>>> const char* search_string = IA32_ONLY("model name") >>>>>> AMD64_ONLY("model >>>>>> name") >>>>>> IA64_ONLY("") SPARC_ONLY("cpu") >>>>>> ARM32_ONLY("Processor") >>>>>> PPC_ONLY("Processor") >>>>>> AARCH64_ONLY("Processor"); >>>>>> >>>>>> and: >>>>>> >>>>>> strncpy(cpuinfo, IA32_ONLY("x86_32") AMD64_ONLY("x86_32") >>>>>> IA64_ONLY("IA64") SPARC_ONLY("sparcv9") >>>>>> ARM32_ONLY("ARM") PPC_ONLY("PPC64") >>>>>> AARCH64_ONLY("AArch64"), >>>>>> length); >>>>>> >>>>>> attached are two alternate patches how to fix this, either by not >>>>>> using the >>>>>> *_ONLY macros, or by defining an UNKOWN_ARCH_ONLY macro. >>>>>> >>>>>> Two other issues: >>>>>> - The zero builds only define ARM, not ARM32, so the clause >>>>>> should be ARM_ONLY (or at least an ARM_ONLY added). >>>>>> - The cpuinfo string seems to be wrong for AMD64_ONLY. >>>>>> >>>>>> Verified that zero builds without errors with on of these patches. >>>>>> >>>>>> Attaching both patches. >>>>>> >>>>>> Matthias >>>>>> >>>>> >>> >> From coleen.phillimore at oracle.com Thu Sep 10 03:57:17 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 09 Sep 2015 23:57:17 -0400 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. Message-ID: <55F0FF9D.8030309@oracle.com> Summary: add zero architectures for default cases Reviewed-by: coleenp, dholmes Contributed-by: Matthias Klose Tested on linux zero and linux x64. open webrev at http://cr.openjdk.java.net/~coleenp/8135298/ bug link https://bugs.openjdk.java.net/browse/JDK-8135298 Thanks, Coleen From david.holmes at oracle.com Thu Sep 10 04:18:55 2015 From: david.holmes at oracle.com (David Holmes) Date: Thu, 10 Sep 2015 14:18:55 +1000 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <55F0FF9D.8030309@oracle.com> References: <55F0FF9D.8030309@oracle.com> Message-ID: <55F104AF.5070009@oracle.com> Thanks Coleen! Looks good. David On 10/09/2015 1:57 PM, Coleen Phillimore wrote: > Summary: add zero architectures for default cases > Reviewed-by: coleenp, dholmes > Contributed-by: Matthias Klose > > Tested on linux zero and linux x64. > > open webrev at http://cr.openjdk.java.net/~coleenp/8135298/ > bug link https://bugs.openjdk.java.net/browse/JDK-8135298 > > Thanks, > Coleen From tobias.hartmann at oracle.com Thu Sep 10 04:51:56 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 10 Sep 2015 06:51:56 +0200 Subject: [9] RFR(S): 8080999: MemoryPoolMXBean.getUsageThresholdCount() returns incorrect value In-Reply-To: <55E813FD.8050207@oracle.com> References: <55E813FD.8050207@oracle.com> Message-ID: <55F10C6C.1010709@oracle.com> [CC'ing runtime for wider audience] On 03.09.2015 11:33, Tobias Hartmann wrote: > Hi, > > please review the following patch. > > https://bugs.openjdk.java.net/browse/JDK-8080999 > http://cr.openjdk.java.net/~thartmann/8080999/webrev.00/ > > Problem: > The compiler test [1] sets a maximum usage threshold for a code heap memory pool, allocates enough memory to hit the threshold and checks if the threshold counter was incremented (see 'CodeCacheUtils.hitUsageThreshold' [2]). On success, the allocated memory is freed and we start over again. The counter should only be incremented the first time the threshold is crossed. Subsequent threshold hits are only counted if the usage fell below the threshold value. The test times out waiting for the threshold counter to be incremented. > > The problem is that there is a race between SensorInfo::set_gauge_sensor_level() which checks if the sensor should be triggered and SensorInfo::clear() which resets the sensor once the usage falls below the threshold value. The race occurs if SensorInfo::process_pending_requests() invokes SensorInfo::clear() because there is a pending request to clear the sensor. In the meantime SensorInfo::set_gauge_sensor_level() may be called because the sensor is triggered again. We now have to remove the pending clear request because the sensor should be activated (see [3]). However, since SensorInfo::clear() does not check the value of '_pending_clear_count' again, it still clears the sensor. The sensor may then be triggered again although the threshold value was only crossed once. As a result, the output of getUsageThresholdCount() is higher than expected. > > Here is the detailed trace of events (the info in brackets is the output of SensorInfo::print): > > ... > > [on count = 1 pending_triggers = 0 pending_clears = 0] > > WB.freeCodeBlob() > LowMemoryDetector::detect_low_memory() > SensorInfo::set_gauge_sensor_level() > is_below_low -> > _pending_clear_count++; > > [on count = 1 pending_triggers = 0 pending_clears = 1] > > WB.allocateCodeBlob() > LowMemoryDetector::detect_low_memory() > SensorInfo::set_gauge_sensor_level() > is_over_high -> > _pending_trigger_count++; > _pending_clear_count = 0; <-- remove pending clear request because the sensor is triggered again > > [on count = 1 pending_triggers = 1 pending_clears = 0] > > LowMemoryDetector::process_sensor_changes() > SensorInfo::process_pending_requests() <-- still sees the old _pending_clear_count == 1 because there is a race > SensorInfo::clear() > > [off count = 2 pending_triggers = 0 pending_clears = 0] > > WB.fullGC() > LowMemoryDetector::detect_low_memory() > SensorInfo::set_gauge_sensor_level() > is_over_high -> _pending_trigger_count++; > > [off count = 2 pending_triggers = 1 pending_clears = 0] > > At this point the threshold count should be 2 because the threshold was only hit once. However, we have a pending trigger request. Now there are tow possibilities: > 1) We check the counter before the pending trigger is processed: We continue because as expected the count is 2 but we fail later because the overall count does not match the expected value (RuntimeException: Unexpected threshold usage count (assert failed: 11 == 10)). > 2) We check after the pending trigger is processed: The counter is 3 and we time out waiting for it to be 2. > > Solution: > I added asserts to the code to make sure that such race conditions are detected. I changed the implementation of SensorInfo::clear() to acquire the Service_lock and bail out if _pending_clear_count was reset in the meantime (i.e., if we lost a race to SensorInfo::set_gauge_sensor_level()). I also added a missing > > 335 _sensor_count += count; > > to SensorInfo::clear() because it may trigger counter increments as well. > > Testing: > - 15k runs of failing testcase > - JPRT > > Thanks, > Tobias > > > [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/UsageThresholdExceededTest.java > [2] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/CodeCacheUtils.java#l50 > [3] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/src/share/vm/services/lowMemoryDetector.cpp#l222 > From goetz.lindenmaier at sap.com Thu Sep 10 06:51:59 2015 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 10 Sep 2015 06:51:59 +0000 Subject: RFR(XXS): 8135179: Fix conversion warning after 8067341 In-Reply-To: References: Message-ID: <4295855A5C1DE049A61835A1887419CC41E77638@DEWDFEMB12A.global.corp.sap> Hi Volker, thanks for this fix! It's good. Could somebody please sponsor this one-liner? Thanks! Best regards, Goetz. -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Volker Simonis Sent: Dienstag, 8. September 2015 18:57 To: HotSpot Open Source Developers Subject: RFR(XXS): 8135179: Fix conversion warning after 8067341 Hi, can somebody please review and sponsor the following trivial fix: http://cr.openjdk.java.net/~simonis/webrevs/2015/8135179/ https://bugs.openjdk.java.net/browse/JDK-8135179 Change "8067341: Modify PLAB sizing algorithm to waste less" introduced the following code: size_t const cur_plab_sz = (double)total_waste_allowed / G1LastPLABAverageOccupancy; which triggers a conversion warning with older versions of GCC and potentially other compilers as well: hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp: In member function 'virtual void G1EvacStats::adjust_desired_plab_sz()': hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp:96: warning: converting to 'size_t' from 'double' make[4]: *** [g1EvacStats.o] Error 1 The warning can be easily fixed as follows: size_t const cur_plab_sz = (sizte_t)((double)total_waste_allowed / G1LastPLABAverageOccupancy); Thank you and best regards, Volker PS: as a side note I want to mention that we are currently disabling '-Wconversion' for GCC 4.3 and later because it produces too many warnings which are by default treated as errors. All the warnings produced by '-Wconversion' are about narrowing conversions which may result in a potential data loss. It may be a good idea to fix all these implicit narrowing conversions and re-enable -Wconversion warning by default. But unfortunately there's quite some code which has to be changed in order to meet the requirements of '-Wconversion'. A quick research showed that there are currently 137 different implicit narrowing conversions spread over 1785 different code locations. For more details see "8135181: Re-enable '-Wconversion' for GCC 4.3 and later" (https://bugs.openjdk.java.net/browse/JDK-8135181) which I've opened to track the issue. What's the general opinion? Would it be worth wile to fix all these conversion warnings by inserting implicit casts? From vladimir.kozlov at oracle.com Thu Sep 10 08:06:39 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 10 Sep 2015 01:06:39 -0700 Subject: [9] RFR(S): 8080999: MemoryPoolMXBean.getUsageThresholdCount() returns incorrect value In-Reply-To: <55E813FD.8050207@oracle.com> References: <55E813FD.8050207@oracle.com> Message-ID: <55F13A0F.1060208@oracle.com> Tobias, Your evaluation seems correct. Looks good to me. Thanks, Vladimir On 9/3/15 2:33 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch. > > https://bugs.openjdk.java.net/browse/JDK-8080999 > http://cr.openjdk.java.net/~thartmann/8080999/webrev.00/ > > Problem: > The compiler test [1] sets a maximum usage threshold for a code heap memory pool, allocates enough memory to hit the threshold and checks if the threshold counter was incremented (see 'CodeCacheUtils.hitUsageThreshold' [2]). On success, the allocated memory is freed and we start over again. The counter should only be incremented the first time the threshold is crossed. Subsequent threshold hits are only counted if the usage fell below the threshold value. The test times out waiting for the threshold counter to be incremented. > > The problem is that there is a race between SensorInfo::set_gauge_sensor_level() which checks if the sensor should be triggered and SensorInfo::clear() which resets the sensor once the usage falls below the threshold value. The race occurs if SensorInfo::process_pending_requests() invokes SensorInfo::clear() because there is a pending request to clear the sensor. In the meantime SensorInfo::set_gauge_sensor_level() may be called because the sensor is triggered again. We now have to remove the pending clear request because the sensor should be activated (see [3]). However, since SensorInfo::clear() does not check the value of '_pending_clear_count' again, it still clears the sensor. The sensor may then be triggered again although the threshold value was only crossed once. As a result, the output of getUsageThresholdCount() is higher than expected. > > Here is the detailed trace of events (the info in brackets is the output of SensorInfo::print): > > ... > > [on count = 1 pending_triggers = 0 pending_clears = 0] > > WB.freeCodeBlob() > LowMemoryDetector::detect_low_memory() > SensorInfo::set_gauge_sensor_level() > is_below_low -> > _pending_clear_count++; > > [on count = 1 pending_triggers = 0 pending_clears = 1] > > WB.allocateCodeBlob() > LowMemoryDetector::detect_low_memory() > SensorInfo::set_gauge_sensor_level() > is_over_high -> > _pending_trigger_count++; > _pending_clear_count = 0; <-- remove pending clear request because the sensor is triggered again > > [on count = 1 pending_triggers = 1 pending_clears = 0] > > LowMemoryDetector::process_sensor_changes() > SensorInfo::process_pending_requests() <-- still sees the old _pending_clear_count == 1 because there is a race > SensorInfo::clear() > > [off count = 2 pending_triggers = 0 pending_clears = 0] > > WB.fullGC() > LowMemoryDetector::detect_low_memory() > SensorInfo::set_gauge_sensor_level() > is_over_high -> _pending_trigger_count++; > > [off count = 2 pending_triggers = 1 pending_clears = 0] > > At this point the threshold count should be 2 because the threshold was only hit once. However, we have a pending trigger request. Now there are tow possibilities: > 1) We check the counter before the pending trigger is processed: We continue because as expected the count is 2 but we fail later because the overall count does not match the expected value (RuntimeException: Unexpected threshold usage count (assert failed: 11 == 10)). > 2) We check after the pending trigger is processed: The counter is 3 and we time out waiting for it to be 2. > > Solution: > I added asserts to the code to make sure that such race conditions are detected. I changed the implementation of SensorInfo::clear() to acquire the Service_lock and bail out if _pending_clear_count was reset in the meantime (i.e., if we lost a race to SensorInfo::set_gauge_sensor_level()). I also added a missing > > 335 _sensor_count += count; > > to SensorInfo::clear() because it may trigger counter increments as well. > > Testing: > - 15k runs of failing testcase > - JPRT > > Thanks, > Tobias > > > [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/UsageThresholdExceededTest.java > [2] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/CodeCacheUtils.java#l50 > [3] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/src/share/vm/services/lowMemoryDetector.cpp#l222 > From sgehwolf at redhat.com Thu Sep 10 08:46:45 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 10 Sep 2015 10:46:45 +0200 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <55F104AF.5070009@oracle.com> References: <55F0FF9D.8030309@oracle.com> <55F104AF.5070009@oracle.com> Message-ID: <1441874805.3458.11.camel@redhat.com> On Thu, 2015-09-10 at 14:18 +1000, David Holmes wrote: > Thanks Coleen! Looks good. > > David > > On 10/09/2015 1:57 PM, Coleen Phillimore wrote: > > Summary: add zero architectures for default cases > > Reviewed-by: coleenp, dholmes > > Contributed-by: Matthias Klose > > > > Tested on linux zero and linux x64. > > > > open webrev at http://cr.openjdk.java.net/~coleenp/8135298/ > > bug link https://bugs.openjdk.java.net/browse/JDK-8135298 > > > > Thanks, > > Coleen Thanks, Coleen, for getting this pushed! Cheers, Severin From marcus.larsson at oracle.com Thu Sep 10 09:04:49 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Thu, 10 Sep 2015 11:04:49 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F05228.7080905@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> Message-ID: <55F147B1.40505@oracle.com> Hi, On 2015-09-09 17:37, Rachel Protacio wrote: > Hi! > > For runtime logging, there are many locations where it is necessary to > have a block led by a conditional checking whether the > debug/info/trace logging is enabled with the appropriate tag. Would it > be possible to add macros to logging/log.hpp such as > > #define log_debug_is_enabled(...) log_is_enabled(Debug, __VA_ARGS__) > #define log_trace_is_enabled(...) log_is_enabled(Trace, __VA_ARGS__) > > etc.? If they are added to your framework patch, that will help Max > and me as we write our logging code. It could be done, but I think we should go either with the log_is_enabled(level, tags) macro, or individual macros for each log level like you suggest, and not both. My intention has been to minimize the number of macros introduced with this patch to reduce the risk for namespace issues. I can see that having the separate macros helps readability a little, so if we're okay with the increased number of macros this seems like a good idea to me. If we agree to go with separate macros I suggest we shorten the names and use macros like log_is_debug(...), log_is_trace(...). > > Additionally, the print format for the logging was not immediately > intuitive. For example, as I'm working on the safepoint logging, I > might have an output line like > > [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) > > Are the spaces after debug intended to line up the tags with each > other (since the longest one is warning, it seems to fit)? If so, is > that meant for readability? And what happens if the number of seconds > goes into double digits - won't the alignment be off? > Yes, decorators are padded to avoid jagged logs and help readability. Since the levels are known beforehand, a fixed padding is used for that decorator. For other decorators such as tags or timestamps, the padding will grow to the size of the longest (so far) seen decoration. This means the decorator prefix length will either stay the same or increase, but never decrease. After a while it should stabilize around some fitting length and not grow significantly. For example: [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) [0.656s][debug ][safepoint, some_other_tag] Safepoint synchronization initiated. (20) [0.657s][debug ][safepoint, ] Safepoint synchronization initiated. (20) [10.657s][debug ][safepoint, ] Safepoint synchronization initiated. (20) Thanks, Marcus > > Thanks, > Rachel > > On 9/7/2015 9:33 AM, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch adding the unified logging >> framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. >> Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which >> contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log >> call has a corresponding LogTagSet instance, which keeps a track of >> all the outputs it should write the log message to (and their >> levels). Having tags as template arguments allows mapping directly >> from a set of tags to the LogTagSet instance, which means that the >> overhead for disabled logging should be low. Currently each log >> message can be tagged with up to 5 tags, but this can be increased if >> ever required (and with C++11's variadic templates the limit can be >> removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by >> command line arguments (-Xlog) or by DCMD. Both methods will in turn >> use the LogConfiguration class to perform the parsing & >> configuration. This configuration includes iterating over all >> LogTagSet instances and updating them accordingly. The >> LogTagLevelExpression class is used to represent the selection of >> tags and levels for a given configuration request (the >> "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances >> of this class is kept in LogTagSet to track what decorators to use >> (this is the union of all decorators used by its outputs). Each log >> call will create a LogDecorations instance (note: different classes), >> which will contain the actual decoration strings for the log message. >> These decorations are used for each output the tagset is set to log >> on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with >> argument. (This allows GC logging to prefix messages of certain >> tagsets with GCId.) Prefixes are implemented using template >> specializations based on the specified tags, with the >> general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log >> outputs. LogFileStreamOutput corresponds to FILE* stream based log >> outputs. LogFileOutput builds on this and adds the file management >> and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will >> be added at a later stage. >> >> Thanks, >> Marcus > From cnewland at chrisnewland.com Thu Sep 10 09:07:57 2015 From: cnewland at chrisnewland.com (Chris Newland) Date: Thu, 10 Sep 2015 10:07:57 +0100 Subject: LogCompilation suggestion: output system locale Message-ID: <49fb1d245d3c1f113d85dff4832a7af2.squirrel@excalibur.xssl.net> Hi, It would be useful for JITWatch to know the system locale when the HotSpot log was output in order to parse some of the numeric values (stamp attribute etc.). Would you consider this patch to output the locale inside the log header tag? Kind regards, Chris diff -r 4142c190cd5c src/share/vm/utilities/ostream.cpp --- a/src/share/vm/utilities/ostream.cpp Thu Sep 03 16:14:02 2015 -0700 +++ b/src/share/vm/utilities/ostream.cpp Thu Sep 10 09:54:29 2015 +0100 @@ -35,6 +35,10 @@ #include "utilities/top.hpp" #include "utilities/xmlstream.hpp" +#include +#include +using namespace std; + extern "C" void jio_print(const char* s); // Declarationtion of jvm method outputStream::outputStream(int width) { @@ -957,6 +961,12 @@ xs->tail("release"); xs->head("info"); xs->text("%s", VM_Version::internal_vm_info_string()); xs->cr(); xs->tail("info"); + + locale l(""); + + xs->head("locale"); xs->text("%s", l.name().c_str()); xs->cr(); + xs->tail("locale"); + xs->tail("vm_version"); // Record information about the command-line invocation. xs->head("vm_arguments"); // Cf. Arguments::print_on() From thomas.schatzl at oracle.com Thu Sep 10 09:17:49 2015 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 10 Sep 2015 11:17:49 +0200 Subject: RFR(XXS): 8135179: Fix conversion warning after 8067341 In-Reply-To: References: Message-ID: <1441876669.2217.11.camel@oracle.com> Hi, On Tue, 2015-09-08 at 18:56 +0200, Volker Simonis wrote: > Hi, > > can somebody please review and sponsor the following trivial fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2015/8135179/ > https://bugs.openjdk.java.net/browse/JDK-8135179 > > Change "8067341: Modify PLAB sizing algorithm to waste less" > introduced the following code: > > size_t const cur_plab_sz = (double)total_waste_allowed / > G1LastPLABAverageOccupancy; > > which triggers a conversion warning with older versions of GCC and > potentially other compilers as well: > > hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp: In member function > 'virtual void G1EvacStats::adjust_desired_plab_sz()': > hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp:96: warning: converting > to 'size_t' from 'double' > make[4]: *** [g1EvacStats.o] Error 1 > > The warning can be easily fixed as follows: > > size_t const cur_plab_sz = (sizte_t)((double)total_waste_allowed / > G1LastPLABAverageOccupancy); looks good. Thanks for the fix. I can push the change. > Thank you and best regards, > Volker > > > PS: as a side note I want to mention that we are currently disabling > '-Wconversion' for GCC 4.3 and later because it produces too many > warnings which are by default treated as errors. All the warnings > produced by '-Wconversion' are about narrowing conversions which may > result in a potential data loss. > > It may be a good idea to fix all these implicit narrowing conversions > and re-enable -Wconversion warning by default. > > But unfortunately there's quite some code which has to be changed in > order to meet the requirements of '-Wconversion'. A quick research > showed that there are currently 137 different implicit narrowing > conversions spread over 1785 different code locations. > > For more details see "8135181: Re-enable '-Wconversion' for GCC 4.3 > and later" (https://bugs.openjdk.java.net/browse/JDK-8135181) which > I've opened to track the issue. > > What's the general opinion? Would it be worth wile to fix all these > conversion warnings by inserting implicit casts? I would think so that the goal is to gradually enable more warnings. I guess it has been turned off because nobody stepped up fixing these issues. :) Thanks, Thomas From kirk at kodewerk.com Thu Sep 10 09:27:28 2015 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Thu, 10 Sep 2015 11:27:28 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> <55EFED73.6070307@oracle.com> <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> Message-ID: Hi Vitaly, I think it would be much better to go towards a digital logging system. However, as difficult as it is to talk a tag system, I think it would be more difficult to talk about a digital system. > It sounds like you're proposing a hierarchical naming scheme for log statements; if anyone is familiar with the Graphite tool, it does something similar for metric names. You can then wildcard certain parts of the path when using its query tools. Is that what you have in mind? > > This is similar to what I have in mind. > That's flexible but can lead to other issues. > Agreed, there is no free lunch. The question now becomes, where do you want to pay the cost. Next how much of that cost is real? I try to answer that below by adding some context. As I mentioned before, if you don?t want to pay the cost you can simply define levels if that is what you like and that is how your taxonomy works. This isn?t about preventing people from using levels, it?s about not forcing them to use them. It?s also about the advantages that tags can cover situations that you?ve yet to imagine. In fact it seems as if tags are in because there were situations that were not imagined in the first spec. And thats ok, we can?t always imagine everything that could happen. All we can hope for is that when the unimagined happens, the system is flexible enough to cope with if. > Every log line will need to know which path it's using, which can lead to an explosion of these paths as JVM devs will need to know what tags already exist for the (sub)component they're logging from. It's much easier to quickly decide what level is appropriate. > If you put this into the context of GC logging, what is a warning log level? IMHO, it makes no sense. I can?t even put a reliable warning metric into Censum that would be a warning in all circumstances. > Consumers will still need to know what tags exist to get the right level of logging; wildcarding parts may be too much, fully specifying all desired tags can become unwieldy. > > Indeed, on the surface it looks like it can. However if you look at the current logging system in GC there are not that many flags that most people would be interested in. Certainly the amount isn?t unwieldy. Converting that to a tag system would be straight forward. Moving to levels? I wouldn?t even no where to start. So theory doesn?t really meet practice in my experience. I?m happy that we?re able to engage in this conversation though I?m not sure how useful it is. What I?m trying to impart here is my experiences tuning the very bad and broken logging that I often run into in many applications. It seems that my experiences are quite different than yours (as in the group that is implementing this JEP). FWIW, what I can say is the most successful engagements have been where I?ve been able to convince teams that logging should be an architectural level that involves conversations with operations and/or support groups to make sure that their needs are meet. Most of that work involves deriving a taxonomy or a way of categorizing the types of log messages that need to be produced. None of the current logging Java logging frameworks adequately support the outputs of this activity. Defining custom levels doesn?t work and with some frameworks leads to some very disastrous result *with no fault to the application developers* As you can see, the shape of the name space and/or the names used to describe the categories of messages is, or should be beyond the scope of this JEP. Unfortunately it isn?t. IMHO, although this version of the JEP is much much better than the original version, it has still over reached. It?s over reached because from the start the wrong logging model was used to help define the JEP. If you absolutely disagree I?ll let you push on without interruption. > Before going further, I'll stop to confirm that this is your gist. > > As for Chronicle, that seems like a separate issue altogether which is the mechanics of the logging. I do agree that logging needs to be as efficient as possible as it's done synchronously. This also implies the filtering scheme has to be efficient when the JVM decides whether the log should be emitted or not. > > Yes, performance is a different (yet some what related) issue. My apology for not clearly articulating that. The idea behind Chronicle is to not filter messages. In fact it drops them on disk in a raw format. If you want to filter and reformat you can do that out of band. When logging latencies are significant, which they quite often are, I first look to how to reduce volume and then I work to strip out all the decorators and filtering. In fact, I?ve been recommending and moving apps to use more compact, information dense binary formats. All of this gets particularly critical in virtualized environments where disk is typically NAS. In all cases the less you do, the better things get. Kind regards, Kirk Pepperdine From volker.simonis at gmail.com Thu Sep 10 10:56:40 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 10 Sep 2015 12:56:40 +0200 Subject: RFR(XXS): 8135179: Fix conversion warning after 8067341 In-Reply-To: <1441876669.2217.11.camel@oracle.com> References: <1441876669.2217.11.camel@oracle.com> Message-ID: On Thu, Sep 10, 2015 at 11:17 AM, Thomas Schatzl wrote: > Hi, > > On Tue, 2015-09-08 at 18:56 +0200, Volker Simonis wrote: >> Hi, >> >> can somebody please review and sponsor the following trivial fix: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2015/8135179/ >> https://bugs.openjdk.java.net/browse/JDK-8135179 >> >> Change "8067341: Modify PLAB sizing algorithm to waste less" >> introduced the following code: >> >> size_t const cur_plab_sz = (double)total_waste_allowed / >> G1LastPLABAverageOccupancy; >> >> which triggers a conversion warning with older versions of GCC and >> potentially other compilers as well: >> >> hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp: In member function >> 'virtual void G1EvacStats::adjust_desired_plab_sz()': >> hotspot-rt/src/share/vm/gc/g1/g1EvacStats.cpp:96: warning: converting >> to 'size_t' from 'double' >> make[4]: *** [g1EvacStats.o] Error 1 >> >> The warning can be easily fixed as follows: >> >> size_t const cur_plab_sz = (sizte_t)((double)total_waste_allowed / >> G1LastPLABAverageOccupancy); > > looks good. Thanks for the fix. I can push the change. > Hi Thomas, thanks a lot for your help. Regards, Volker >> Thank you and best regards, >> Volker >> >> >> PS: as a side note I want to mention that we are currently disabling >> '-Wconversion' for GCC 4.3 and later because it produces too many >> warnings which are by default treated as errors. All the warnings >> produced by '-Wconversion' are about narrowing conversions which may >> result in a potential data loss. >> >> It may be a good idea to fix all these implicit narrowing conversions >> and re-enable -Wconversion warning by default. >> >> But unfortunately there's quite some code which has to be changed in >> order to meet the requirements of '-Wconversion'. A quick research >> showed that there are currently 137 different implicit narrowing >> conversions spread over 1785 different code locations. >> >> For more details see "8135181: Re-enable '-Wconversion' for GCC 4.3 >> and later" (https://bugs.openjdk.java.net/browse/JDK-8135181) which >> I've opened to track the issue. >> >> What's the general opinion? Would it be worth wile to fix all these >> conversion warnings by inserting implicit casts? > > I would think so that the goal is to gradually enable more warnings. I > guess it has been turned off because nobody stepped up fixing these > issues. :) > > Thanks, > Thomas > > From tobias.hartmann at oracle.com Thu Sep 10 11:01:17 2015 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 10 Sep 2015 13:01:17 +0200 Subject: [9] RFR(S): 8080999: MemoryPoolMXBean.getUsageThresholdCount() returns incorrect value In-Reply-To: <55F13A0F.1060208@oracle.com> References: <55E813FD.8050207@oracle.com> <55F13A0F.1060208@oracle.com> Message-ID: <55F162FD.4080206@oracle.com> Thanks, Vladimir! Best, Tobias On 10.09.2015 10:06, Vladimir Kozlov wrote: > Tobias, > > Your evaluation seems correct. Looks good to me. > > Thanks, > Vladimir > > On 9/3/15 2:33 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch. >> >> https://bugs.openjdk.java.net/browse/JDK-8080999 >> http://cr.openjdk.java.net/~thartmann/8080999/webrev.00/ >> >> Problem: >> The compiler test [1] sets a maximum usage threshold for a code heap memory pool, allocates enough memory to hit the threshold and checks if the threshold counter was incremented (see 'CodeCacheUtils.hitUsageThreshold' [2]). On success, the allocated memory is freed and we start over again. The counter should only be incremented the first time the threshold is crossed. Subsequent threshold hits are only counted if the usage fell below the threshold value. The test times out waiting for the threshold counter to be incremented. >> >> The problem is that there is a race between SensorInfo::set_gauge_sensor_level() which checks if the sensor should be triggered and SensorInfo::clear() which resets the sensor once the usage falls below the threshold value. The race occurs if SensorInfo::process_pending_requests() invokes SensorInfo::clear() because there is a pending request to clear the sensor. In the meantime SensorInfo::set_gauge_sensor_level() may be called because the sensor is triggered again. We now have to remove the pending clear request because the sensor should be activated (see [3]). However, since SensorInfo::clear() does not check the value of '_pending_clear_count' again, it still clears the sensor. The sensor may then be triggered again although the threshold value was only crossed once. As a result, the output of getUsageThresholdCount() is higher than expected. >> >> Here is the detailed trace of events (the info in brackets is the output of SensorInfo::print): >> >> ... >> >> [on count = 1 pending_triggers = 0 pending_clears = 0] >> >> WB.freeCodeBlob() >> LowMemoryDetector::detect_low_memory() >> SensorInfo::set_gauge_sensor_level() >> is_below_low -> >> _pending_clear_count++; >> >> [on count = 1 pending_triggers = 0 pending_clears = 1] >> >> WB.allocateCodeBlob() >> LowMemoryDetector::detect_low_memory() >> SensorInfo::set_gauge_sensor_level() >> is_over_high -> >> _pending_trigger_count++; >> _pending_clear_count = 0; <-- remove pending clear request because the sensor is triggered again >> >> [on count = 1 pending_triggers = 1 pending_clears = 0] >> >> LowMemoryDetector::process_sensor_changes() >> SensorInfo::process_pending_requests() <-- still sees the old _pending_clear_count == 1 because there is a race >> SensorInfo::clear() >> >> [off count = 2 pending_triggers = 0 pending_clears = 0] >> >> WB.fullGC() >> LowMemoryDetector::detect_low_memory() >> SensorInfo::set_gauge_sensor_level() >> is_over_high -> _pending_trigger_count++; >> >> [off count = 2 pending_triggers = 1 pending_clears = 0] >> >> At this point the threshold count should be 2 because the threshold was only hit once. However, we have a pending trigger request. Now there are tow possibilities: >> 1) We check the counter before the pending trigger is processed: We continue because as expected the count is 2 but we fail later because the overall count does not match the expected value (RuntimeException: Unexpected threshold usage count (assert failed: 11 == 10)). >> 2) We check after the pending trigger is processed: The counter is 3 and we time out waiting for it to be 2. >> >> Solution: >> I added asserts to the code to make sure that such race conditions are detected. I changed the implementation of SensorInfo::clear() to acquire the Service_lock and bail out if _pending_clear_count was reset in the meantime (i.e., if we lost a race to SensorInfo::set_gauge_sensor_level()). I also added a missing >> >> 335 _sensor_count += count; >> >> to SensorInfo::clear() because it may trigger counter increments as well. >> >> Testing: >> - 15k runs of failing testcase >> - JPRT >> >> Thanks, >> Tobias >> >> >> [1] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/UsageThresholdExceededTest.java >> [2] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/test/compiler/codecache/jmx/CodeCacheUtils.java#l50 >> [3] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/da1c9ea76ce5/src/share/vm/services/lowMemoryDetector.cpp#l222 >> From coleen.phillimore at oracle.com Thu Sep 10 12:18:22 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 10 Sep 2015 08:18:22 -0400 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <1441874805.3458.11.camel@redhat.com> References: <55F0FF9D.8030309@oracle.com> <55F104AF.5070009@oracle.com> <1441874805.3458.11.camel@redhat.com> Message-ID: <55F1750E.1060509@oracle.com> On 9/10/15 4:46 AM, Severin Gehwolf wrote: > On Thu, 2015-09-10 at 14:18 +1000, David Holmes wrote: >> Thanks Coleen! Looks good. >> >> David >> >> On 10/09/2015 1:57 PM, Coleen Phillimore wrote: >>> Summary: add zero architectures for default cases >>> Reviewed-by: coleenp, dholmes >>> Contributed-by: Matthias Klose >>> >>> Tested on linux zero and linux x64. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8135298/ >>> bug link https://bugs.openjdk.java.net/browse/JDK-8135298 >>> >>> Thanks, >>> Coleen > Thanks, Coleen, for getting this pushed! You're welcome. As I said in the patch thread, zero doesn't build again on hs-rt because of a GC change. Coleen > > Cheers, > Severin > From sgehwolf at redhat.com Thu Sep 10 12:28:21 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 10 Sep 2015 14:28:21 +0200 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <55F1750E.1060509@oracle.com> References: <55F0FF9D.8030309@oracle.com> <55F104AF.5070009@oracle.com> <1441874805.3458.11.camel@redhat.com> <55F1750E.1060509@oracle.com> Message-ID: <1441888101.3458.35.camel@redhat.com> On Thu, 2015-09-10 at 08:18 -0400, Coleen Phillimore wrote: > > On 9/10/15 4:46 AM, Severin Gehwolf wrote: > > On Thu, 2015-09-10 at 14:18 +1000, David Holmes wrote: > >> Thanks Coleen! Looks good. > >> > >> David > >> > >> On 10/09/2015 1:57 PM, Coleen Phillimore wrote: > >>> Summary: add zero architectures for default cases > >>> Reviewed-by: coleenp, dholmes > >>> Contributed-by: Matthias Klose > >>> > >>> Tested on linux zero and linux x64. > >>> > >>> open webrev at http://cr.openjdk.java.net/~coleenp/8135298/ > >>> bug link https://bugs.openjdk.java.net/browse/JDK-8135298 > >>> > >>> Thanks, > >>> Coleen > > Thanks, Coleen, for getting this pushed! > > You're welcome. As I said in the patch thread, zero doesn't build again > on hs-rt because of a GC change. Interesting. The Zero builder did not catch this yet[1] (one day old build success). Thanks for the heads-up. Cheers, Severin [1] http://builder.classpath.org/jenkins/job/OpenJDK9_hs_rt_Zero/ From kirk at kodewerk.com Thu Sep 10 12:43:42 2015 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Thu, 10 Sep 2015 14:43:42 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <587fd8e63e19f2fb4239fd779132a6ad.squirrel@excalibur.xssl.net> References: <9adc01fefc6ff77b4958a6e7d65e6a25.squirrel@excalibur.xssl.net> <55EEC481.7030605@oracle.com> <322e9bf05fb80f27698d60d0c7de3a84.squirrel@excalibur.xssl.net> <55EFEF93.4050402@oracle.com> <263F3ED8-9F4C-4902-B154-70EF8D2B66F0@kodewerk.com> <587fd8e63e19f2fb4239fd779132a6ad.squirrel@excalibur.xssl.net> Message-ID: Hi Chris, I just looked at the comments from John Rose. His concern is how UL will manage multi-line log entries. The other concern that came out of John?s comments reminded me of a number of engagements where logging rates created a synchronization bottleneck in the async appender in Log4J. I?ve run into similar problems with other logging frameworks. As it stands Censum is able to cope with mixed output that has been redirected from stdout. However that turning parsing the logs into a run to failure activity where failure is the longest path through the code. So, more garbage == longer time to parse. I don?t know if JITWatch can do something similar but I suspect you should be able to piece something together that works similarly so if a file contains other unrelated records you should be able to pass over them without harm. Kind regards, Kirk > On Sep 9, 2015, at 1:22 PM, Chris Newland wrote: > > Thanks for the offer Kirk, I'd definitely be interested in hearing your > thoughts. > > Marcus, my headache with the existing logging is different VM sources > colliding, resulting in split lines that I can't deterministically > reassemble. I believe this is caused by no locking around the tty. > > Will this problem be solved by unified logging when there is a single > destination? > > Thanks, > > Chris > > On Wed, September 9, 2015 10:36, Kirk Pepperdine wrote: >> Hi Chris, >> >> >> My understanding is that current log formats won? change unless the >> authors of the log messages decide to change it them selves. I?e already >> started making changes to Censum? parsers in preparation for the >> impending mess that will come as users start to sort out how to best use >> it. I? happy to share some thoughts with you if you like. >> >> I? not so happy with the mixing of levels and tags and decorators and >> the other things in there. High performance journaling requires that you >> get out of the way and stay out of the way. Adding all this extra >> formatting and bits definitely puts you in the way. However, having a >> unified logging framework is long overdue so.. we can go back and forth >> on the details of the implementation but the general direction is good. >> >> Regards, >> Kirk >> >> >>> On Sep 9, 2015, at 10:36 AM, Marcus Larsson >>> wrote: >>> >>> >>> Hi, >>> >>> >>> On 2015-09-09 09:48, Chris Newland wrote: >>> >>>> Hi Marcus, >>>> >>>> >>>> Thanks for your answers. >>>> >>>> >>>> The nested XML structure of LogCompilation supports the description >>>> of arbitrarily complicated JIT parsing chains. >>>> >>>> The JIT compiler can log the start of a compilation with an opening >>>> XML >>>> tag and then hand off to the various subsystems (inlining, escape >>>> analysis, reg alloc etc.) which output child XML nodes and once they >>>> return the parent tag is closed. >>>> >>>> Do you have any thoughts on how this could be acheived in a non-XML >>>> format? >>>> >>>> Kind regards, >>>> >>>> >>>> Chris >>>> >>> >>> I'm not suggesting we go away from the XML format, I just mean that the >>> current XML-producing code could be modified to use the unified logging >>> as its output instead of a file. UL could be happily unaware of the >>> fact that XML is being written to some log file, and there won't be any >>> need to implement XML support in the UL framework itself. >>> >>> Thanks, >>> Marcus >>> >>> >>>> >>>>>> Once implemented, will the current output formats still be >>>>>> available? >>>>>> >>>>> Currently there is no specific support for XML output in the >>>>> unified logging framework. Output can be directed either to stdout, >>>>> stderr or plain text files. I'm not entirely sure how the migration >>>>> of the XML logging will be done. Perhaps it will be preceded by an >>>>> RFE to add better support >>>>> for XML outputs in the unified logging framework, or it might just >>>>> be treated as a regular text file. >>>>> >>>>> Thanks, >>>>> Marcus >>>>> >>>>> >>> >> >> > > From bengt.rutisson at oracle.com Thu Sep 10 12:37:22 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Thu, 10 Sep 2015 14:37:22 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <55F0BA00.5000704@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> Message-ID: <55F17982.3080606@oracle.com> Hi Jon, Thanks for looking at this! On 2015-09-10 01:00, Jon Masamitsu wrote: > Bengt, > > When a CMS concurrent collection hands off to a STW foreground > collection, > (done in acquire_control_and_collect()), I expected a new GCId would be > used. Did I miss it? That STW collection does not go through > do_collection(). The call to acquire_control_and_collect() originates from GenCollectedHeap::collect_generation() and there is a new GCIdMark in there that will create a new GCId for the STW collection. That's the "extra" GCIdMark that Per asked about in his review. *But* I wanted to verify that it worked properly since you asked about it and I realized that there is another bug. The GCId that is set up for the concurrent cycle, the one set up in ConcurrentMarkSweepThread::run(), will is still active over the complete STW foreground collection. That's fine in my model since the new GCIdMark in GenCollectedHeap::collect_generation() will cache that GCId. But the ConcurrentMarkSweep thread is not completely idle even though control has been left to the foreground collection in acquire_control_and_collect(). So, there is some logging done (by fore example CMSPhaseAccounting) that is done at the same time as the foreground collection. This logging will now use the foreground GCId instead of the CMS GCId. I haven't had time to dig in to the details of that just yet. But this is an unintended change of the logging, so I would like to fix it. Hopefully I'll have an updated webrev tomorrow. Thanks, Bengt > > Jon > > On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >> >> Hi again, >> >> I found an issue with how G1 young collections kick off concurrent >> marks. There is a short window where we might have two active GC IDs >> at the same time. I've fixed this and updated the webrevs. In case >> anyone had already started looking at the webrevs you need to do a >> refresh now. The fix is in >> G1CollectedHeap::do_collection_pause_at_safepoint(). >> >> Thanks, >> Bengt >> >> On 09/09/15 13:38, Bengt Rutisson wrote: >>> >>> Hi Per, >>> >>> Thanks for looking at this! >>> >>> On 2015-09-08 15:23, Per Liden wrote: >>>> Hi Bengt, >>>> >>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> This is mostly a GC related patch, but it adds a field to the Thread >>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>> >>>>> Could I have a couple of reviews for this patch? >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>> >>>> >>>> Looks good. I think this is a nice simplification, especially for >>>> G1's concurrent parts. Just two comments: >>>> >>>> genCollectedHeap.cpp: >>>> --------------------- >>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in different >>>> scopes. Do we really want that second mark? >>> >>> >>> We potentially do two GCs in GenCollectedHeap::do_collection(). >>> First a young GC and then potentially a full GC. These two should >>> have different GC IDs. So, yes, we need two GCIdMarks in this >>> method. The scoping could be better, but I think that is a >>> refactoring that should be done separately from my current patch. >>> I'll talk to Jesper about it since he has been cleaning up this code >>> lately. >>> >>>> >>>> >>>> gcId.cpp: >>>> -------- >>>> - I think you might have left currentNamedthread() in there. You >>>> probably just want to use Thread::current() instead. >>> >>> Yes, good catch. Thanks. >>> >>> However, after thinking some more about the implications of moving >>> the GC ID out from the GCTracer I figured it would be possible to >>> just store the previous GC ID in the GCIdMark and then restore it in >>> the destructor of that class. That way we don't have to store >>> anything in the Thread class but can still have both a concurrent GC >>> ID and a STW GC ID active at the same time. This also removes the >>> need to copy the GC ID to the worker tasks. >>> >>> Here's an updated webrev with a solution that does not add anything >>> to the Thread class: >>> >>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>> >>> And here's a diff compared to the last version: >>> >>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>> >>> Unfotunately the webrev tool had some bad luck when creating the >>> diff webrev so at least the g1CollectedHeap.cpp seem to contain the >>> complete changes rather than just the diff compared to the 01 >>> version. The rest of the diff looks correct as far as I can tell. >>> >>> Thanks, >>> Bengt >>>> >>>> >>>> cheers, >>>> /Per >>>> >>>> >>>>> Currently the GC ID, that is used for event tracing and logging, is >>>>> stored in the GCTracer object. That has caused some minor problems >>>>> since >>>>> there are multiple GCTracers active at the same time. The correct >>>>> GC IDs >>>>> need to be passed around in many places. >>>>> >>>>> For some upcoming GC logging changes I would like to have a more >>>>> consistent way of finding the currently active GC ID. I've played >>>>> around >>>>> with a couple of different solutions, but eventually I found that the >>>>> simplest solution is to store the current GC ID in the thread. >>>>> That way >>>>> there is a single way to look it up in all places. It is also fairly >>>>> straight forward to set it up. >>>>> >>>>> I've reworked the GCId class a bit to support this and I've >>>>> introduced a >>>>> GCIdMark class that is a scoped object that helps to set up and tear >>>>> down a current GC ID. By moving the GC ID out from the GCTracer >>>>> class I >>>>> got rid of a few minor issues as well. One is that we no longer >>>>> need to >>>>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>>>> before since we did logging *after* we had told the corresponding >>>>> GCTracer that the concurrent cycle was aborted and it had thrown >>>>> its GC >>>>> ID away. Now the GC ID can stay in scope until we have completed all >>>>> logging. >>>>> >>>>> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC >>>>> used >>>>> to have to create a new GC ID since their logging was done before >>>>> we had >>>>> the correct GCTracer set up. Now the GC ID can be available early >>>>> enough >>>>> for this logging to be using the same (and correct) GC ID as the >>>>> rest of >>>>> the GC. Same for HeapDumpAfterFullGC and >>>>> PrintClassHistogramAfterFullGC. >>>>> >>>>> I've added an uint to the Thread class to keep track of the currently >>>>> active GC ID. In the current code there are actually only >>>>> NamedThreads >>>>> that need the GC ID. So, I'm not sure where the best place is for >>>>> this >>>>> field is. But it seems like the Thread class contains most of the >>>>> data, >>>>> even data that is only used by some subclasses, so I opted for >>>>> putting >>>>> the field in Thread as opposed to in NamedThread. This opens up for >>>>> possible future extensions when Java threads may do part of the GC >>>>> work. >>>>> However, I'm open to moving the field down to NamedThread if that is >>>>> seen as better. >>>>> >>>>> In the GCTracer class there were many asserts that were checking that >>>>> the GC ID was properly set up. Mostly these assert verify that >>>>> start/end >>>>> is called correctly and that the other methods are called inside >>>>> of the >>>>> start/end scope. Now that the GC ID is moved out of the GCTracer >>>>> class >>>>> these asserts had to be dealt with. I figured there were three >>>>> ways to >>>>> handle it; just remove them, replace them with check that the GC >>>>> ID from >>>>> the current thread is correct, or implement a new status field to >>>>> keep >>>>> track of GC start/end state. Personally I'm not sure these asserts >>>>> are >>>>> valuable enough to spend time on, so I went for the first approach: >>>>> removing them. I don't think making them use the thread GC ID will be >>>>> appropriate. But if anyone feels strongly about these asserts I can >>>>> implement a separate start/end state. >>>>> >>>>> There are a few "Tasks" in the GC code that are executed by worker >>>>> threads. To make the worker threads use the correct GC ID I make sure >>>>> that the Task constructors copy the GC ID from the initiating thread >>>>> into a local variable. When the task is executed in its worker >>>>> thread it >>>>> sets up the GC ID based on the local variable. >>>>> >>>>> The proposed change does not alter any logging (except for the >>>>> small bug >>>>> fix for >>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. This >>>>> >>>>> means that no existing tests need to be updated. In particular the >>>>> test/gc/logging/TestGCId.java test passes even after these changes. >>>>> >>>>> >>>>> A big thanks to Per, who pre-reviewed these changes and came with >>>>> some >>>>> really good feedback. >>>>> >>>>> Thanks, >>>>> Bengt >>> >> > From alexander.kulyakhtin at oracle.com Thu Sep 10 14:53:56 2015 From: alexander.kulyakhtin at oracle.com (Alexander Kulyakhtin) Date: Thu, 10 Sep 2015 07:53:56 -0700 (PDT) Subject: Fwd: RFR 8134641: serviceability/dcmd/compiler/CodelistTest.java fails on sun.misc.Unsafe.getUnsafe Message-ID: <56417dd2-21fd-4bfa-8bdc-5e7e4bfec6c9@default> (Forwarding the request from the serviceability group as this test is for the compiler) Hi, Could you, please, review the following small test-only change: Issue: https://bugs.openjdk.java.net/browse/JDK-8134641 "serviceability/dcmd/compiler/CodelistTest.java fails with "Test failed on: sun.misc.Unsafe.getUnsafe()Lsun/misc/Unsafe;" Webrev: http://cr.openjdk.java.net/~akulyakh/8134641_01/index.html The test calls Jcmd (diagnostic command tool) Compiler.codelist and then parses the output, making sure that the first methods in the list is valid by reflection. However Unsafe.getUnsafe() method is hidden from reflection. Before the fix the test did not take that into account and failed whenever Unsafe.getUnsafe happened to be among the methods to be validated. The test has been changed to skip Unsafe.getUnsafe() method if present in the test input. Best regards, Alexander From vitalyd at gmail.com Thu Sep 10 15:15:55 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 10 Sep 2015 11:15:55 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: References: <55ED9210.6030904@oracle.com> <82A61449-4247-4C1E-8B09-8C12FDAD573F@kodewerk.com> <55EEA5A6.8020602@oracle.com> <55EEC9BC.3030005@oracle.com> <55EFED73.6070307@oracle.com> <9A842240-4D69-4371-A5C9-5ED352DFFC0F@gmail.com> Message-ID: > > If you put this into the context of GC logging, what is a warning log > level? IMHO, it makes no sense. I can?t even put a reliable warning metric > into Censum that would be a warning in all circumstances. There may not be a warning level for GC, I don't know. But one warning I could conceivably think of is a GC not meeting its pause time goal. Other types of warnings could be, e.g., JIT compilation bail outs, JIT compilation failures, code cache usage approaching capacity, etc. There won't be a hard and fast rule on this, but as mentioned, I suspect it's easier to decide whether something is error/warn/info/debug (or akin) than what arbitrary tag to use. Indeed, on the surface it looks like it can. However if you look at the > current logging system in GC there are not that many flags that most people > would be interested in. Certainly the amount isn?t unwieldy. Converting > that to a tag system would be straight forward. Moving to levels? I > wouldn?t even no where to start. So theory doesn?t really meet practice in > my experience. It's not just GC logging though, there are other components that are loggable and this JEP is covering the entire JVM. Overall, there are *tons* of different flags in Hotspot as it is, and granted some of them don't control logging, but the general issue of "too many knobs" is akin to "too many tags". Yes, performance is a different (yet some what related) issue. My apology > for not clearly articulating that. The idea behind Chronicle is to not > filter messages. In fact it drops them on disk in a raw format. If you want > to filter and reformat you can do that out of band. When logging latencies > are significant, which they quite often are, I first look to how to reduce > volume and then I work to strip out all the decorators and filtering. In > fact, I?ve been recommending and moving apps to use more compact, > information dense binary formats. All of this gets particularly critical in > virtualized environments where disk is typically NAS. In all cases the less > you do, the better things get. Sure, text logging is always going to be slower than proprietary binary formats. But Chronicle is meant for logging of data volumes much much larger than what a JVM is logging, and not spitting out text is more important in that case. But if you move to binary format, now you need to (a) write tools for reading it, (b) document the format so others can write tools, (c) possibly version it, etc. But ultimately, if you're logging so much that it's becoming a performance bottleneck, then I suspect there are other issues at play. Even if you write to mmap'd files or write asynchronously, if you outpace the i/o subsystem and/or hit memory pressure, OS's such as linux will panic and the entire machine can stall until all the writebacks complete. On Thu, Sep 10, 2015 at 5:27 AM, Kirk Pepperdine wrote: > Hi Vitaly, > > I think it would be much better to go towards a digital logging system. > However, as difficult as it is to talk a tag system, I think it would be > more difficult to talk about a digital system. > > It sounds like you're proposing a hierarchical naming scheme for log > statements; if anyone is familiar with the Graphite tool, it does something > similar for metric names. You can then wildcard certain parts of the path > when using its query tools. Is that what you have in mind? > > This is similar to what I have in mind. > > That's flexible but can lead to other issues. > > Agreed, there is no free lunch. The question now becomes, where do you > want to pay the cost. Next how much of that cost is real? I try to answer > that below by adding some context. As I mentioned before, if you don?t want > to pay the cost you can simply define levels if that is what you like and > that is how your taxonomy works. This isn?t about preventing people from > using levels, it?s about not forcing them to use them. It?s also about the > advantages that tags can cover situations that you?ve yet to imagine. In > fact it seems as if tags are in because there were situations that were not > imagined in the first spec. And thats ok, we can?t always imagine > everything that could happen. All we can hope for is that when the > unimagined happens, the system is flexible enough to cope with if. > > Every log line will need to know which path it's using, which can lead to > an explosion of these paths as JVM devs will need to know what tags already > exist for the (sub)component they're logging from. It's much easier to > quickly decide what level is appropriate. > > If you put this into the context of GC logging, what is a warning log > level? IMHO, it makes no sense. I can?t even put a reliable warning metric > into Censum that would be a warning in all circumstances. > > Consumers will still need to know what tags exist to get the right level > of logging; wildcarding parts may be too much, fully specifying all desired > tags can become unwieldy. > > Indeed, on the surface it looks like it can. However if you look at the > current logging system in GC there are not that many flags that most people > would be interested in. Certainly the amount isn?t unwieldy. Converting > that to a tag system would be straight forward. Moving to levels? I > wouldn?t even no where to start. So theory doesn?t really meet practice in > my experience. > > I?m happy that we?re able to engage in this conversation though I?m not > sure how useful it is. What I?m trying to impart here is my experiences > tuning the very bad and broken logging that I often run into in many > applications. It seems that my experiences are quite different than yours > (as in the group that is implementing this JEP). FWIW, what I can say is > the most successful engagements have been where I?ve been able to convince > teams that logging should be an architectural level that involves > conversations with operations and/or support groups to make sure that their > needs are meet. Most of that work involves deriving a taxonomy or a way of > categorizing the types of log messages that need to be produced. None of > the current logging Java logging frameworks adequately support the outputs > of this activity. Defining custom levels doesn?t work and with some > frameworks leads to some very disastrous result *with no fault to the > application developers* As you can see, the shape of the name space and/or > the names used to describe the categories of messages is, or should be > beyond the scope of this JEP. Unfortunately it isn?t. IMHO, although this > version of the JEP is much much better than the original version, it has > still over reached. It?s over reached because from the start the wrong > logging model was used to help define the JEP. If you absolutely disagree > I?ll let you push on without interruption. > > Before going further, I'll stop to confirm that this is your gist. > > As for Chronicle, that seems like a separate issue altogether which is the > mechanics of the logging. I do agree that logging needs to be as efficient > as possible as it's done synchronously. This also implies the filtering > scheme has to be efficient when the JVM decides whether the log should be > emitted or not. > > Yes, performance is a different (yet some what related) issue. My apology > for not clearly articulating that. The idea behind Chronicle is to not > filter messages. In fact it drops them on disk in a raw format. If you want > to filter and reformat you can do that out of band. When logging latencies > are significant, which they quite often are, I first look to how to reduce > volume and then I work to strip out all the decorators and filtering. In > fact, I?ve been recommending and moving apps to use more compact, > information dense binary formats. All of this gets particularly critical in > virtualized environments where disk is typically NAS. In all cases the less > you do, the better things get. > > > Kind regards, > Kirk Pepperdine > > From rachel.protacio at oracle.com Thu Sep 10 15:53:11 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Thu, 10 Sep 2015 11:53:11 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F147B1.40505@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> Message-ID: <55F1A767.8040002@oracle.com> On 9/10/2015 5:04 AM, Marcus Larsson wrote: > Hi, > > On 2015-09-09 17:37, Rachel Protacio wrote: >> Hi! >> >> For runtime logging, there are many locations where it is necessary >> to have a block led by a conditional checking whether the >> debug/info/trace logging is enabled with the appropriate tag. Would >> it be possible to add macros to logging/log.hpp such as >> >> #define log_debug_is_enabled(...) log_is_enabled(Debug, __VA_ARGS__) >> #define log_trace_is_enabled(...) log_is_enabled(Trace, __VA_ARGS__) >> >> etc.? If they are added to your framework patch, that will help Max >> and me as we write our logging code. > > It could be done, but I think we should go either with the > log_is_enabled(level, tags) macro, or individual macros for each log > level like you suggest, and not both. My intention has been to > minimize the number of macros introduced with this patch to reduce the > risk for namespace issues. I can see that having the separate macros > helps readability a little, so if we're okay with the increased number > of macros this seems like a good idea to me. > > If we agree to go with separate macros I suggest we shorten the names > and use macros like log_is_debug(...), log_is_trace(...). Oh, I see what you're saying. On second thought, I'm fine with using log_is_enabled(Level, tags) - I think it is equally clear. However, I would like to bring up one other point. In the block-code situations, it is currently necessary to write code that looks like if (log_is_enabled(Debug, )) { log_debug()("Logging statement to print"); } Which redundantly executes log_is_enabled() in the third line. Honestly, I'm okay with that. The issue is that in cases where it is necessary to pass the logging output stream to another function, the code ends up looking like if (log_is_enabled(Debug, )) { ResourceMark rm; print_on(LogHandle()::debug_stream()); } or even LogHandle() log; if (log.is_debug()) { ResourceMark rm; print_on(log.debug_stream()); } Is that correct? Neither seems ideal since they add extra clutter to the code and all these three examples are somewhat inconsistent with each other. Is there a cleaner way of doing this that is already in the code or that could be added? > >> >> Additionally, the print format for the logging was not immediately >> intuitive. For example, as I'm working on the safepoint logging, I >> might have an output line like >> >> [0.655s][debug ][safepoint] Safepoint synchronization initiated. >> (20) >> >> Are the spaces after debug intended to line up the tags with each >> other (since the longest one is warning, it seems to fit)? If so, is >> that meant for readability? And what happens if the number of seconds >> goes into double digits - won't the alignment be off? >> > > Yes, decorators are padded to avoid jagged logs and help readability. > Since the levels are known beforehand, a fixed padding is used for > that decorator. For other decorators such as tags or timestamps, the > padding will grow to the size of the longest (so far) seen decoration. > This means the decorator prefix length will either stay the same or > increase, but never decrease. After a while it should stabilize around > some fitting length and not grow significantly. > > For example: > [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) > [0.656s][debug ][safepoint, some_other_tag] Safepoint synchronization > initiated. (20) > [0.657s][debug ][safepoint, ] Safepoint synchronization > initiated. (20) > [10.657s][debug ][safepoint, ] Safepoint > synchronization initiated. (20) \ Ok, thanks for clarifying. Rachel > > > Thanks, > Marcus > >> >> Thanks, >> Rachel >> >> On 9/7/2015 9:33 AM, Marcus Larsson wrote: >>> Hi, >>> >>> Please review the following patch adding the unified logging >>> framework to hotspot. >>> >>> JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>> >>> See the JEP description for a general overview of the new feature. >>> Below are some notes on the implementation. >>> >>> The patch adds the new 'share/vm/logging' subdirectory containing >>> the unified logging framework. The main entry point is log.hpp, >>> which contains the necessary macros and definitions to use the >>> framework. >>> >>> Log tags are defined/listed in logTag.hpp, and are passed as >>> template arguments to the Log class. Every combination of tags used >>> in a log call has a corresponding LogTagSet instance, which keeps a >>> track of all the outputs it should write the log message to (and >>> their levels). Having tags as template arguments allows mapping >>> directly from a set of tags to the LogTagSet instance, which means >>> that the overhead for disabled logging should be low. Currently each >>> log message can be tagged with up to 5 tags, but this can be >>> increased if ever required (and with C++11's variadic templates the >>> limit can be removed completely). >>> >>> The LogConfiguration class keeps track of configured outputs >>> (stdout, stderr, and possible file outputs). Configuration is done >>> either by command line arguments (-Xlog) or by DCMD. Both methods >>> will in turn use the LogConfiguration class to perform the parsing & >>> configuration. This configuration includes iterating over all >>> LogTagSet instances and updating them accordingly. The >>> LogTagLevelExpression class is used to represent the selection of >>> tags and levels for a given configuration request (the >>> "what"-expression). >>> >>> The LogDecorators class contains a selection of decorators. >>> Instances of this class is kept in LogTagSet to track what >>> decorators to use (this is the union of all decorators used by its >>> outputs). Each log call will create a LogDecorations instance (note: >>> different classes), which will contain the actual decoration strings >>> for the log message. These decorations are used for each output the >>> tagset is set to log on, and are then discarded. >>> >>> The LogPrefix class allows messages of specific sets of tags to be >>> prefixed. The prefix should supply a printf-style format with >>> argument. (This allows GC logging to prefix messages of certain >>> tagsets with GCId.) Prefixes are implemented using template >>> specializations based on the specified tags, with the >>> general/unspecialized case giving an empty prefix. >>> >>> The LogOutput class defines the interface for all types of log >>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>> outputs. LogFileOutput builds on this and adds the file management >>> and log rotation support. >>> >>> A simple jtreg test is included in this patch. Additional tests will >>> be added at a later stage. >>> >>> Thanks, >>> Marcus >> > From aph at redhat.com Thu Sep 10 16:05:43 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 10 Sep 2015 17:05:43 +0100 Subject: 8136165: AARCH64: Tidy up compiled native calls Message-ID: <55F1AA57.70307@redhat.com> Compiled native calls are rather messy at the moment, with uncommon code mixed in with the usual path, causing a lot of branching. This patch does little more than move things around, ensuing that the fast path is straight through from entry to exit. http://cr.openjdk.java.net/~aph/8136165-1/ Andrew. From aph at redhat.com Thu Sep 10 17:19:16 2015 From: aph at redhat.com (Andrew Haley) Date: Thu, 10 Sep 2015 18:19:16 +0100 Subject: 8136165: AARCH64: Tidy up compiled native calls In-Reply-To: <55F1AA57.70307@redhat.com> References: <55F1AA57.70307@redhat.com> Message-ID: <55F1BB94.4080408@redhat.com> I withdraw this patch: I've seen some regressions. Andrew. From igor.ignatyev at oracle.com Thu Sep 10 17:23:55 2015 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 10 Sep 2015 20:23:55 +0300 Subject: RFR(XS): 8132980: Improve stability of whitebox methods getCodeBlob and freeCodeBlob In-Reply-To: <55F06B7D.2020309@oracle.com> References: <55EF0ED4.2030301@oracle.com> <55F0617A.60003@oracle.com> <55F06B7D.2020309@oracle.com> Message-ID: <55F1BCAB.4010309@oracle.com> Kirill, the change looks good to me. thanks for fixing that. Igor On 09/09/2015 08:25 PM, Kirill Zhaldybin wrote: > Stanislav, > > "addr == 0" in freeCodeBlob is just ignored since it's traditional for > C/C++ to allow free(0) but when we are trying to read something from > addr == 0 in getCodeBlob it looks like error (and crash JVM actually). > > Thank you. > > Regards. Kirill > > On 09.09.2015 19:42, Stas Smirnov wrote: >> Hi, >> >> changes looks good, one small remark, in one case when "addr == 0" you >> just make a return and in another case you THROW_MSG_NULL is there any >> specific reason for a different way of handling the same condition? >> >> On 08.09.2015 19:37, Kirill Zhaldybin wrote: >>> Hi! >>> >>> Could you please review the following patch for WB methods? >>> >>> I added checks for negative size and null address values (where >>> applicable) for getCodeBlob, allocateCodeBlob and freeCodeBlob, added >>> regression test. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8132980 >>> Webrev: >>> hotspot: >>> http://cr.openjdk.java.net/~kzhaldyb/webrevs/JDK-8132980/webrev.00/ >>> >>> Thanks, >>> Kirill >> > From ioi.lam at oracle.com Thu Sep 10 17:57:07 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 10 Sep 2015 10:57:07 -0700 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55EFE0B2.2060100@oracle.com> References: <55ED9210.6030904@oracle.com> <55EF4492.6000904@oracle.com> <55EF4AC7.40809@oracle.com> <55EFE0B2.2060100@oracle.com> Message-ID: <55F1C473.7070405@oracle.com> I have two issues: [1] There's a limit of 512 characters: void Log::vwrite(const char* fmt, va_list args) { char buf[LogBufferSize]; static const size_t LogBufferSize = 512; I would need to log the names of loaded classes, which could be much longer. Would it be possible to add a new function like LogHandle(classload) log; log.puts("a very very very long string .........."); [2] Although the Log::write function is declared with ATTRIBUTE_PRINTF, it doesn't seem to do the checking. static void write(const char* fmt, ...) ATTRIBUTE_PRINTF(1, 2); I added these void LogConfiguration::post_initialize() { assert(LogConfiguration_lock != NULL, "Lock must be initialized before post-initialization"); LogDiagnosticCommand::registerCommand(); LogHandle(logging) log; log.info("Log configuration fully initialized."); + log_info(logging)("hello %s"); + tty->print_cr("hello %s"); I get a warning on the tty->print_cr line but not on the log_info(logging) line. **NOTICE** Dtrace support disabled: This JDK does not support SDT probes /home/iklam/jdk/ul/hotspot/src/share/vm/logging/logConfiguration.cpp: In static member function `static void LogConfiguration::post_initialize()': /home/iklam/jdk/ul/hotspot/src/share/vm/logging/logConfiguration.cpp:52:27: error: format `%s' expects a matching `char*' argument [-Werror=format=] tty->print_cr("hello %s"); ^ cc1plus: all warnings being treated as errors make[4]: *** [logConfiguration.o] Error 1 My gcc is $ gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.8.2-19ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) Thanks - Ioi On 9/9/15 12:33 AM, Marcus Larsson wrote: > Hi, > > On 2015-09-08 22:53, Ioi Lam wrote: >> Also, is it possible to combine the output of two tags into a single >> file, like this? >> >> -Xlog:gc:file=out.txt -Xlog:comp:file=out.txt >> >> this way, I can have both GC and Compiler logs that's in an >> (approximate) temporal order. The alternatives are: >> >> - write both to stdout -- but I need to filter out junk that I don't >> want >> - write to different files -- but I lose the relative ordering >> between the two sets of logs >> >> From the current patch, it seems like if I assign the same file name >> to two tags, fopen() will be called twice and the result will be >> platform dependent. >> >> - Ioi > > This is indeed possible! The syntax for configuring this would be > -Xlog:gc,comp:file=out.txt. Configuring logging for the same output > multiple times only overrides/replaces the previous configuration. The > second fopen will not happen, because the log configuration will > notice the output already exists and simply reconfigure the existing > output. > > Thanks, > Marcus > >> >> On 9/8/15 1:26 PM, Ioi Lam wrote: >>> Hi Marcus, >>> >>> I am eagerly waiting for this to implement a new logging feature in >>> the VM. How much testing has been done on this patch? I can see only >>> two locations where any logs are bring generated: >>> >>> + log_error(logging)("%s", errbuf); >>> + log_error(logging)("Could not open log file '%s' (%s).\n", >>> _file_name, strerror(errno)); >>> >>> Do you have a follow-on patch that's already in work to test the >>> various functionalists of the Unified JVM Logging framework? I am >>> asking so that I can know how much surprise to expect when I start >>> using it. >>> >>> Thanks >>> - Ioi >>> >>> >>> On 9/7/15 6:33 AM, Marcus Larsson wrote: >>>> Hi, >>>> >>>> Please review the following patch adding the unified logging >>>> framework to hotspot. >>>> >>>> JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>>> >>>> See the JEP description for a general overview of the new feature. >>>> Below are some notes on the implementation. >>>> >>>> The patch adds the new 'share/vm/logging' subdirectory containing >>>> the unified logging framework. The main entry point is log.hpp, >>>> which contains the necessary macros and definitions to use the >>>> framework. >>>> >>>> Log tags are defined/listed in logTag.hpp, and are passed as >>>> template arguments to the Log class. Every combination of tags used >>>> in a log call has a corresponding LogTagSet instance, which keeps a >>>> track of all the outputs it should write the log message to (and >>>> their levels). Having tags as template arguments allows mapping >>>> directly from a set of tags to the LogTagSet instance, which means >>>> that the overhead for disabled logging should be low. Currently >>>> each log message can be tagged with up to 5 tags, but this can be >>>> increased if ever required (and with C++11's variadic templates the >>>> limit can be removed completely). >>>> >>>> The LogConfiguration class keeps track of configured outputs >>>> (stdout, stderr, and possible file outputs). Configuration is done >>>> either by command line arguments (-Xlog) or by DCMD. Both methods >>>> will in turn use the LogConfiguration class to perform the parsing >>>> & configuration. This configuration includes iterating over all >>>> LogTagSet instances and updating them accordingly. The >>>> LogTagLevelExpression class is used to represent the selection of >>>> tags and levels for a given configuration request (the >>>> "what"-expression). >>>> >>>> The LogDecorators class contains a selection of decorators. >>>> Instances of this class is kept in LogTagSet to track what >>>> decorators to use (this is the union of all decorators used by its >>>> outputs). Each log call will create a LogDecorations instance >>>> (note: different classes), which will contain the actual decoration >>>> strings for the log message. These decorations are used for each >>>> output the tagset is set to log on, and are then discarded. >>>> >>>> The LogPrefix class allows messages of specific sets of tags to be >>>> prefixed. The prefix should supply a printf-style format with >>>> argument. (This allows GC logging to prefix messages of certain >>>> tagsets with GCId.) Prefixes are implemented using template >>>> specializations based on the specified tags, with the >>>> general/unspecialized case giving an empty prefix. >>>> >>>> The LogOutput class defines the interface for all types of log >>>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>>> outputs. LogFileOutput builds on this and adds the file management >>>> and log rotation support. >>>> >>>> A simple jtreg test is included in this patch. Additional tests >>>> will be added at a later stage. >>>> >>>> Thanks, >>>> Marcus >>> >> > From coleen.phillimore at oracle.com Thu Sep 10 20:30:18 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 10 Sep 2015 16:30:18 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <55F1E85A.9090201@oracle.com> Hi Marcus, This code looks well written and documented in terms of appropriate comments in the header files. I have a few comments: 1. The NEW and strdup_check_oom() and other memory allocation functions use a combination of mtInternal and mtOther, neither of which is really going to be helpful for finding memory leaks in this. Can you add an mtLogging tag and use that? There is one strdup without an NMT tag (somewhere). 2. The header files end with #endif but no // BLAH to match the #ifdef in the header guard. Can you add all of these? 3. Nice job adding #includes in alphabetical order. This is the only file that they are not: http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logFileOutput.hpp.html 4. Some classes don't have storage allocation classes. It looks like they should be VALUE_OBJ_CLASS_SPEC. Eg. LogOutputList, LogDecorations, LogTagSet, LogTag (AllStatic), 5. In http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logTag.hpp.html there is unlikely to be a general 'rt' tag since runtime covers a broad set of different features in the vm. Around line 51 in this file, can you explain that MaxTags is that you can only give so many tags in any given logging call, not that we can only define 5 tags total. Also, giving a description of how to specify more than one tag and what it means (union or intersection of that logging tag) would be good to have here too. On line 58, can Count be "TagCount" or something a bit more descriptive? 6. In log.hpp can this really long line be broken up to something reasonable around 100 chars? template class Log { Thank you for the examples on how to use these. The template macro combinations are pretty insane. We used to not be allowed to use templates in Hotspot. 7. In general, could you add a blank line between the header guards in the file and the #include statements? #ifndef SHARE_VM_LOGGING_LOG_HPP #define SHARE_VM_LOGGING_LOG_HPP #include "logging/logLevel.hpp" 8. In logTag.hpp there is MaxTags = 5 but the log.hpp file definitions have only 5 tags. It seems like MaxTags should be specified there instead or at the top level. 9. At the end of logTagSet.hpp, I can't guess what this line is: template LogTagSet LogTagSetMapping::_tagset(T0, T1, T2, T3, T4); it looks like a constructor call. yes it is. Can you add a comment or something for people who's decoding skills with templates isn't as strong? And you have to define the static field _tagset in the .hpp file because it's a template. Can you break up the long template line for LogTagSetMapping also? This is all I have for now. This is a much larger change than I appreciated in my pre-review, but I still don't see anything bad at all. Thanks, Coleen On 9/7/15 9:33 AM, Marcus Larsson wrote: > Hi, > > Please review the following patch adding the unified logging framework > to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. > Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the > unified logging framework. The main entry point is log.hpp, which > contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template > arguments to the Log class. Every combination of tags used in a log > call has a corresponding LogTagSet instance, which keeps a track of > all the outputs it should write the log message to (and their levels). > Having tags as template arguments allows mapping directly from a set > of tags to the LogTagSet instance, which means that the overhead for > disabled logging should be low. Currently each log message can be > tagged with up to 5 tags, but this can be increased if ever required > (and with C++11's variadic templates the limit can be removed > completely). > > The LogConfiguration class keeps track of configured outputs (stdout, > stderr, and possible file outputs). Configuration is done either by > command line arguments (-Xlog) or by DCMD. Both methods will in turn > use the LogConfiguration class to perform the parsing & configuration. > This configuration includes iterating over all LogTagSet instances and > updating them accordingly. The LogTagLevelExpression class is used to > represent the selection of tags and levels for a given configuration > request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances > of this class is kept in LogTagSet to track what decorators to use > (this is the union of all decorators used by its outputs). Each log > call will create a LogDecorations instance (note: different classes), > which will contain the actual decoration strings for the log message. > These decorations are used for each output the tagset is set to log > on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be > prefixed. The prefix should supply a printf-style format with > argument. (This allows GC logging to prefix messages of certain > tagsets with GCId.) Prefixes are implemented using template > specializations based on the specified tags, with the > general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log > outputs. LogFileStreamOutput corresponds to FILE* stream based log > outputs. LogFileOutput builds on this and adds the file management and > log rotation support. > > A simple jtreg test is included in this patch. Additional tests will > be added at a later stage. > > Thanks, > Marcus From sangheon.kim at oracle.com Fri Sep 11 00:01:18 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Thu, 10 Sep 2015 17:01:18 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing Message-ID: <55F219CE.3010006@oracle.com> Hi all, Please review this patch for command-line validation for GC flags. This REDO patch is adding ranges and implementing constraint functions for GC flags. Original CR of JDK-8078555 was backout as it made a test failure from 'TestOptionsWithRanges.java'. And also there were some discussion of OOM handling. Most parts are same as JDK-8078555 except below: 1. Changed 'range' for some flags. 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags make this test unstable as it tries to allocate huge amount of memory. And below are the suggestion note for JDK-8078555: 1. Exponential notation for 'double' type variable parse: Previously there were some discussion for maximum value for double type flags from code review of JDK-8059557 and JDK-8112746. And Kim and I decided not to add upper limit unless there are problems with DBL_MAX. And as 255 is the maximum length that can be passed via command-line, we introduced exponential notation to avoid this limit. ( arguments.cpp ) 2. These GC flags ranges are not ideal ranges but ranges which don't make problem with current source code. If one flag makes some problem but hard to find good range, I added some ranges. 3. There are some constraint functions to avoid overflow. 4. Test applications are changed: as some of them assumed to be ParallelGC or to check it's output messages. 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. CR: https://bugs.openjdk.java.net/browse/JDK-8134995 Webrev: http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 Testing: JPRT, UTE(vm.quick-pcl) and test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. Thanks, Sangheon From david.holmes at oracle.com Fri Sep 11 06:06:19 2015 From: david.holmes at oracle.com (David Holmes) Date: Fri, 11 Sep 2015 16:06:19 +1000 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <55F26F5B.3020604@oracle.com> Hi Marcus, Just a quick scan of this ... thread.cpp: You moved create_vm_timer to the start of the method: 3292 jint Threads::create_vm(JavaVMInitArgs* args, bool* canTryAgain) { 3293 extern void JDK_Version_init(); 3294 3295 // Record VM creation timing statistics 3296 TraceVmCreationTime create_vm_timer; 3297 create_vm_timer.start(); but the TimeStamp class has direct and indirect dependencies on the os:: subsystem which has not been initialized yet. You get away with it when only calling start() but even that need not be guaranteed on all platforms if some configuration needs to be done regarding javaTimeMillis. Please move it to after os::init(). Thanks, David ----- On 7/09/2015 11:33 PM, Marcus Larsson wrote: > Hi, > > Please review the following patch adding the unified logging framework > to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. Below > are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the > unified logging framework. The main entry point is log.hpp, which > contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template > arguments to the Log class. Every combination of tags used in a log call > has a corresponding LogTagSet instance, which keeps a track of all the > outputs it should write the log message to (and their levels). Having > tags as template arguments allows mapping directly from a set of tags to > the LogTagSet instance, which means that the overhead for disabled > logging should be low. Currently each log message can be tagged with up > to 5 tags, but this can be increased if ever required (and with C++11's > variadic templates the limit can be removed completely). > > The LogConfiguration class keeps track of configured outputs (stdout, > stderr, and possible file outputs). Configuration is done either by > command line arguments (-Xlog) or by DCMD. Both methods will in turn use > the LogConfiguration class to perform the parsing & configuration. This > configuration includes iterating over all LogTagSet instances and > updating them accordingly. The LogTagLevelExpression class is used to > represent the selection of tags and levels for a given configuration > request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances of > this class is kept in LogTagSet to track what decorators to use (this is > the union of all decorators used by its outputs). Each log call will > create a LogDecorations instance (note: different classes), which will > contain the actual decoration strings for the log message. These > decorations are used for each output the tagset is set to log on, and > are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be > prefixed. The prefix should supply a printf-style format with argument. > (This allows GC logging to prefix messages of certain tagsets with > GCId.) Prefixes are implemented using template specializations based on > the specified tags, with the general/unspecialized case giving an empty > prefix. > > The LogOutput class defines the interface for all types of log outputs. > LogFileStreamOutput corresponds to FILE* stream based log outputs. > LogFileOutput builds on this and adds the file management and log > rotation support. > > A simple jtreg test is included in this patch. Additional tests will be > added at a later stage. > > Thanks, > Marcus From marcus.larsson at oracle.com Fri Sep 11 08:25:35 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 11 Sep 2015 10:25:35 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F1E85A.9090201@oracle.com> References: <55ED9210.6030904@oracle.com> <55F1E85A.9090201@oracle.com> Message-ID: <55F28FFF.1000803@oracle.com> Hi, On 2015-09-10 22:30, Coleen Phillimore wrote: > > Hi Marcus, > > This code looks well written and documented in terms of appropriate > comments in the header files. > > I have a few comments: > > 1. The NEW and strdup_check_oom() and other memory allocation > functions use a combination of mtInternal and mtOther, neither of > which is really going to be helpful for finding memory leaks in > this. Can you add an mtLogging tag and use that? There is one > strdup without an NMT tag (somewhere). Ah, nice catch. I'll fix this! > > 2. The header files end with #endif but no // BLAH to match the > #ifdef in the header guard. Can you add all of these? OK. > > 3. Nice job adding #includes in alphabetical order. This is the only > file that they are not: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logFileOutput.hpp.html > Darn it, thought I had them all. Will fix. > > 4. Some classes don't have storage allocation classes. It looks like > they should be VALUE_OBJ_CLASS_SPEC. Eg. LogOutputList, > LogDecorations, LogTagSet, LogTag (AllStatic), OK. > > 5. In > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logTag.hpp.html > there is unlikely to be a general 'rt' tag since runtime covers a > broad set of different features in the vm. I'll remove it along with all the other currently unused tags. If they're needed we can just add them with the change adding logging for them. > > Around line 51 in this file, can you explain that MaxTags is that you > can only give so many tags in any given logging call, not that we can > only define 5 tags total. Also, giving a description of how to > specify more than one tag and what it means (union or intersection of > that logging tag) would be good to have here too. I'll try to clarify this with some comments. > > On line 58, can Count be "TagCount" or something a bit more descriptive? All enums introduced follow this pattern, and when they're used they must be prefixed with the class they belong to, in this case LogTag::Count. I think this looks good but I can change it if you want to. If so, do you want me to make similar changes to LogLevel::Count and LogDecorators::Count as well? > > 6. In log.hpp can this really long line be broken up to something > reasonable around 100 chars? > > template T2 = LogTag::__NO_TAG, LogTagType T3 = LogTag::__NO_TAG, LogTagType T4 > = LogTag::__NO_TAG, LogTagType GuardTag = LogTag::__NO_TAG> > class Log { OK. > > Thank you for the examples on how to use these. The template macro > combinations are pretty insane. We used to not be allowed to use > templates in Hotspot. They would be simpler if we had C++11's variadic templates. Now we're sort of emulating this with up to five tags, which makes it a bit messy. > > 7. In general, could you add a blank line between the header guards in > the file and the #include statements? > > #ifndef SHARE_VM_LOGGING_LOG_HPP > #define SHARE_VM_LOGGING_LOG_HPP > #include "logging/logLevel.hpp" OK. > > 8. In logTag.hpp there is MaxTags = 5 but the log.hpp file definitions > have only 5 tags. It seems like MaxTags should be specified there > instead or at the top level. I really wanted to have it in the Log class, but then to use it one would have to specify the tag template argument (Log::MaxTags rather than Log::MaxTags). Should I rename it to LogMaxTags and make it a global constant (in log.hpp) instead? > > 9. At the end of logTagSet.hpp, I can't guess what this line is: > > template LogTagType T4> > LogTagSet LogTagSetMapping::_tagset(T0, T1, T2, > T3, T4); > > it looks like a constructor call. yes it is. Can you add a comment > or something for people who's decoding skills with templates isn't as > strong? And you have to define the static field _tagset in the .hpp > file because it's a template. I'll add comments. > > Can you break up the long template line for LogTagSetMapping also? OK. > > This is all I have for now. This is a much larger change than I > appreciated in my pre-review, but I still don't see anything bad at all. > > Thanks, > Coleen Thank you for reviewing! Regards, Marcus > > On 9/7/15 9:33 AM, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch adding the unified logging >> framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. >> Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which >> contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log >> call has a corresponding LogTagSet instance, which keeps a track of >> all the outputs it should write the log message to (and their >> levels). Having tags as template arguments allows mapping directly >> from a set of tags to the LogTagSet instance, which means that the >> overhead for disabled logging should be low. Currently each log >> message can be tagged with up to 5 tags, but this can be increased if >> ever required (and with C++11's variadic templates the limit can be >> removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by >> command line arguments (-Xlog) or by DCMD. Both methods will in turn >> use the LogConfiguration class to perform the parsing & >> configuration. This configuration includes iterating over all >> LogTagSet instances and updating them accordingly. The >> LogTagLevelExpression class is used to represent the selection of >> tags and levels for a given configuration request (the >> "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances >> of this class is kept in LogTagSet to track what decorators to use >> (this is the union of all decorators used by its outputs). Each log >> call will create a LogDecorations instance (note: different classes), >> which will contain the actual decoration strings for the log message. >> These decorations are used for each output the tagset is set to log >> on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with >> argument. (This allows GC logging to prefix messages of certain >> tagsets with GCId.) Prefixes are implemented using template >> specializations based on the specified tags, with the >> general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log >> outputs. LogFileStreamOutput corresponds to FILE* stream based log >> outputs. LogFileOutput builds on this and adds the file management >> and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will >> be added at a later stage. >> >> Thanks, >> Marcus > From marcus.larsson at oracle.com Fri Sep 11 08:25:59 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 11 Sep 2015 10:25:59 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F26F5B.3020604@oracle.com> References: <55ED9210.6030904@oracle.com> <55F26F5B.3020604@oracle.com> Message-ID: <55F29017.8080808@oracle.com> Hi David, On 2015-09-11 08:06, David Holmes wrote: > Hi Marcus, > > Just a quick scan of this ... > > thread.cpp: > > You moved create_vm_timer to the start of the method: > > 3292 jint Threads::create_vm(JavaVMInitArgs* args, bool* canTryAgain) { > 3293 extern void JDK_Version_init(); > 3294 > 3295 // Record VM creation timing statistics > 3296 TraceVmCreationTime create_vm_timer; > 3297 create_vm_timer.start(); > > but the TimeStamp class has direct and indirect dependencies on the > os:: subsystem which has not been initialized yet. You get away with > it when only calling start() but even that need not be guaranteed on > all platforms if some configuration needs to be done regarding > javaTimeMillis. Please move it to after os::init(). > > Thanks, > David > ----- Nice catch, I'll move it. Thank you for looking at this! Regards, Marcus > > On 7/09/2015 11:33 PM, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch adding the unified logging framework >> to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. Below >> are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which >> contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log call >> has a corresponding LogTagSet instance, which keeps a track of all the >> outputs it should write the log message to (and their levels). Having >> tags as template arguments allows mapping directly from a set of tags to >> the LogTagSet instance, which means that the overhead for disabled >> logging should be low. Currently each log message can be tagged with up >> to 5 tags, but this can be increased if ever required (and with C++11's >> variadic templates the limit can be removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by >> command line arguments (-Xlog) or by DCMD. Both methods will in turn use >> the LogConfiguration class to perform the parsing & configuration. This >> configuration includes iterating over all LogTagSet instances and >> updating them accordingly. The LogTagLevelExpression class is used to >> represent the selection of tags and levels for a given configuration >> request (the "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances of >> this class is kept in LogTagSet to track what decorators to use (this is >> the union of all decorators used by its outputs). Each log call will >> create a LogDecorations instance (note: different classes), which will >> contain the actual decoration strings for the log message. These >> decorations are used for each output the tagset is set to log on, and >> are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with argument. >> (This allows GC logging to prefix messages of certain tagsets with >> GCId.) Prefixes are implemented using template specializations based on >> the specified tags, with the general/unspecialized case giving an empty >> prefix. >> >> The LogOutput class defines the interface for all types of log outputs. >> LogFileStreamOutput corresponds to FILE* stream based log outputs. >> LogFileOutput builds on this and adds the file management and log >> rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will be >> added at a later stage. >> >> Thanks, >> Marcus From aph at redhat.com Fri Sep 11 09:22:11 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 11 Sep 2015 10:22:11 +0100 Subject: nmethod::make_not_entrant() and native methods Message-ID: <55F29D43.6070102@redhat.com> On AArch64 we have to place a NOP at the start of every compiled method so that we can, if needs be, patch its verified entry point. This is because of an architectural restriction that only nops, traps, and branches may be patched while code is running. I am guessing that we do not need a NOP at the start of a compiled native wrapper because it will never be deoptimized. Is that right? Thanks, Andrew. From coleen.phillimore at oracle.com Fri Sep 11 12:38:09 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 11 Sep 2015 08:38:09 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F28FFF.1000803@oracle.com> References: <55ED9210.6030904@oracle.com> <55F1E85A.9090201@oracle.com> <55F28FFF.1000803@oracle.com> Message-ID: <55F2CB31.80505@oracle.com> Marcus, Thanks for the reply. Two short comments (see if you can find them). On 9/11/15 4:25 AM, Marcus Larsson wrote: > Hi, > > On 2015-09-10 22:30, Coleen Phillimore wrote: >> >> Hi Marcus, >> >> This code looks well written and documented in terms of appropriate >> comments in the header files. >> >> I have a few comments: >> >> 1. The NEW and strdup_check_oom() and other memory allocation >> functions use a combination of mtInternal and mtOther, neither of >> which is really going to be helpful for finding memory leaks in >> this. Can you add an mtLogging tag and use that? There is one >> strdup without an NMT tag (somewhere). > > Ah, nice catch. I'll fix this! > >> >> 2. The header files end with #endif but no // BLAH to match the >> #ifdef in the header guard. Can you add all of these? > > OK. > >> >> 3. Nice job adding #includes in alphabetical order. This is the >> only file that they are not: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logFileOutput.hpp.html >> > > Darn it, thought I had them all. Will fix. > >> >> 4. Some classes don't have storage allocation classes. It looks like >> they should be VALUE_OBJ_CLASS_SPEC. Eg. LogOutputList, >> LogDecorations, LogTagSet, LogTag (AllStatic), > > OK. > >> >> 5. In >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logTag.hpp.html >> there is unlikely to be a general 'rt' tag since runtime covers a >> broad set of different features in the vm. > > I'll remove it along with all the other currently unused tags. If > they're needed we can just add them with the change adding logging for > them. > >> >> Around line 51 in this file, can you explain that MaxTags is that you >> can only give so many tags in any given logging call, not that we can >> only define 5 tags total. Also, giving a description of how to >> specify more than one tag and what it means (union or intersection of >> that logging tag) would be good to have here too. > > I'll try to clarify this with some comments. > >> >> On line 58, can Count be "TagCount" or something a bit more descriptive? > > All enums introduced follow this pattern, and when they're used they > must be prefixed with the class they belong to, in this case > LogTag::Count. I think this looks good but I can change it if you want > to. If so, do you want me to make similar changes to LogLevel::Count > and LogDecorators::Count as well? I only saw Count in logTag.cpp so it looked odd. Qualified, it looks ok. Can you qualify it in logTag.cpp instead? > >> >> 6. In log.hpp can this really long line be broken up to something >> reasonable around 100 chars? >> >> template > T2 = LogTag::__NO_TAG, LogTagType T3 = LogTag::__NO_TAG, LogTagType >> T4 = LogTag::__NO_TAG, LogTagType GuardTag = LogTag::__NO_TAG> >> class Log { > > OK. > >> >> Thank you for the examples on how to use these. The template macro >> combinations are pretty insane. We used to not be allowed to use >> templates in Hotspot. > > They would be simpler if we had C++11's variadic templates. Now we're > sort of emulating this with up to five tags, which makes it a bit messy. > >> >> 7. In general, could you add a blank line between the header guards >> in the file and the #include statements? >> >> #ifndef SHARE_VM_LOGGING_LOG_HPP >> #define SHARE_VM_LOGGING_LOG_HPP >> #include "logging/logLevel.hpp" > > OK. > >> >> 8. In logTag.hpp there is MaxTags = 5 but the log.hpp file >> definitions have only 5 tags. It seems like MaxTags should be >> specified there instead or at the top level. > > I really wanted to have it in the Log class, but then to use it one > would have to specify the tag template argument > (Log::MaxTags rather than Log::MaxTags). Should I > rename it to LogMaxTags and make it a global constant (in log.hpp) > instead? How about just adding a comment then. I see your point. thanks, Coleen > >> >> 9. At the end of logTagSet.hpp, I can't guess what this line is: >> >> template > LogTagType T4> >> LogTagSet LogTagSetMapping::_tagset(T0, T1, T2, >> T3, T4); >> >> it looks like a constructor call. yes it is. Can you add a comment >> or something for people who's decoding skills with templates isn't as >> strong? And you have to define the static field _tagset in the .hpp >> file because it's a template. > > I'll add comments. > >> >> Can you break up the long template line for LogTagSetMapping also? > > OK. > >> >> This is all I have for now. This is a much larger change than I >> appreciated in my pre-review, but I still don't see anything bad at all. >> >> Thanks, >> Coleen > > Thank you for reviewing! > > Regards, > Marcus > >> >> On 9/7/15 9:33 AM, Marcus Larsson wrote: >>> Hi, >>> >>> Please review the following patch adding the unified logging >>> framework to hotspot. >>> >>> JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>> >>> See the JEP description for a general overview of the new feature. >>> Below are some notes on the implementation. >>> >>> The patch adds the new 'share/vm/logging' subdirectory containing >>> the unified logging framework. The main entry point is log.hpp, >>> which contains the necessary macros and definitions to use the >>> framework. >>> >>> Log tags are defined/listed in logTag.hpp, and are passed as >>> template arguments to the Log class. Every combination of tags used >>> in a log call has a corresponding LogTagSet instance, which keeps a >>> track of all the outputs it should write the log message to (and >>> their levels). Having tags as template arguments allows mapping >>> directly from a set of tags to the LogTagSet instance, which means >>> that the overhead for disabled logging should be low. Currently each >>> log message can be tagged with up to 5 tags, but this can be >>> increased if ever required (and with C++11's variadic templates the >>> limit can be removed completely). >>> >>> The LogConfiguration class keeps track of configured outputs >>> (stdout, stderr, and possible file outputs). Configuration is done >>> either by command line arguments (-Xlog) or by DCMD. Both methods >>> will in turn use the LogConfiguration class to perform the parsing & >>> configuration. This configuration includes iterating over all >>> LogTagSet instances and updating them accordingly. The >>> LogTagLevelExpression class is used to represent the selection of >>> tags and levels for a given configuration request (the >>> "what"-expression). >>> >>> The LogDecorators class contains a selection of decorators. >>> Instances of this class is kept in LogTagSet to track what >>> decorators to use (this is the union of all decorators used by its >>> outputs). Each log call will create a LogDecorations instance (note: >>> different classes), which will contain the actual decoration strings >>> for the log message. These decorations are used for each output the >>> tagset is set to log on, and are then discarded. >>> >>> The LogPrefix class allows messages of specific sets of tags to be >>> prefixed. The prefix should supply a printf-style format with >>> argument. (This allows GC logging to prefix messages of certain >>> tagsets with GCId.) Prefixes are implemented using template >>> specializations based on the specified tags, with the >>> general/unspecialized case giving an empty prefix. >>> >>> The LogOutput class defines the interface for all types of log >>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>> outputs. LogFileOutput builds on this and adds the file management >>> and log rotation support. >>> >>> A simple jtreg test is included in this patch. Additional tests will >>> be added at a later stage. >>> >>> Thanks, >>> Marcus >> > From marcus.larsson at oracle.com Fri Sep 11 13:03:50 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 11 Sep 2015 15:03:50 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F2CB31.80505@oracle.com> References: <55ED9210.6030904@oracle.com> <55F1E85A.9090201@oracle.com> <55F28FFF.1000803@oracle.com> <55F2CB31.80505@oracle.com> Message-ID: <55F2D136.5080204@oracle.com> On 2015-09-11 14:38, Coleen Phillimore wrote: > > Marcus, Thanks for the reply. Two short comments (see if you can > find them). > > On 9/11/15 4:25 AM, Marcus Larsson wrote: >> Hi, >> >> On 2015-09-10 22:30, Coleen Phillimore wrote: >>> >>> Hi Marcus, >>> >>> This code looks well written and documented in terms of appropriate >>> comments in the header files. >>> >>> I have a few comments: >>> >>> 1. The NEW and strdup_check_oom() and other memory allocation >>> functions use a combination of mtInternal and mtOther, neither of >>> which is really going to be helpful for finding memory leaks in >>> this. Can you add an mtLogging tag and use that? There is one >>> strdup without an NMT tag (somewhere). >> >> Ah, nice catch. I'll fix this! >> >>> >>> 2. The header files end with #endif but no // BLAH to match the >>> #ifdef in the header guard. Can you add all of these? >> >> OK. >> >>> >>> 3. Nice job adding #includes in alphabetical order. This is the >>> only file that they are not: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logFileOutput.hpp.html >>> >> >> Darn it, thought I had them all. Will fix. >> >>> >>> 4. Some classes don't have storage allocation classes. It looks >>> like they should be VALUE_OBJ_CLASS_SPEC. Eg. LogOutputList, >>> LogDecorations, LogTagSet, LogTag (AllStatic), >> >> OK. >> >>> >>> 5. In >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/src/share/vm/logging/logTag.hpp.html >>> there is unlikely to be a general 'rt' tag since runtime covers a >>> broad set of different features in the vm. >> >> I'll remove it along with all the other currently unused tags. If >> they're needed we can just add them with the change adding logging >> for them. >> >>> >>> Around line 51 in this file, can you explain that MaxTags is that >>> you can only give so many tags in any given logging call, not that >>> we can only define 5 tags total. Also, giving a description of how >>> to specify more than one tag and what it means (union or >>> intersection of that logging tag) would be good to have here too. >> >> I'll try to clarify this with some comments. >> >>> >>> On line 58, can Count be "TagCount" or something a bit more >>> descriptive? >> >> All enums introduced follow this pattern, and when they're used they >> must be prefixed with the class they belong to, in this case >> LogTag::Count. I think this looks good but I can change it if you >> want to. If so, do you want me to make similar changes to >> LogLevel::Count and LogDecorators::Count as well? > > I only saw Count in logTag.cpp so it looked odd. Qualified, it looks > ok. Can you qualify it in logTag.cpp instead? Sounds good! >> >>> >>> 6. In log.hpp can this really long line be broken up to something >>> reasonable around 100 chars? >>> >>> template >> LogTagType T2 = LogTag::__NO_TAG, LogTagType T3 = LogTag::__NO_TAG, >>> LogTagType T4 = LogTag::__NO_TAG, LogTagType GuardTag = >>> LogTag::__NO_TAG> >>> class Log { >> >> OK. >> >>> >>> Thank you for the examples on how to use these. The template macro >>> combinations are pretty insane. We used to not be allowed to use >>> templates in Hotspot. >> >> They would be simpler if we had C++11's variadic templates. Now we're >> sort of emulating this with up to five tags, which makes it a bit messy. >> >>> >>> 7. In general, could you add a blank line between the header guards >>> in the file and the #include statements? >>> >>> #ifndef SHARE_VM_LOGGING_LOG_HPP >>> #define SHARE_VM_LOGGING_LOG_HPP >>> #include "logging/logLevel.hpp" >> >> OK. >> >>> >>> 8. In logTag.hpp there is MaxTags = 5 but the log.hpp file >>> definitions have only 5 tags. It seems like MaxTags should be >>> specified there instead or at the top level. >> >> I really wanted to have it in the Log class, but then to use it one >> would have to specify the tag template argument >> (Log::MaxTags rather than Log::MaxTags). Should I >> rename it to LogMaxTags and make it a global constant (in log.hpp) >> instead? > > How about just adding a comment then. I see your point. Alright, I'll do that. Thanks again, Marcus > > thanks, > Coleen >> >>> >>> 9. At the end of logTagSet.hpp, I can't guess what this line is: >>> >>> template >> T3, LogTagType T4> >>> LogTagSet LogTagSetMapping::_tagset(T0, T1, T2, >>> T3, T4); >>> >>> it looks like a constructor call. yes it is. Can you add a comment >>> or something for people who's decoding skills with templates isn't >>> as strong? And you have to define the static field _tagset in the >>> .hpp file because it's a template. >> >> I'll add comments. >> >>> >>> Can you break up the long template line for LogTagSetMapping also? >> >> OK. >> >>> >>> This is all I have for now. This is a much larger change than I >>> appreciated in my pre-review, but I still don't see anything bad at >>> all. >>> >>> Thanks, >>> Coleen >> >> Thank you for reviewing! >> >> Regards, >> Marcus >> >>> >>> On 9/7/15 9:33 AM, Marcus Larsson wrote: >>>> Hi, >>>> >>>> Please review the following patch adding the unified logging >>>> framework to hotspot. >>>> >>>> JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>>> >>>> See the JEP description for a general overview of the new feature. >>>> Below are some notes on the implementation. >>>> >>>> The patch adds the new 'share/vm/logging' subdirectory containing >>>> the unified logging framework. The main entry point is log.hpp, >>>> which contains the necessary macros and definitions to use the >>>> framework. >>>> >>>> Log tags are defined/listed in logTag.hpp, and are passed as >>>> template arguments to the Log class. Every combination of tags used >>>> in a log call has a corresponding LogTagSet instance, which keeps a >>>> track of all the outputs it should write the log message to (and >>>> their levels). Having tags as template arguments allows mapping >>>> directly from a set of tags to the LogTagSet instance, which means >>>> that the overhead for disabled logging should be low. Currently >>>> each log message can be tagged with up to 5 tags, but this can be >>>> increased if ever required (and with C++11's variadic templates the >>>> limit can be removed completely). >>>> >>>> The LogConfiguration class keeps track of configured outputs >>>> (stdout, stderr, and possible file outputs). Configuration is done >>>> either by command line arguments (-Xlog) or by DCMD. Both methods >>>> will in turn use the LogConfiguration class to perform the parsing >>>> & configuration. This configuration includes iterating over all >>>> LogTagSet instances and updating them accordingly. The >>>> LogTagLevelExpression class is used to represent the selection of >>>> tags and levels for a given configuration request (the >>>> "what"-expression). >>>> >>>> The LogDecorators class contains a selection of decorators. >>>> Instances of this class is kept in LogTagSet to track what >>>> decorators to use (this is the union of all decorators used by its >>>> outputs). Each log call will create a LogDecorations instance >>>> (note: different classes), which will contain the actual decoration >>>> strings for the log message. These decorations are used for each >>>> output the tagset is set to log on, and are then discarded. >>>> >>>> The LogPrefix class allows messages of specific sets of tags to be >>>> prefixed. The prefix should supply a printf-style format with >>>> argument. (This allows GC logging to prefix messages of certain >>>> tagsets with GCId.) Prefixes are implemented using template >>>> specializations based on the specified tags, with the >>>> general/unspecialized case giving an empty prefix. >>>> >>>> The LogOutput class defines the interface for all types of log >>>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>>> outputs. LogFileOutput builds on this and adds the file management >>>> and log rotation support. >>>> >>>> A simple jtreg test is included in this patch. Additional tests >>>> will be added at a later stage. >>>> >>>> Thanks, >>>> Marcus >>> >> > From marcus.larsson at oracle.com Fri Sep 11 13:26:45 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 11 Sep 2015 15:26:45 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F1C473.7070405@oracle.com> References: <55ED9210.6030904@oracle.com> <55EF4492.6000904@oracle.com> <55EF4AC7.40809@oracle.com> <55EFE0B2.2060100@oracle.com> <55F1C473.7070405@oracle.com> Message-ID: <55F2D695.2070001@oracle.com> Hi, Thanks for your input. On 2015-09-10 19:57, Ioi Lam wrote: > I have two issues: > > [1] There's a limit of 512 characters: > > void Log::vwrite(const char* fmt, > va_list args) { > char buf[LogBufferSize]; > static const size_t LogBufferSize = 512; > > I would need to log the names of loaded classes, which could be much > longer. Would it be possible to add a new function like > > LogHandle(classload) log; > log.puts("a very very very long string .........."); This seems like a reasonable idea to me if you need such functionality. > > [2] Although the Log::write function is declared with > ATTRIBUTE_PRINTF, it doesn't seem to do the checking. > > static void write(const char* fmt, ...) ATTRIBUTE_PRINTF(1, 2); > > I added these > > void LogConfiguration::post_initialize() { > assert(LogConfiguration_lock != NULL, "Lock must be initialized > before post-initialization"); > LogDiagnosticCommand::registerCommand(); > LogHandle(logging) log; > log.info("Log configuration fully initialized."); > + log_info(logging)("hello %s"); > + tty->print_cr("hello %s"); > > I get a warning on the tty->print_cr line but not on the > log_info(logging) line. > > **NOTICE** Dtrace support disabled: This JDK does not support SDT probes > /home/iklam/jdk/ul/hotspot/src/share/vm/logging/logConfiguration.cpp: > In static member function `static void > LogConfiguration::post_initialize()': > /home/iklam/jdk/ul/hotspot/src/share/vm/logging/logConfiguration.cpp:52:27: > error: format `%s' expects a matching `char*' argument [-Werror=format=] > tty->print_cr("hello %s"); > ^ > cc1plus: all warnings being treated as errors > make[4]: *** [logConfiguration.o] Error 1 That's weird. It seems the printf-attribute is lost when splitting the function definition from the declaration. When I merge them the warning appears as it should. Will update the patch with this fix! Thanks, Marcus > > My gcc is > > $ gcc -v > Using built-in specs. > COLLECT_GCC=gcc > COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper > Target: x86_64-linux-gnu > Configured with: ../src/configure -v --with-pkgversion='Ubuntu > 4.8.2-19ubuntu1' > --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs > --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr > --program-suffix=-4.8 --enable-shared --enable-linker-build-id > --libexecdir=/usr/lib --without-included-gettext > --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 > --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu > --enable-libstdcxx-debug --enable-libstdcxx-time=yes > --enable-gnu-unique-object --disable-libmudflap --enable-plugin > --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk > --enable-gtk-cairo > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre > --enable-java-home > --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 > --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 > --with-arch-directory=amd64 > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc > --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 > --with-multilib-list=m32,m64,mx32 --with-tune=generic > --enable-checking=release --build=x86_64-linux-gnu > --host=x86_64-linux-gnu --target=x86_64-linux-gnu > Thread model: posix > gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) > > Thanks > - Ioi > > On 9/9/15 12:33 AM, Marcus Larsson wrote: >> Hi, >> >> On 2015-09-08 22:53, Ioi Lam wrote: >>> Also, is it possible to combine the output of two tags into a single >>> file, like this? >>> >>> -Xlog:gc:file=out.txt -Xlog:comp:file=out.txt >>> >>> this way, I can have both GC and Compiler logs that's in an >>> (approximate) temporal order. The alternatives are: >>> >>> - write both to stdout -- but I need to filter out junk that I don't >>> want >>> - write to different files -- but I lose the relative ordering >>> between the two sets of logs >>> >>> From the current patch, it seems like if I assign the same file name >>> to two tags, fopen() will be called twice and the result will be >>> platform dependent. >>> >>> - Ioi >> >> This is indeed possible! The syntax for configuring this would be >> -Xlog:gc,comp:file=out.txt. Configuring logging for the same output >> multiple times only overrides/replaces the previous configuration. >> The second fopen will not happen, because the log configuration will >> notice the output already exists and simply reconfigure the existing >> output. >> >> Thanks, >> Marcus >> >>> >>> On 9/8/15 1:26 PM, Ioi Lam wrote: >>>> Hi Marcus, >>>> >>>> I am eagerly waiting for this to implement a new logging feature in >>>> the VM. How much testing has been done on this patch? I can see >>>> only two locations where any logs are bring generated: >>>> >>>> + log_error(logging)("%s", errbuf); >>>> + log_error(logging)("Could not open log file '%s' (%s).\n", >>>> _file_name, strerror(errno)); >>>> >>>> Do you have a follow-on patch that's already in work to test the >>>> various functionalists of the Unified JVM Logging framework? I am >>>> asking so that I can know how much surprise to expect when I start >>>> using it. >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> On 9/7/15 6:33 AM, Marcus Larsson wrote: >>>>> Hi, >>>>> >>>>> Please review the following patch adding the unified logging >>>>> framework to hotspot. >>>>> >>>>> JEP: >>>>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>>>> >>>>> See the JEP description for a general overview of the new feature. >>>>> Below are some notes on the implementation. >>>>> >>>>> The patch adds the new 'share/vm/logging' subdirectory containing >>>>> the unified logging framework. The main entry point is log.hpp, >>>>> which contains the necessary macros and definitions to use the >>>>> framework. >>>>> >>>>> Log tags are defined/listed in logTag.hpp, and are passed as >>>>> template arguments to the Log class. Every combination of tags >>>>> used in a log call has a corresponding LogTagSet instance, which >>>>> keeps a track of all the outputs it should write the log message >>>>> to (and their levels). Having tags as template arguments allows >>>>> mapping directly from a set of tags to the LogTagSet instance, >>>>> which means that the overhead for disabled logging should be low. >>>>> Currently each log message can be tagged with up to 5 tags, but >>>>> this can be increased if ever required (and with C++11's variadic >>>>> templates the limit can be removed completely). >>>>> >>>>> The LogConfiguration class keeps track of configured outputs >>>>> (stdout, stderr, and possible file outputs). Configuration is done >>>>> either by command line arguments (-Xlog) or by DCMD. Both methods >>>>> will in turn use the LogConfiguration class to perform the parsing >>>>> & configuration. This configuration includes iterating over all >>>>> LogTagSet instances and updating them accordingly. The >>>>> LogTagLevelExpression class is used to represent the selection of >>>>> tags and levels for a given configuration request (the >>>>> "what"-expression). >>>>> >>>>> The LogDecorators class contains a selection of decorators. >>>>> Instances of this class is kept in LogTagSet to track what >>>>> decorators to use (this is the union of all decorators used by its >>>>> outputs). Each log call will create a LogDecorations instance >>>>> (note: different classes), which will contain the actual >>>>> decoration strings for the log message. These decorations are used >>>>> for each output the tagset is set to log on, and are then discarded. >>>>> >>>>> The LogPrefix class allows messages of specific sets of tags to be >>>>> prefixed. The prefix should supply a printf-style format with >>>>> argument. (This allows GC logging to prefix messages of certain >>>>> tagsets with GCId.) Prefixes are implemented using template >>>>> specializations based on the specified tags, with the >>>>> general/unspecialized case giving an empty prefix. >>>>> >>>>> The LogOutput class defines the interface for all types of log >>>>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>>>> outputs. LogFileOutput builds on this and adds the file management >>>>> and log rotation support. >>>>> >>>>> A simple jtreg test is included in this patch. Additional tests >>>>> will be added at a later stage. >>>>> >>>>> Thanks, >>>>> Marcus >>>> >>> >> > From marcus.larsson at oracle.com Fri Sep 11 13:40:44 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Fri, 11 Sep 2015 15:40:44 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F1A767.8040002@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> Message-ID: <55F2D9DC.5010401@oracle.com> On 2015-09-10 17:53, Rachel Protacio wrote: > On 9/10/2015 5:04 AM, Marcus Larsson wrote: >> Hi, >> >> On 2015-09-09 17:37, Rachel Protacio wrote: >>> Hi! >>> >>> For runtime logging, there are many locations where it is necessary >>> to have a block led by a conditional checking whether the >>> debug/info/trace logging is enabled with the appropriate tag. Would >>> it be possible to add macros to logging/log.hpp such as >>> >>> #define log_debug_is_enabled(...) log_is_enabled(Debug, __VA_ARGS__) >>> #define log_trace_is_enabled(...) log_is_enabled(Trace, __VA_ARGS__) >>> >>> etc.? If they are added to your framework patch, that will help Max >>> and me as we write our logging code. >> >> It could be done, but I think we should go either with the >> log_is_enabled(level, tags) macro, or individual macros for each log >> level like you suggest, and not both. My intention has been to >> minimize the number of macros introduced with this patch to reduce >> the risk for namespace issues. I can see that having the separate >> macros helps readability a little, so if we're okay with the >> increased number of macros this seems like a good idea to me. >> >> If we agree to go with separate macros I suggest we shorten the names >> and use macros like log_is_debug(...), log_is_trace(...). > Oh, I see what you're saying. On second thought, I'm fine with using > log_is_enabled(Level, tags) - I think it is equally clear. However, I > would like to bring up one other point. In the block-code situations, > it is currently necessary to write code that looks like > > if (log_is_enabled(Debug, )) { > > log_debug()("Logging statement to print"); > } > > Which redundantly executes log_is_enabled() in the third line. > Honestly, I'm okay with that. The issue is that in cases where it is > necessary to pass the logging output stream to another function, the > code ends up looking like > > if (log_is_enabled(Debug, )) { > > ResourceMark rm; > print_on(LogHandle()::debug_stream()); > } > > or even > > LogHandle() log; > if (log.is_debug()) { > > ResourceMark rm; > print_on(log.debug_stream()); > } > > Is that correct? Neither seems ideal since they add extra clutter to > the code and all these three examples are somewhat inconsistent with > each other. Is there a cleaner way of doing this that is already in > the code or that could be added? Yes that's correct. I'm not sure how this could be simplified further, I think the clutter might be necessary. The last example is typically how I imagined more involved logging would work. The macros are a convenience for quick and simple logging. If you have an idea of how to improve the this I'm happy to consider it. Thanks, Marcus > >> >>> >>> Additionally, the print format for the logging was not immediately >>> intuitive. For example, as I'm working on the safepoint logging, I >>> might have an output line like >>> >>> [0.655s][debug ][safepoint] Safepoint synchronization initiated. >>> (20) >>> >>> Are the spaces after debug intended to line up the tags with each >>> other (since the longest one is warning, it seems to fit)? If so, is >>> that meant for readability? And what happens if the number of >>> seconds goes into double digits - won't the alignment be off? >>> >> >> Yes, decorators are padded to avoid jagged logs and help readability. >> Since the levels are known beforehand, a fixed padding is used for >> that decorator. For other decorators such as tags or timestamps, the >> padding will grow to the size of the longest (so far) seen >> decoration. This means the decorator prefix length will either stay >> the same or increase, but never decrease. After a while it should >> stabilize around some fitting length and not grow significantly. >> >> For example: >> [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) >> [0.656s][debug ][safepoint, some_other_tag] Safepoint >> synchronization initiated. (20) >> [0.657s][debug ][safepoint, ] Safepoint >> synchronization initiated. (20) >> [10.657s][debug ][safepoint, ] Safepoint >> synchronization initiated. (20) \ > Ok, thanks for clarifying. > Rachel >> >> >> Thanks, >> Marcus >> >>> >>> Thanks, >>> Rachel >>> >>> On 9/7/2015 9:33 AM, Marcus Larsson wrote: >>>> Hi, >>>> >>>> Please review the following patch adding the unified logging >>>> framework to hotspot. >>>> >>>> JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>>> >>>> See the JEP description for a general overview of the new feature. >>>> Below are some notes on the implementation. >>>> >>>> The patch adds the new 'share/vm/logging' subdirectory containing >>>> the unified logging framework. The main entry point is log.hpp, >>>> which contains the necessary macros and definitions to use the >>>> framework. >>>> >>>> Log tags are defined/listed in logTag.hpp, and are passed as >>>> template arguments to the Log class. Every combination of tags used >>>> in a log call has a corresponding LogTagSet instance, which keeps a >>>> track of all the outputs it should write the log message to (and >>>> their levels). Having tags as template arguments allows mapping >>>> directly from a set of tags to the LogTagSet instance, which means >>>> that the overhead for disabled logging should be low. Currently >>>> each log message can be tagged with up to 5 tags, but this can be >>>> increased if ever required (and with C++11's variadic templates the >>>> limit can be removed completely). >>>> >>>> The LogConfiguration class keeps track of configured outputs >>>> (stdout, stderr, and possible file outputs). Configuration is done >>>> either by command line arguments (-Xlog) or by DCMD. Both methods >>>> will in turn use the LogConfiguration class to perform the parsing >>>> & configuration. This configuration includes iterating over all >>>> LogTagSet instances and updating them accordingly. The >>>> LogTagLevelExpression class is used to represent the selection of >>>> tags and levels for a given configuration request (the >>>> "what"-expression). >>>> >>>> The LogDecorators class contains a selection of decorators. >>>> Instances of this class is kept in LogTagSet to track what >>>> decorators to use (this is the union of all decorators used by its >>>> outputs). Each log call will create a LogDecorations instance >>>> (note: different classes), which will contain the actual decoration >>>> strings for the log message. These decorations are used for each >>>> output the tagset is set to log on, and are then discarded. >>>> >>>> The LogPrefix class allows messages of specific sets of tags to be >>>> prefixed. The prefix should supply a printf-style format with >>>> argument. (This allows GC logging to prefix messages of certain >>>> tagsets with GCId.) Prefixes are implemented using template >>>> specializations based on the specified tags, with the >>>> general/unspecialized case giving an empty prefix. >>>> >>>> The LogOutput class defines the interface for all types of log >>>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>>> outputs. LogFileOutput builds on this and adds the file management >>>> and log rotation support. >>>> >>>> A simple jtreg test is included in this patch. Additional tests >>>> will be added at a later stage. >>>> >>>> Thanks, >>>> Marcus >>> >> > From coleen.phillimore at oracle.com Fri Sep 11 13:51:20 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 11 Sep 2015 09:51:20 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F2D9DC.5010401@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> Message-ID: <55F2DC58.7070004@oracle.com> I'm not a big fan of having these blanks in the logging lines. I don't think making the line length longer is going to be helpful and what people are looking for is the message at the end, not which tag and level they used. I think they look kind of strange. Coleen On 9/11/15 9:40 AM, Marcus Larsson wrote: > Yes, decorators are padded to avoid jagged logs and help readability. > Since the levels are known beforehand, a fixed padding is used for > that decorator. For other decorators such as tags or timestamps, the > padding will grow to the size of the longest (so far) seen decoration. > This means the decorator prefix length will either stay the same or > increase, but never decrease. After a while it should stabilize around > some fitting length and not grow significantly. > > For example: > [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) > [0.656s][debug ][safepoint, some_other_tag] Safepoint synchronization > initiated. (20) > [0.657s][debug ][safepoint, ] Safepoint synchronization > initiated. (20) > [10.657s][debug ][safepoint, ] Safepoint > synchronization initiated. (20) \ From max.ockner at oracle.com Fri Sep 11 14:11:25 2015 From: max.ockner at oracle.com (Max Ockner) Date: Fri, 11 Sep 2015 10:11:25 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F2DC58.7070004@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> Message-ID: <55F2E10D.8040800@oracle.com> TL;DR at the bottom. I have encountered some of the same issues with "levels" as many of the other responders. Approximately half of the runtime team's logging plans (all of the command line options) are made significantly more complicated by the way levels are set up. The other half of our logging requirements actually fit very well with levels. First I'll mention the pain points of converting to Unified Logging. UL wants me to organize my logging messages by subsystem (tag), which I completely agree with. However, each subsystem is only allowed 5 nested logging commands even though it is most common that a single subsystem has several disjoint types of logging going on. Here is an example in which it makes more sense to split a subsystem into 5 different tags than it does to use levels: We have a ClassLoading system with quite a lot of stuff that can be logged. ClassLoading, ClassUnloading, ClassLoaderData, LoaderConstraints, ClassLoadingPreorder, and a couple more. If we want all of these logging options to be accessible through the ClassLoading tag (which we absolutely *do* want), then we will need to assign one logging feature per level. So which one is Trace level? That is, which feature do we only want to see if everything else is being printed as well? What if we want to use print LoaderConstraints on its own? With the given design, you can't separate different types of logging in this way without having entirely different tags. Because of this, the runtime team plans to use a large number of tags (more than 75 different tags) despite having nowhere close to that many subsystems. There are also many instances of logging options being called with "Verbose" or "WizardMode", which is analogous to setting level=trace. An undesirable side effect of this is if we want logging option X to have a default output and a more verbose output, then option X in its plain form is forced to have a level lower than trace. We have had to compromise some of the logging functionality so far by combining Verbose and WizardMode so that there is at most one level of super verbose info above any command line option. Of course, it would be nice to condense a lot of these options into single logging statements for their respective subsystems, but this would majorly affect the existing Java code for a lot of our customers as well as a lot of developers. On a more positive note, our error and warning level logging messages fit much better into the UL framework since the subsystem and severity of the error naturally imply which tag and level should be used. TL;DR: -Levels help with structuring a lot of runtime's logging messages. -Levels create undesirable constraints on some (half) of runtime's logging messages. -If users and developers weren't already using our command line options in such delicate ways, it would be much easier for runtime to bite the bullet and completely adhere to the tag=subsystem convention. - Unified Logging is good. I hope this helps. Max Ockner From vitalyd at gmail.com Fri Sep 11 15:07:28 2015 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 11 Sep 2015 11:07:28 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F2E10D.8040800@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F2E10D.8040800@oracle.com> Message-ID: So perhaps ClassLoading is too coarse a tag then? If you did have a tag per class loading, unloading, constraints, etc are there also verbosity levels within each? sent from my phone On Sep 11, 2015 10:12 AM, "Max Ockner" wrote: > > TL;DR at the bottom. > > I have encountered some of the same issues with "levels" as many of the > other responders. Approximately half of the runtime team's logging plans > (all of the command line options) are made significantly more complicated > by the way levels are set up. The other half of our logging requirements > actually fit very well with levels. > > First I'll mention the pain points of converting to Unified Logging. UL > wants me to organize my logging messages by subsystem (tag), which I > completely agree with. However, each subsystem is only allowed 5 nested > logging commands even though it is most common that a single subsystem has > several disjoint types of logging going on. Here is an example in which it > makes more sense to split a subsystem into 5 different tags than it does to > use levels: > > We have a ClassLoading system with quite a lot of stuff that can be > logged. ClassLoading, ClassUnloading, ClassLoaderData, LoaderConstraints, > ClassLoadingPreorder, and a couple more. If we want all of these logging > options to be accessible through the ClassLoading tag (which we absolutely > *do* want), then we will need to assign one logging feature per level. So > which one is Trace level? That is, which feature do we only want to see if > everything else is being printed as well? What if we want to use print > LoaderConstraints on its own? With the given design, you can't separate > different types of logging in this way without having entirely different > tags. > > Because of this, the runtime team plans to use a large number of tags > (more than 75 different tags) despite having nowhere close to that many > subsystems. There are also many instances of logging options being called > with "Verbose" or "WizardMode", which is analogous to setting level=trace. > An undesirable side effect of this is if we want logging option X to have a > default output and a more verbose output, then option X in its plain form > is forced to have a level lower than trace. We have had to compromise some > of the logging functionality so far by combining Verbose and WizardMode so > that there is at most one level of super verbose info above any command > line option. Of course, it would be nice to condense a lot of these > options into single logging statements for their respective subsystems, but > this would majorly affect the existing Java code for a lot of our customers > as well as a lot of developers. > > On a more positive note, our error and warning level logging messages fit > much better into the UL framework since the subsystem and severity of the > error naturally imply which tag and level should be used. > > TL;DR: > -Levels help with structuring a lot of runtime's logging messages. > -Levels create undesirable constraints on some (half) of runtime's logging > messages. > -If users and developers weren't already using our command line options in > such delicate ways, it would be much easier for runtime to bite the bullet > and completely adhere to the tag=subsystem convention. > - Unified Logging is good. > > I hope this helps. > Max Ockner > From max.ockner at oracle.com Fri Sep 11 15:30:24 2015 From: max.ockner at oracle.com (Max Ockner) Date: Fri, 11 Sep 2015 11:30:24 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F2E10D.8040800@oracle.com> Message-ID: <55F2F390.7090604@oracle.com> There would be at most one additional level of verbosity, which corresponds to calling the logging flag with "Verbose" or "WizardMode". In this case, I'm not sure if there are any additional levels of verbosity. On 9/11/2015 11:07 AM, Vitaly Davidovich wrote: > > So perhaps ClassLoading is too coarse a tag then? If you did have a > tag per class loading, unloading, constraints, etc are there also > verbosity levels within each? > > sent from my phone > > On Sep 11, 2015 10:12 AM, "Max Ockner" > wrote: > > > TL;DR at the bottom. > > I have encountered some of the same issues with "levels" as many > of the other responders. Approximately half of the runtime team's > logging plans (all of the command line options) are made > significantly more complicated by the way levels are set up. The > other half of our logging requirements actually fit very well with > levels. > > First I'll mention the pain points of converting to Unified > Logging. UL wants me to organize my logging messages by subsystem > (tag), which I completely agree with. However, each subsystem is > only allowed 5 nested logging commands even though it is most > common that a single subsystem has several disjoint types of > logging going on. Here is an example in which it makes more sense > to split a subsystem into 5 different tags than it does to use levels: > > We have a ClassLoading system with quite a lot of stuff that can > be logged. ClassLoading, ClassUnloading, ClassLoaderData, > LoaderConstraints, ClassLoadingPreorder, and a couple more. If we > want all of these logging options to be accessible through the > ClassLoading tag (which we absolutely *do* want), then we will > need to assign one logging feature per level. So which one is > Trace level? That is, which feature do we only want to see if > everything else is being printed as well? What if we want to use > print LoaderConstraints on its own? With the given design, you > can't separate different types of logging in this way without > having entirely different tags. > > Because of this, the runtime team plans to use a large number of > tags (more than 75 different tags) despite having nowhere close to > that many subsystems. There are also many instances of logging > options being called with "Verbose" or "WizardMode", which is > analogous to setting level=trace. An undesirable side effect of > this is if we want logging option X to have a default output and a > more verbose output, then option X in its plain form is forced to > have a level lower than trace. We have had to compromise some of > the logging functionality so far by combining Verbose and > WizardMode so that there is at most one level of super verbose > info above any command line option. Of course, it would be nice > to condense a lot of these options into single logging statements > for their respective subsystems, but this would majorly affect the > existing Java code for a lot of our customers as well as a lot of > developers. > > On a more positive note, our error and warning level logging > messages fit much better into the UL framework since the subsystem > and severity of the error naturally imply which tag and level > should be used. > > TL;DR: > -Levels help with structuring a lot of runtime's logging messages. > -Levels create undesirable constraints on some (half) of runtime's > logging messages. > -If users and developers weren't already using our command line > options in such delicate ways, it would be much easier for runtime > to bite the bullet and completely adhere to the tag=subsystem > convention. > - Unified Logging is good. > > I hope this helps. > Max Ockner > From gerard.ziemski at oracle.com Fri Sep 11 18:13:12 2015 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Fri, 11 Sep 2015 13:13:12 -0500 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F219CE.3010006@oracle.com> References: <55F219CE.3010006@oracle.com> Message-ID: <55F319B8.1000803@oracle.com> hi Sangheon, #1 test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java Please change the comment to: + /* + * Exclude below options as their maximum value would consume too much memory + * and would affect other tests that run in parallel. + */ #2 What tests did you run? Did you run test/runtime/CommandLine/OptionsValidation on all platforms (including embedded)? cheers On 09/10/2015 07:01 PM, sangheon.kim wrote: > Hi all, > > Please review this patch for command-line validation for GC flags. > This REDO patch is adding ranges and implementing constraint functions for GC flags. > > Original CR of JDK-8078555 was backout as it made a test failure from 'TestOptionsWithRanges.java'. > And also there were some discussion of OOM handling. > > Most parts are same as JDK-8078555 except below: > 1. Changed 'range' for some flags. > 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags make this test unstable as it tries to allocate > huge amount of memory. > > And below are the suggestion note for JDK-8078555: > 1. Exponential notation for 'double' type variable parse: Previously there were some discussion for maximum value for > double type flags from code review of JDK-8059557 and JDK-8112746. And Kim and I decided not to add upper limit unless > there are problems with DBL_MAX. And as 255 is the maximum length that can be passed via command-line, we introduced > exponential notation to avoid this limit. ( arguments.cpp ) > 2. These GC flags ranges are not ideal ranges but ranges which don't make problem with current source code. > If one flag makes some problem but hard to find good range, I added some ranges. > 3. There are some constraint functions to avoid overflow. > 4. Test applications are changed: as some of them assumed to be ParallelGC or to check it's output messages. > 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8134995 > > Webrev: > http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ > http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 > > Testing: > JPRT, UTE(vm.quick-pcl) and test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. > > Thanks, > Sangheon > > From kirk.pepperdine at gmail.com Fri Sep 11 19:19:11 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Fri, 11 Sep 2015 21:19:11 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F2F390.7090604@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F2E10D.8040800@oracle.com> <55F2F390.7090604@oracle.com> Message-ID: > On Sep 11, 2015, at 5:30 PM, Max Ockner wrote: > > There would be at most one additional level of verbosity, which corresponds to calling the logging flag with "Verbose" or "WizardMode". In this case, I'm not sure if there are any additional levels of verbosity. This is an excellent example of some of the things I?ve been so clumsily trying to say. This use case could easily be managed with nested tags. You can achieve levels with tags. You can create nomenclatures that make sense for your domain. it simplifies the spec and the implementation. There are just too many reasons to drop levels and add proper support for nested tags. There are also many examples of where this works very well. SNMP for example. The spec is very simple as are the supporting implementations. Unfortunately Java logging systems have saddled us with a naive implementations that we simply can?t seem to shake. Kind regards, Kirk From sangheon.kim at oracle.com Fri Sep 11 19:24:49 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Fri, 11 Sep 2015 12:24:49 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F319B8.1000803@oracle.com> References: <55F219CE.3010006@oracle.com> <55F319B8.1000803@oracle.com> Message-ID: <55F32A81.8000307@oracle.com> Hi Gerard, Thank you for looking at this. On 09/11/2015 11:13 AM, gerard ziemski wrote: > hi Sangheon, > > #1 test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java > > Please change the comment to: > > + /* > + * Exclude below options as their maximum value would consume > too much memory > + * and would affect other tests that run in parallel. > + */ Okay, I will fix as you suggested. > > > #2 What tests did you run? Did you run > test/runtime/CommandLine/OptionsValidation on all platforms (including > embedded)? No. I ran tests under test/runtime/CommandLine/OptionsValidation (especially TestOptionsWithRanges.java) for all platforms except embedded. Let me back after testing on embedded. Thanks, Sangheon > > > cheers > > > On 09/10/2015 07:01 PM, sangheon.kim wrote: >> Hi all, >> >> Please review this patch for command-line validation for GC flags. >> This REDO patch is adding ranges and implementing constraint >> functions for GC flags. >> >> Original CR of JDK-8078555 was backout as it made a test failure from >> 'TestOptionsWithRanges.java'. >> And also there were some discussion of OOM handling. >> >> Most parts are same as JDK-8078555 except below: >> 1. Changed 'range' for some flags. >> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags >> make this test unstable as it tries to allocate >> huge amount of memory. >> >> And below are the suggestion note for JDK-8078555: >> 1. Exponential notation for 'double' type variable parse: Previously >> there were some discussion for maximum value for >> double type flags from code review of JDK-8059557 and JDK-8112746. >> And Kim and I decided not to add upper limit unless >> there are problems with DBL_MAX. And as 255 is the maximum length >> that can be passed via command-line, we introduced >> exponential notation to avoid this limit. ( arguments.cpp ) >> 2. These GC flags ranges are not ideal ranges but ranges which don't >> make problem with current source code. >> If one flag makes some problem but hard to find good range, I >> added some ranges. >> 3. There are some constraint functions to avoid overflow. >> 4. Test applications are changed: as some of them assumed to be >> ParallelGC or to check it's output messages. >> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8134995 >> >> Webrev: >> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >> >> Testing: >> JPRT, UTE(vm.quick-pcl) and >> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >> >> Thanks, >> Sangheon >> >> From sangheon.kim at oracle.com Sat Sep 12 08:38:28 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Sat, 12 Sep 2015 01:38:28 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F32A81.8000307@oracle.com> References: <55F219CE.3010006@oracle.com> <55F319B8.1000803@oracle.com> <55F32A81.8000307@oracle.com> Message-ID: <55F3E484.70107@oracle.com> Hi Gerard, On 09/11/2015 12:24 PM, sangheon.kim wrote: > Hi Gerard, > > Thank you for looking at this. > > On 09/11/2015 11:13 AM, gerard ziemski wrote: >> hi Sangheon, >> >> #1 test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >> >> Please change the comment to: >> >> + /* >> + * Exclude below options as their maximum value would >> consume too much memory >> + * and would affect other tests that run in parallel. >> + */ > Okay, I will fix as you suggested. > >> >> >> #2 What tests did you run? Did you run >> test/runtime/CommandLine/OptionsValidation on all platforms >> (including embedded)? > No. > I ran tests under test/runtime/CommandLine/OptionsValidation > (especially TestOptionsWithRanges.java) for all platforms except > embedded. > Let me back after testing on embedded. I ran for embedded (linux-arm64, linux-armvh, linux-armvfpsflt, linux-armvfphflt, linux-armsflt) and all of them PASSED for test/runtime/CommandLine/OptionsValidation. Thanks, Sangheon > > Thanks, > Sangheon > > >> >> >> cheers >> >> >> On 09/10/2015 07:01 PM, sangheon.kim wrote: >>> Hi all, >>> >>> Please review this patch for command-line validation for GC flags. >>> This REDO patch is adding ranges and implementing constraint >>> functions for GC flags. >>> >>> Original CR of JDK-8078555 was backout as it made a test failure >>> from 'TestOptionsWithRanges.java'. >>> And also there were some discussion of OOM handling. >>> >>> Most parts are same as JDK-8078555 except below: >>> 1. Changed 'range' for some flags. >>> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags >>> make this test unstable as it tries to allocate >>> huge amount of memory. >>> >>> And below are the suggestion note for JDK-8078555: >>> 1. Exponential notation for 'double' type variable parse: Previously >>> there were some discussion for maximum value for >>> double type flags from code review of JDK-8059557 and JDK-8112746. >>> And Kim and I decided not to add upper limit unless >>> there are problems with DBL_MAX. And as 255 is the maximum length >>> that can be passed via command-line, we introduced >>> exponential notation to avoid this limit. ( arguments.cpp ) >>> 2. These GC flags ranges are not ideal ranges but ranges which don't >>> make problem with current source code. >>> If one flag makes some problem but hard to find good range, I >>> added some ranges. >>> 3. There are some constraint functions to avoid overflow. >>> 4. Test applications are changed: as some of them assumed to be >>> ParallelGC or to check it's output messages. >>> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>> >>> Testing: >>> JPRT, UTE(vm.quick-pcl) and >>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >>> >>> Thanks, >>> Sangheon >>> >>> > From cnewland at chrisnewland.com Sat Sep 12 13:39:12 2015 From: cnewland at chrisnewland.com (Chris Newland) Date: Sat, 12 Sep 2015 14:39:12 +0100 Subject: Change in JDK9 LogCompilation XML header Message-ID: <037d62ed7e44b369c738e38a742e4bbc.squirrel@excalibur.xssl.net> Hi, Recently it looks like the code to output the XML LogCompilation header has changed resulting in two of the tags and not printing a cr before the closing tag: -XX:+UnlockDiagnosticVMOptions -XX:+TraceClassLoading -XX:+LogCompilation Historically the opening tag, text content, and closing tag have always been on separate lines. The other header tags still insert a cr before the closing tag. Difference is in share/vm/runtime/arguments.cpp jdk8u-dev: void Arguments::print_jvm_args_on(outputStream* st) { if (_num_jvm_args > 0) { for (int i=0; i < _num_jvm_args; i++) { st->print("%s ", _jvm_args_array[i]); } st->cr(); } } jdk9: void Arguments::print_jvm_args_on(outputStream* st) { if (_num_jvm_args > 0) { for (int i=0; i < _num_jvm_args; i++) { st->print("%s ", _jvm_args_array[i]); } } } If the cr was removed from arguments.cpp for a good reason then it can be added into utilities/ostream.cpp with if (Arguments::num_jvm_args() > 0) { xs->head("args"); Arguments::print_jvm_args_on(xs->text()); + xs->cr(); xs->tail("args"); } Thanks, Chris From christian.thalinger at oracle.com Mon Sep 14 07:24:52 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Sun, 13 Sep 2015 21:24:52 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface Message-ID: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> The JEP itself can be found here: https://bugs.openjdk.java.net/browse/JDK-8062493 Here are the webrevs: http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: https://bugs.openjdk.java.net/browse/JDK-8134994 SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. The integration will happen under the bug number: https://bugs.openjdk.java.net/browse/JDK-8136421 From erik.osterlund at lnu.se Mon Sep 14 09:35:03 2015 From: erik.osterlund at lnu.se (=?utf-8?B?RXJpayDDlnN0ZXJsdW5k?=) Date: Mon, 14 Sep 2015 09:35:03 +0000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> Message-ID: <7ECDFFE9-FEFB-4538-9427-9B8EC8714E55@lnu.se> Hi Christian, It is very interesting to expose a Java-level API for the compilers, and it makes me happy to read about it. Out of curiosity, I?m wondering two things. 1. Are there intentions to target release builds eventually, so library implementors can use custom JITs, and not just for experimenting? 2. In that case, I wonder what impact these potential library JITs will have on the GC interface/ABI. I assume it is intended that the already implemented GCs will be used by this new JIT. In that case, I assume there needs to be some kind of interface between the JIT and GC. In particular what I am thinking about is exposing some current GC ABI assumptions including what GC barriers there could be for memory accesses (e.g. barriers only for references, not primitives), then never being able to change the such assumptions further down the road because third party libraries with custom JITs suddenly depend on previously valid ABI and compiler details that have to be set in stone for compatibility reasons. Any comments on this? Is this an issue? Thanks, /Erik > On 14 Sep 2015, at 09:24, Christian Thalinger wrote: > > The JEP itself can be found here: > > https://bugs.openjdk.java.net/browse/JDK-8062493 > > Here are the webrevs: > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ > > The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: > > https://bugs.openjdk.java.net/browse/JDK-8134994 > > SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. > > The integration will happen under the bug number: > > https://bugs.openjdk.java.net/browse/JDK-8136421 > From marcus.larsson at oracle.com Mon Sep 14 13:18:10 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Mon, 14 Sep 2015 15:18:10 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F2DC58.7070004@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> Message-ID: <55F6C912.2020504@oracle.com> On 2015-09-11 15:51, Coleen Phillimore wrote: > > I'm not a big fan of having these blanks in the logging lines. I > don't think making the line length longer is going to be helpful and > what people are looking for is the message at the end, not which tag > and level they used. I think they look kind of strange. > > Coleen While it might look a bit weird, I think it can help readers to quickly find the start of the actual log message. Maybe we could leave it as it is for now, deferring the decision if we want the padding or not until we have some real logs to look at. If we find the padding inconvenient at that point we can just remove it in a follow-up RFE. Thanks, Marcus > > On 9/11/15 9:40 AM, Marcus Larsson wrote: >> Yes, decorators are padded to avoid jagged logs and help readability. >> Since the levels are known beforehand, a fixed padding is used for >> that decorator. For other decorators such as tags or timestamps, the >> padding will grow to the size of the longest (so far) seen >> decoration. This means the decorator prefix length will either stay >> the same or increase, but never decrease. After a while it should >> stabilize around some fitting length and not grow significantly. >> >> For example: >> [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) >> [0.656s][debug ][safepoint, some_other_tag] Safepoint >> synchronization initiated. (20) >> [0.657s][debug ][safepoint, ] Safepoint >> synchronization initiated. (20) >> [10.657s][debug ][safepoint, ] Safepoint >> synchronization initiated. (20) \ > From gerard.ziemski at oracle.com Mon Sep 14 14:23:52 2015 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 14 Sep 2015 09:23:52 -0500 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F3E484.70107@oracle.com> References: <55F219CE.3010006@oracle.com> <55F319B8.1000803@oracle.com> <55F32A81.8000307@oracle.com> <55F3E484.70107@oracle.com> Message-ID: <55F6D878.1070903@oracle.com> Thank you. I have no more comments - reviewed. cheers On 09/12/2015 03:38 AM, sangheon.kim wrote: > Hi Gerard, > > On 09/11/2015 12:24 PM, sangheon.kim wrote: >> Hi Gerard, >> >> Thank you for looking at this. >> >> On 09/11/2015 11:13 AM, gerard ziemski wrote: >>> hi Sangheon, >>> >>> #1 test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>> >>> Please change the comment to: >>> >>> + /* >>> + * Exclude below options as their maximum value would consume too much memory >>> + * and would affect other tests that run in parallel. >>> + */ >> Okay, I will fix as you suggested. >> >>> >>> >>> #2 What tests did you run? Did you run test/runtime/CommandLine/OptionsValidation on all platforms (including embedded)? >> No. >> I ran tests under test/runtime/CommandLine/OptionsValidation (especially TestOptionsWithRanges.java) for all platforms >> except embedded. >> Let me back after testing on embedded. > I ran for embedded (linux-arm64, linux-armvh, linux-armvfpsflt, linux-armvfphflt, linux-armsflt) and all of them PASSED > for test/runtime/CommandLine/OptionsValidation. > > Thanks, > Sangheon > > >> >> Thanks, >> Sangheon >> >> >>> >>> >>> cheers >>> >>> >>> On 09/10/2015 07:01 PM, sangheon.kim wrote: >>>> Hi all, >>>> >>>> Please review this patch for command-line validation for GC flags. >>>> This REDO patch is adding ranges and implementing constraint functions for GC flags. >>>> >>>> Original CR of JDK-8078555 was backout as it made a test failure from 'TestOptionsWithRanges.java'. >>>> And also there were some discussion of OOM handling. >>>> >>>> Most parts are same as JDK-8078555 except below: >>>> 1. Changed 'range' for some flags. >>>> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags make this test unstable as it tries to allocate >>>> huge amount of memory. >>>> >>>> And below are the suggestion note for JDK-8078555: >>>> 1. Exponential notation for 'double' type variable parse: Previously there were some discussion for maximum value for >>>> double type flags from code review of JDK-8059557 and JDK-8112746. And Kim and I decided not to add upper limit unless >>>> there are problems with DBL_MAX. And as 255 is the maximum length that can be passed via command-line, we introduced >>>> exponential notation to avoid this limit. ( arguments.cpp ) >>>> 2. These GC flags ranges are not ideal ranges but ranges which don't make problem with current source code. >>>> If one flag makes some problem but hard to find good range, I added some ranges. >>>> 3. There are some constraint functions to avoid overflow. >>>> 4. Test applications are changed: as some of them assumed to be ParallelGC or to check it's output messages. >>>> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>>> >>>> Testing: >>>> JPRT, UTE(vm.quick-pcl) and test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >>>> >>>> Thanks, >>>> Sangheon >>>> >>>> >> > > From ioi.lam at oracle.com Mon Sep 14 16:01:53 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 14 Sep 2015 09:01:53 -0700 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F6C912.2020504@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> Message-ID: <55F6EF71.80005@oracle.com> On 9/14/15 6:18 AM, Marcus Larsson wrote: > > On 2015-09-11 15:51, Coleen Phillimore wrote: >> >> I'm not a big fan of having these blanks in the logging lines. I >> don't think making the line length longer is going to be helpful and >> what people are looking for is the message at the end, not which tag >> and level they used. I think they look kind of strange. >> >> Coleen > > While it might look a bit weird, I think it can help readers to > quickly find the start of the actual log message. Maybe we could leave > it as it is for now, deferring the decision if we want the padding or > not until we have some real logs to look at. If we find the padding > inconvenient at that point we can just remove it in a follow-up RFE. > I agree. Eliminating the extra spaces is quite easy with a sed script. It's harder to use a script to add the spaces back to align the output. - Ioi > Thanks, > Marcus > >> >> On 9/11/15 9:40 AM, Marcus Larsson wrote: >>> Yes, decorators are padded to avoid jagged logs and help >>> readability. Since the levels are known beforehand, a fixed padding >>> is used for that decorator. For other decorators such as tags or >>> timestamps, the padding will grow to the size of the longest (so >>> far) seen decoration. This means the decorator prefix length will >>> either stay the same or increase, but never decrease. After a while >>> it should stabilize around some fitting length and not grow >>> significantly. >>> >>> For example: >>> [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) >>> [0.656s][debug ][safepoint, some_other_tag] Safepoint >>> synchronization initiated. (20) >>> [0.657s][debug ][safepoint, ] Safepoint >>> synchronization initiated. (20) >>> [10.657s][debug ][safepoint, ] Safepoint >>> synchronization initiated. (20) \ >> > From sangheon.kim at oracle.com Mon Sep 14 16:04:04 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Mon, 14 Sep 2015 12:04:04 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F6D878.1070903@oracle.com> References: <55F219CE.3010006@oracle.com> <55F319B8.1000803@oracle.com> <55F32A81.8000307@oracle.com> <55F3E484.70107@oracle.com> <55F6D878.1070903@oracle.com> Message-ID: <55F6EFF4.4030508@oracle.com> Hi Gerard, Thanks for the review. Sangheon On 09/14/2015 10:23 AM, gerard ziemski wrote: > Thank you. I have no more comments - reviewed. > > > cheers > > > On 09/12/2015 03:38 AM, sangheon.kim wrote: >> Hi Gerard, >> >> On 09/11/2015 12:24 PM, sangheon.kim wrote: >>> Hi Gerard, >>> >>> Thank you for looking at this. >>> >>> On 09/11/2015 11:13 AM, gerard ziemski wrote: >>>> hi Sangheon, >>>> >>>> #1 >>>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>> >>>> Please change the comment to: >>>> >>>> + /* >>>> + * Exclude below options as their maximum value would >>>> consume too much memory >>>> + * and would affect other tests that run in parallel. >>>> + */ >>> Okay, I will fix as you suggested. >>> >>>> >>>> >>>> #2 What tests did you run? Did you run >>>> test/runtime/CommandLine/OptionsValidation on all platforms >>>> (including embedded)? >>> No. >>> I ran tests under test/runtime/CommandLine/OptionsValidation >>> (especially TestOptionsWithRanges.java) for all platforms >>> except embedded. >>> Let me back after testing on embedded. >> I ran for embedded (linux-arm64, linux-armvh, linux-armvfpsflt, >> linux-armvfphflt, linux-armsflt) and all of them PASSED >> for test/runtime/CommandLine/OptionsValidation. >> >> Thanks, >> Sangheon >> >> >>> >>> Thanks, >>> Sangheon >>> >>> >>>> >>>> >>>> cheers >>>> >>>> >>>> On 09/10/2015 07:01 PM, sangheon.kim wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch for command-line validation for GC flags. >>>>> This REDO patch is adding ranges and implementing constraint >>>>> functions for GC flags. >>>>> >>>>> Original CR of JDK-8078555 was backout as it made a test failure >>>>> from 'TestOptionsWithRanges.java'. >>>>> And also there were some discussion of OOM handling. >>>>> >>>>> Most parts are same as JDK-8078555 except below: >>>>> 1. Changed 'range' for some flags. >>>>> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These >>>>> flags make this test unstable as it tries to allocate >>>>> huge amount of memory. >>>>> >>>>> And below are the suggestion note for JDK-8078555: >>>>> 1. Exponential notation for 'double' type variable parse: >>>>> Previously there were some discussion for maximum value for >>>>> double type flags from code review of JDK-8059557 and JDK-8112746. >>>>> And Kim and I decided not to add upper limit unless >>>>> there are problems with DBL_MAX. And as 255 is the maximum length >>>>> that can be passed via command-line, we introduced >>>>> exponential notation to avoid this limit. ( arguments.cpp ) >>>>> 2. These GC flags ranges are not ideal ranges but ranges which >>>>> don't make problem with current source code. >>>>> If one flag makes some problem but hard to find good range, I >>>>> added some ranges. >>>>> 3. There are some constraint functions to avoid overflow. >>>>> 4. Test applications are changed: as some of them assumed to be >>>>> ParallelGC or to check it's output messages. >>>>> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >>>>> >>>>> CR: >>>>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>>>> >>>>> Testing: >>>>> JPRT, UTE(vm.quick-pcl) and >>>>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >>>>> >>>>> >>>>> Thanks, >>>>> Sangheon >>>>> >>>>> >>> >> >> From kirk.pepperdine at gmail.com Mon Sep 14 16:37:43 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Mon, 14 Sep 2015 18:37:43 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F6C912.2020504@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> Message-ID: <6FA04C2D-1CFB-4B79-AE0B-132DDB54AA5E@gmail.com> Hi, In my opinion, excessive spaces don?t enhance readability and and generally make parsing more difficult. Kind regards, Kirk Pepperdine > On Sep 14, 2015, at 3:18 PM, Marcus Larsson wrote: > > > On 2015-09-11 15:51, Coleen Phillimore wrote: >> >> I'm not a big fan of having these blanks in the logging lines. I don't think making the line length longer is going to be helpful and what people are looking for is the message at the end, not which tag and level they used. I think they look kind of strange. >> >> Coleen > > While it might look a bit weird, I think it can help readers to quickly find the start of the actual log message. Maybe we could leave it as it is for now, deferring the decision if we want the padding or not until we have some real logs to look at. If we find the padding inconvenient at that point we can just remove it in a follow-up RFE. > > Thanks, > Marcus > >> >> On 9/11/15 9:40 AM, Marcus Larsson wrote: >>> Yes, decorators are padded to avoid jagged logs and help readability. Since the levels are known beforehand, a fixed padding is used for that decorator. For other decorators such as tags or timestamps, the padding will grow to the size of the longest (so far) seen decoration. This means the decorator prefix length will either stay the same or increase, but never decrease. After a while it should stabilize around some fitting length and not grow significantly. >>> >>> For example: >>> [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) >>> [0.656s][debug ][safepoint, some_other_tag] Safepoint synchronization initiated. (20) >>> [0.657s][debug ][safepoint, ] Safepoint synchronization initiated. (20) >>> [10.657s][debug ][safepoint, ] Safepoint synchronization initiated. (20) \ >> > From vladimir.kozlov at oracle.com Mon Sep 14 17:07:43 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 14 Sep 2015 10:07:43 -0700 Subject: nmethod::make_not_entrant() and native methods In-Reply-To: <55F29D43.6070102@redhat.com> References: <55F29D43.6070102@redhat.com> Message-ID: <55F6FEDF.1040603@oracle.com> I am not sure. When we do class redefining we may deoptimize all nmethods. Vladimir On 9/11/15 2:22 AM, Andrew Haley wrote: > On AArch64 we have to place a NOP at the start of every compiled > method so that we can, if needs be, patch its verified entry point. > This is because of an architectural restriction that only nops, traps, > and branches may be patched while code is running. > > I am guessing that we do not need a NOP at the start of a compiled > native wrapper because it will never be deoptimized. Is that right? > > Thanks, > Andrew. > From aph at redhat.com Mon Sep 14 17:12:57 2015 From: aph at redhat.com (Andrew Haley) Date: Mon, 14 Sep 2015 18:12:57 +0100 Subject: nmethod::make_not_entrant() and native methods In-Reply-To: <55F6FEDF.1040603@oracle.com> References: <55F29D43.6070102@redhat.com> <55F6FEDF.1040603@oracle.com> Message-ID: <55F70019.4080307@redhat.com> On 09/14/2015 06:07 PM, Vladimir Kozlov wrote: > When we do class redefining we may deoptimize all nmethods. Got it, thanks. Andrew. From christian.thalinger at oracle.com Mon Sep 14 17:40:04 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 14 Sep 2015 07:40:04 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <7ECDFFE9-FEFB-4538-9427-9B8EC8714E55@lnu.se> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <7ECDFFE9-FEFB-4538-9427-9B8EC8714E55@lnu.se> Message-ID: <90AFFD76-5EEF-4D6F-AE43-DE63463B4C04@oracle.com> > On Sep 13, 2015, at 11:35 PM, Erik ?sterlund wrote: > > Hi Christian, > > It is very interesting to expose a Java-level API for the compilers, and it makes me happy to read about it. > > Out of curiosity, I?m wondering two things. > > 1. Are there intentions to target release builds eventually, so library implementors can use custom JITs, and not just for experimenting? The JEP is targeted for JDK 9 so this feature should be (if nothing serious goes wrong) in JDK 9 GA. The JVMCI will be hidden behind experimental flags but you can turn it on any time in Oracle JDK product builds. > > 2. In that case, I wonder what impact these potential library JITs will have on the GC interface/ABI. > I assume it is intended that the already implemented GCs will be used by this new JIT. > In that case, I assume there needs to be some kind of interface between the JIT and GC. > > In particular what I am thinking about is exposing some current GC ABI assumptions including what GC barriers there could be for memory accesses (e.g. barriers only for references, not primitives), then never being able to change the such assumptions further down the road because third party libraries with custom JITs suddenly depend on previously valid ABI and compiler details that have to be set in stone for compatibility reasons. > > Any comments on this? Is this an issue? This API is for HotSpot and all JIT compilers using this interface must follow the rules the HotSpot GCs have. If your JIT supports Parallel GC you have implement whatever barriers Parallel requires. Same for CMS and G1. There is no "GC interface? because there is nothing to be queried really. The type of barrier code JIT compilers need to emit for the different GCs is pre-defined and might also change (for whatever reason). In our current world if something changes we have to change C1 and C2 at the same time. External JIT compilers will definitely lag behind but in order to work properly they have to be updated as well. I hope this answers your question. > > Thanks, > /Erik > >> On 14 Sep 2015, at 09:24, Christian Thalinger wrote: >> >> The JEP itself can be found here: >> >> https://bugs.openjdk.java.net/browse/JDK-8062493 >> >> Here are the webrevs: >> >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >> >> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >> >> https://bugs.openjdk.java.net/browse/JDK-8134994 >> >> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >> >> The integration will happen under the bug number: >> >> https://bugs.openjdk.java.net/browse/JDK-8136421 >> > From kirk.pepperdine at gmail.com Mon Sep 14 21:55:54 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Mon, 14 Sep 2015 23:55:54 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F6EF71.80005@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> Message-ID: > On Sep 14, 2015, at 6:01 PM, Ioi Lam wrote: > > > > On 9/14/15 6:18 AM, Marcus Larsson wrote: >> >> On 2015-09-11 15:51, Coleen Phillimore wrote: >>> >>> I'm not a big fan of having these blanks in the logging lines. I don't think making the line length longer is going to be helpful and what people are looking for is the message at the end, not which tag and level they used. I think they look kind of strange. >>> >>> Coleen >> >> While it might look a bit weird, I think it can help readers to quickly find the start of the actual log message. Maybe we could leave it as it is for now, deferring the decision if we want the padding or not until we have some real logs to look at. If we find the padding inconvenient at that point we can just remove it in a follow-up RFE. >> > I agree. Eliminating the extra spaces is quite easy with a sed script. It's harder to use a script to add the spaces back to align the output. This is a mess that I just finished dealing with. I believe UL will solve the interleaving but not the internal formatting issues but it won?t solve a number of other existing issues. 100081.540: [Full GC100081.540: [GC concurrent-root-region-scan-start] 100081.540: [GC concurrent-root-region-scan-end, 0.0000080 secs] 100081.540: [GC concurrent-mark-start] 100084.860: [SoftReference, 22 refs, 0.0000340 secs]100084.860: [WeakReference, 169 refs, 0.0000180 secs]100084.860: [FinalReference, 18 refs, 0.0000050 secs]100084.860: [PhantomReference, 0 refs, 0.0000040 secs]100084.860: [JNI Weak Reference, 0.0000040 secs] 510M->510M(512M), 4.3335850 secs] [Eden: 0.0B(25.0M)->0.0B(25.0M) Survivors: 0.0B->0.0B Heap: 510.9M(512.0M)->510.9M(512.0M)] [Times: user=3.44 sys=1.62, real=4.34 secs] This is but 2.5 data points. To make any reasonable decisions I?m looking at several 1000s of these data points and there is no way you?re going to do that without tooling. In fact, there are behaviors that are simply not visible unless you run some analytics over the data. For that reason alone I advocate machine readable should be a top priority. One thing that helps is losing some of the arbitrary formatting. For example, the use of square brackets here look very reasonable. However you can see the GC logs also use square brackets (albeit inconsistently) so now you have potential collisions with the symbols which will increase the complexity of parsing. This is the primary reason there are very few reasonably good GC log analysis tools available for OpenJDK. Sure you can adjust for spaces and other things but over versions you end up with enough differences due to these small formatting changes that also increases the complexity of the parsing. The simplest logs and the most useful ones are produced by Azul and HP (derivative of OpenJDK). There are both simply CSV flat files with some header information. No fuss, no muss? Good for North America but not for the rest of the world... Kind regards, Kirk From ioi.lam at oracle.com Tue Sep 15 02:38:04 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 14 Sep 2015 19:38:04 -0700 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> Message-ID: <55F7848C.3010509@oracle.com> On 9/14/15 2:55 PM, Kirk Pepperdine wrote: >> On Sep 14, 2015, at 6:01 PM, Ioi Lam wrote: >> >> >> >> On 9/14/15 6:18 AM, Marcus Larsson wrote: >>> On 2015-09-11 15:51, Coleen Phillimore wrote: >>>> I'm not a big fan of having these blanks in the logging lines. I don't think making the line length longer is going to be helpful and what people are looking for is the message at the end, not which tag and level they used. I think they look kind of strange. >>>> >>>> Coleen >>> While it might look a bit weird, I think it can help readers to quickly find the start of the actual log message. Maybe we could leave it as it is for now, deferring the decision if we want the padding or not until we have some real logs to look at. If we find the padding inconvenient at that point we can just remove it in a follow-up RFE. >>> >> I agree. Eliminating the extra spaces is quite easy with a sed script. It's harder to use a script to add the spaces back to align the output. > > This is a mess that I just finished dealing with. I believe UL will solve the interleaving but not the internal formatting issues but it won?t solve a number of other existing issues. > > 100081.540: [Full GC100081.540: [GC concurrent-root-region-scan-start] > 100081.540: [GC concurrent-root-region-scan-end, 0.0000080 secs] > 100081.540: [GC concurrent-mark-start] > 100084.860: [SoftReference, 22 refs, 0.0000340 secs]100084.860: [WeakReference, 169 refs, 0.0000180 secs]100084.860: [FinalReference, 18 refs, 0.0000050 secs]100084.860: [PhantomReference, 0 refs, 0.0000040 secs]100084.860: [JNI Weak Reference, 0.0000040 secs] 510M->510M(512M), 4.3335850 secs] > [Eden: 0.0B(25.0M)->0.0B(25.0M) Survivors: 0.0B->0.0B Heap: 510.9M(512.0M)->510.9M(512.0M)] > [Times: user=3.44 sys=1.62, real=4.34 secs] > > This is but 2.5 data points. To make any reasonable decisions I?m looking at several 1000s of these data points and there is no way you?re going to do that without tooling. In fact, there are behaviors that are simply not visible unless you run some analytics over the data. For that reason alone I advocate machine readable should be a top priority. One thing that helps is losing some of the arbitrary formatting. For example, the use of square brackets here look very reasonable. However you can see the GC logs also use square brackets (albeit inconsistently) so now you have potential collisions with the symbols which will increase the complexity of parsing. This is the primary reason there are very few reasonably good GC log analysis tools available for OpenJDK. Sure you can adjust for spaces and other things but over versions you end up with enough differences due to these small formatting changes that also increases the complexity of the parsing. The simplest logs and the most useful ones are produced by Azul and HP (derivative of OpenJDK). There are both simply CSV flat files with some header information. No fuss, no muss? Good for North America but not for the rest of the world... (Kirk, I am not sure if human readable text would necessarily mean unparsable text. I've written many many log parsing scripts using regexp matching. In order to write a good parser, I need to understand what's in the log. It's much easier if the log is human readable, like size = 1234 bytes, speed = 5678 ms rather than 1234,5678 UL allows a mixture of several types of logs (e.g., GC and class loading). I don't know how this can be represented in a CSV file. My criteria for a good log is: + human readable + consistent + easy to parse The common UL code really just handles the stuff in the first 3 [][][] brackets: [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) [0.656s][debug ][safepoint, some_other_tag] Safepoint synchronization initiated. (20) [0.657s][debug ][safepoint, ] Safepoint synchronization initiated. (20) [10.657s][debug ][safepoint, ] Safepoint synchronization initiated. (20) and they meet my 3 criteria. The rest of the log is the responsibility of the individual component, and is really not in the scope of the UL common code. - Ioi > Kind regards, > Kirk > From erik.osterlund at lnu.se Tue Sep 15 07:25:12 2015 From: erik.osterlund at lnu.se (=?utf-8?B?RXJpayDDlnN0ZXJsdW5k?=) Date: Tue, 15 Sep 2015 07:25:12 +0000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <90AFFD76-5EEF-4D6F-AE43-DE63463B4C04@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <7ECDFFE9-FEFB-4538-9427-9B8EC8714E55@lnu.se> <90AFFD76-5EEF-4D6F-AE43-DE63463B4C04@oracle.com> Message-ID: <052BEBC8-1D01-4468-94A7-6DE2C9FDE087@lnu.se> > On 14 Sep 2015, at 19:40, Christian Thalinger wrote: > >> >> On Sep 13, 2015, at 11:35 PM, Erik ?sterlund > wrote: >> >> Hi Christian, >> >> It is very interesting to expose a Java-level API for the compilers, and it makes me happy to read about it. >> >> Out of curiosity, I?m wondering two things. >> >> 1. Are there intentions to target release builds eventually, so library implementors can use custom JITs, and not just for experimenting? > > The JEP is targeted for JDK 9 so this feature should be (if nothing serious goes wrong) in JDK 9 GA. The JVMCI will be hidden behind experimental flags but you can turn it on any time in Oracle JDK product builds. Okay. > >> >> 2. In that case, I wonder what impact these potential library JITs will have on the GC interface/ABI. >> I assume it is intended that the already implemented GCs will be used by this new JIT. >> In that case, I assume there needs to be some kind of interface between the JIT and GC. >> >> In particular what I am thinking about is exposing some current GC ABI assumptions including what GC barriers there could be for memory accesses (e.g. barriers only for references, not primitives), then never being able to change the such assumptions further down the road because third party libraries with custom JITs suddenly depend on previously valid ABI and compiler details that have to be set in stone for compatibility reasons. >> >> Any comments on this? Is this an issue? > > This API is for HotSpot and all JIT compilers using this interface must follow the rules the HotSpot GCs have. If your JIT supports Parallel GC you have implement whatever barriers Parallel requires. Same for CMS and G1. Okay. > There is no "GC interface? because there is nothing to be queried really. The type of barrier code JIT compilers need to emit for the different GCs is pre-defined and might also change (for whatever reason). In our current world if something changes we have to change C1 and C2 at the same time. External JIT compilers will definitely lag behind but in order to work properly they have to be updated as well. I suspected as much. I was just worried that such a feature would make it increasingly difficult to make changes to e.g. GC barriers (for whatever reason). Before you had to change code in HotSpot components (C1, C2, interpreter, runtime) which is possible to deal with and you know if the JVM is fixed properly, code will run well in Java-land. But now you might have to change all third party compilers outside of the JVM as well, and in the worst case different libraries prefer different compilers that are automatically packaged and spread around like a plague into products using dependency tools like maven without the end-user even having a clue there are custom JIT-compilers packaged in their applications making lots of assumptions about JVM/GC internals. The reason I am a bit worried is not just because it?s a hassle to change more compilers, but because I think the issue of the GC having a contract (implicit or explicit, same difference) with third party JITs in Java-land is a very similar issue as having a kernel with an ABI contract and code in user land relying on it. And in the same sense that this ABI locks kernel developers into corners because they cannot break user-space, I fear that we could be locked in as well for the very same reasons: that changing the implicit or explicit GC ABI contract to the JITs (which previously was a bit of hassle but doable) would suddenly break Java-land that has suddenly been infected with assumptions about JVM/GC internals. And we would then not be able to make improvements that break Java-land ABI assumptions, because every new release of Java has to be compatible with the previous releases so that code already written will continue to work. I don?t know, maybe I?m a bit paranoid and overreacting. Maybe compilers will be very few in practice and shipped together with the JVM itself so they are guaranteed to be compatible, rather than spread out in libraries shipped with applications in very few carefully considered cases. And obviously this imaginary JIT apocalypse won?t happen as long as it?s an experimental flag. Just thought I?d share my point of view so that if an unbreakable Java-land ABI would pop up out of nowhere to constrain us, at least we thought about it, saw it coming and are fully aware of it. > I hope this answers your question. Yes it does, thank you! Regards, /Erik > >> >> Thanks, >> /Erik >> >>> On 14 Sep 2015, at 09:24, Christian Thalinger wrote: >>> >>> The JEP itself can be found here: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>> >>> Here are the webrevs: >>> >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>> >>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>> >>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>> >>> The integration will happen under the bug number: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8136421 From kirk.pepperdine at gmail.com Tue Sep 15 07:32:08 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 15 Sep 2015 09:32:08 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F7848C.3010509@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> Message-ID: Hi Loi, > > I am not sure if human readable text would necessarily mean unparsable text. I've written many many log parsing scripts using regexp matching. Well, if we looked at a regex?s needed to parse the examples below it would be? (don?t forget the localization difference between European and North American number formats) \\[(\\d+(?:.|,))s\\]\\[(.+)\\]\\[and now it gets messy for a bit because regex doesn?t handle repeated fields all that well...\\](.+) So messy could be (.+), extract that capture group and then split it, trim out spaces or something like that. Looks ok unless there are square braces in the tags this looks better than before. My only question is, where would dates show up? People like dates as it helps them correlate events to other bits of information they might have in other logs. If we add that for a regular DateTime format and if you allow for one or not the other you get? \\[(?:(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{3}[\\+|\\-]\\d{4}): )?(-?\\d+(?:\\.|,)\\d+? + ?)]\\[(.+)\\]\\[and now it gets messy for a bit\\](.+) and so on? I don?t mind if it remains consistent. However, my experience is that people will inject arbitrary changes (that is changes with no real added value). You don?t really see this if you are supporting tooling for your own in-house applications because you pick up the change, and then move on. If you are supporting multiple version of the JVM then it gets interesting. > > In order to write a good parser, I need to understand what's in the log. It's much easier if the log is human readable, like > > size = 1234 bytes, speed = 5678 ms > > rather than > > 1234,5678 > > UL allows a mixture of several types of logs (e.g., GC and class loading). I don't know how this can be represented in a CSV file. And indeed you don?t want a system that does the equivalent of a pretty print to stdout. It is the programming equivalent of dumpster diving. It prevents one from being able to make reasonable decisions that increase the information density in logs. I say this because I?ll repeat more than 60% of the applications I run into currently have performance issues that are related to how logging as been implemented. In ~40% of the applications I tune the primary problem is logging. My hope was that UL would not add to the problem but I fear it will. My problem with size = xxxx bytes, speed = xxxx ms is an exceptionally information sparse format. And I would argue that you only need to be told one what this represent 1887488K->1887488K(1887488K). It?s verbose enough. So, having a single header size in bytes, speed in ms followed by lines of xxxx,xxxx is a reasonable compromise. > > My criteria for a good log is: > > + human readable > + consistent > + easy to parse +good information density. Kind regards, Kirk From yuri.kushch at gmail.com Tue Sep 15 07:36:59 2015 From: yuri.kushch at gmail.com (=?UTF-8?B?0K7RgNC40Lkg0JrRg9GJ?=) Date: Tue, 15 Sep 2015 10:36:59 +0300 Subject: Getting ready Message-ID: Hello all, I'm going through group's documentation ( http://openjdk.java.net/groups/hotspot/) and I'm not sure where should I start as for help. Browsable source has broken link http://hg.openjdk.java.net/jdk7/hotspot/hotspot/ Probably you have some advises for me? Thanks in advance! From staffan.larsen at oracle.com Tue Sep 15 08:09:37 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 15 Sep 2015 10:09:37 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F7848C.3010509@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> Message-ID: <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> > On 15 sep 2015, at 04:38, Ioi Lam wrote: > > > > On 9/14/15 2:55 PM, Kirk Pepperdine wrote: >>> On Sep 14, 2015, at 6:01 PM, Ioi Lam wrote: >>> >>> >>> >>> On 9/14/15 6:18 AM, Marcus Larsson wrote: >>>> On 2015-09-11 15:51, Coleen Phillimore wrote: >>>>> I'm not a big fan of having these blanks in the logging lines. I don't think making the line length longer is going to be helpful and what people are looking for is the message at the end, not which tag and level they used. I think they look kind of strange. >>>>> >>>>> Coleen >>>> While it might look a bit weird, I think it can help readers to quickly find the start of the actual log message. Maybe we could leave it as it is for now, deferring the decision if we want the padding or not until we have some real logs to look at. If we find the padding inconvenient at that point we can just remove it in a follow-up RFE. >>>> >>> I agree. Eliminating the extra spaces is quite easy with a sed script. It's harder to use a script to add the spaces back to align the output. >> >> This is a mess that I just finished dealing with. I believe UL will solve the interleaving but not the internal formatting issues but it won?t solve a number of other existing issues. >> >> 100081.540: [Full GC100081.540: [GC concurrent-root-region-scan-start] >> 100081.540: [GC concurrent-root-region-scan-end, 0.0000080 secs] >> 100081.540: [GC concurrent-mark-start] >> 100084.860: [SoftReference, 22 refs, 0.0000340 secs]100084.860: [WeakReference, 169 refs, 0.0000180 secs]100084.860: [FinalReference, 18 refs, 0.0000050 secs]100084.860: [PhantomReference, 0 refs, 0.0000040 secs]100084.860: [JNI Weak Reference, 0.0000040 secs] 510M->510M(512M), 4.3335850 secs] >> [Eden: 0.0B(25.0M)->0.0B(25.0M) Survivors: 0.0B->0.0B Heap: 510.9M(512.0M)->510.9M(512.0M)] >> [Times: user=3.44 sys=1.62, real=4.34 secs] >> >> This is but 2.5 data points. To make any reasonable decisions I?m looking at several 1000s of these data points and there is no way you?re going to do that without tooling. In fact, there are behaviors that are simply not visible unless you run some analytics over the data. For that reason alone I advocate machine readable should be a top priority. One thing that helps is losing some of the arbitrary formatting. For example, the use of square brackets here look very reasonable. However you can see the GC logs also use square brackets (albeit inconsistently) so now you have potential collisions with the symbols which will increase the complexity of parsing. This is the primary reason there are very few reasonably good GC log analysis tools available for OpenJDK. Sure you can adjust for spaces and other things but over versions you end up with enough differences due to these small formatting changes that also increases the complexity of the parsing. The simplest logs and the most useful ones are produced by Azul and HP (derivative of OpenJDK). There are both simply CSV flat files with some header information. No fuss, no muss? Good for North America but not for the rest of the world... > > (Kirk, > > I am not sure if human readable text would necessarily mean unparsable text. I've written many many log parsing scripts using regexp matching. > > In order to write a good parser, I need to understand what's in the log. It's much easier if the log is human readable, like > > size = 1234 bytes, speed = 5678 ms > > rather than > > 1234,5678 > > UL allows a mixture of several types of logs (e.g., GC and class loading). I don't know how this can be represented in a CSV file. > > My criteria for a good log is: > > + human readable > + consistent > + easy to parse > > The common UL code really just handles the stuff in the first 3 [][][] brackets: Just for completeness: there can be more than 3 bracket pairs if you configure additional decorations: $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info ][logging] Log configuration fully initialized. /Staffan > > [0.655s][debug ][safepoint] Safepoint synchronization initiated. (20) > [0.656s][debug ][safepoint, some_other_tag] Safepoint > synchronization initiated. (20) > [0.657s][debug ][safepoint, ] Safepoint > synchronization initiated. (20) > [10.657s][debug ][safepoint, ] Safepoint > synchronization initiated. (20) > > and they meet my 3 criteria. The rest of the log is the responsibility of the individual component, and is really not in the scope of the UL common code. > > - Ioi >> Kind regards, >> Kirk >> > From marcus.larsson at oracle.com Tue Sep 15 09:36:53 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Tue, 15 Sep 2015 11:36:53 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <55F7E6B5.2040502@oracle.com> Hi, Here is the updated version of the patch, including the feedback and suggestions from Coleen, David and Ioi. Incremental webrev: http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ Full webrev: http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ Changes include: * New NMT tag 'mtLogging' used by UL * Added storage allocation classes for all new classes * Added Log::puts(const char*) for plain string messages of any length * Fixed broken printf-attribute on Log::write functions * Moved VM startup timestamp to after os::init * Formatting and code style changes Thanks, Marcus On 2015-09-07 15:33, Marcus Larsson wrote: > Hi, > > Please review the following patch adding the unified logging framework > to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. > Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the > unified logging framework. The main entry point is log.hpp, which > contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template > arguments to the Log class. Every combination of tags used in a log > call has a corresponding LogTagSet instance, which keeps a track of > all the outputs it should write the log message to (and their levels). > Having tags as template arguments allows mapping directly from a set > of tags to the LogTagSet instance, which means that the overhead for > disabled logging should be low. Currently each log message can be > tagged with up to 5 tags, but this can be increased if ever required > (and with C++11's variadic templates the limit can be removed > completely). > > The LogConfiguration class keeps track of configured outputs (stdout, > stderr, and possible file outputs). Configuration is done either by > command line arguments (-Xlog) or by DCMD. Both methods will in turn > use the LogConfiguration class to perform the parsing & configuration. > This configuration includes iterating over all LogTagSet instances and > updating them accordingly. The LogTagLevelExpression class is used to > represent the selection of tags and levels for a given configuration > request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances > of this class is kept in LogTagSet to track what decorators to use > (this is the union of all decorators used by its outputs). Each log > call will create a LogDecorations instance (note: different classes), > which will contain the actual decoration strings for the log message. > These decorations are used for each output the tagset is set to log > on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be > prefixed. The prefix should supply a printf-style format with > argument. (This allows GC logging to prefix messages of certain > tagsets with GCId.) Prefixes are implemented using template > specializations based on the specified tags, with the > general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log > outputs. LogFileStreamOutput corresponds to FILE* stream based log > outputs. LogFileOutput builds on this and adds the file management and > log rotation support. > > A simple jtreg test is included in this patch. Additional tests will > be added at a later stage. > > Thanks, > Marcus From volker.simonis at gmail.com Tue Sep 15 09:48:52 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 15 Sep 2015 11:48:52 +0200 Subject: Getting ready In-Reply-To: References: Message-ID: Hi Yuri, that page is terribly outdated :( You can find the current HS development repositories under: http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/ for the JIT compiler team and: http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/ for the GC and Runtime team. The consolidated changes can be found in: http://hg.openjdk.java.net/jdk9/hs/hotspot/ You may also want to have a look at the HotSpot Wiki which has a little more information: https://wiki.openjdk.java.net/display/HotSpot/Main Regards, Volker On Tue, Sep 15, 2015 at 9:36 AM, ???? ??? wrote: > Hello all, > > I'm going through group's documentation ( > http://openjdk.java.net/groups/hotspot/) and I'm not sure where should I > start as for help. Browsable source has broken link > http://hg.openjdk.java.net/jdk7/hotspot/hotspot/ > > Probably you have some advises for me? > > Thanks in advance! From yuri.kushch at gmail.com Tue Sep 15 12:16:45 2015 From: yuri.kushch at gmail.com (=?UTF-8?B?0K7RgNC40Lkg0JrRg9GJ?=) Date: Tue, 15 Sep 2015 15:16:45 +0300 Subject: Getting ready In-Reply-To: References: Message-ID: Thanks! Really appreciate for your help. I'll take a look at documentation and provided links in order to start contributing. It will be great to update the links, I think that it could confuse newbies like me :) 2015-09-15 12:48 GMT+03:00 Volker Simonis : > Hi Yuri, > > that page is terribly outdated :( > > You can find the current HS development repositories under: > > http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/ > > for the JIT compiler team and: > > http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/ > > for the GC and Runtime team. > > The consolidated changes can be found in: > > http://hg.openjdk.java.net/jdk9/hs/hotspot/ > > You may also want to have a look at the HotSpot Wiki which has a > little more information: > > https://wiki.openjdk.java.net/display/HotSpot/Main > > Regards, > Volker > > > On Tue, Sep 15, 2015 at 9:36 AM, ???? ??? wrote: > > Hello all, > > > > I'm going through group's documentation ( > > http://openjdk.java.net/groups/hotspot/) and I'm not sure where should I > > start as for help. Browsable source has broken link > > http://hg.openjdk.java.net/jdk7/hotspot/hotspot/ > > > > Probably you have some advises for me? > > > > Thanks in advance! > From kirk at kodewerk.com Tue Sep 15 12:20:50 2015 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Tue, 15 Sep 2015 14:20:50 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> Message-ID: <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> > > Just for completeness: there can be more than 3 bracket pairs if you configure additional decorations: > > $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello > [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info ][logging] Log configuration fully initialized. Are there a finite number of decorations? Is the order fixed or how would one tell the diff between pid and tid? Kind regards, Kirk From staffan.larsen at oracle.com Tue Sep 15 12:27:24 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 15 Sep 2015 14:27:24 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> Message-ID: <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> > On 15 sep 2015, at 14:20, Kirk Pepperdine wrote: > > >> >> Just for completeness: there can be more than 3 bracket pairs if you configure additional decorations: >> >> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >> [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info ][logging] Log configuration fully initialized. > > Are there a finite number of decorations? Is the order fixed or how would one tell the diff between pid and tid? May I refer to the JEP: ------------ Decorations Logging messages are decorated with information about the message. Here is a list of the possible decorations: ? time -- Current time and date in ISO-8601 format ? uptime -- Time since the start of the JVM in seconds and milliseconds (e.g., 6.567s) ? timemillis -- The same value as generated by System.currentTimeMillis() ? uptimemillis -- Milliseconds since the JVM started ? timenanos -- The same value as generated by System.nanoTime() ? uptimenanos -- Nanoseconds since the JVM started ? pid -- The process identifier ? tid -- The thread identifier ? level -- The level associated with the log message ? tags -- The tag-set associated with the log message Each output can be configured to use a custom set of decorators. The order of them is always the one above though. The decorations to be used can be configured by the user in runtime. Decorations will be prepended to the log message Example: [6.567s][info][gc,old] Old collection complete ------------ From staffan.larsen at oracle.com Tue Sep 15 12:46:19 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 15 Sep 2015 14:46:19 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> Message-ID: <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> Some way of differentiating the tid and pid decorations would be good. Even if you specify them in a different order on the command line, they will always show up in the same order which can be even more confusing. Maybe prefix them with ?p? and ?t?: [2015-09-15T10:08:40.407-0100][0.203s][p91539][t4867][info ][logging] Log configuration fully initialized. Or postfix: [2015-09-15T10:08:40.407-0100][0.203s][91539p][4867t][info ][logging] Log configuration fully initialized. Or label (but this is getting verbose): [2015-09-15T10:08:40.407-0100][0.203s][pid=91539][tid=4867][info ][logging] Log configuration fully initialized. Other ideas? > On 15 sep 2015, at 14:27, Staffan Larsen wrote: > > >> On 15 sep 2015, at 14:20, Kirk Pepperdine wrote: >> >> >>> >>> Just for completeness: there can be more than 3 bracket pairs if you configure additional decorations: >>> >>> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >>> [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info ][logging] Log configuration fully initialized. >> >> Are there a finite number of decorations? Is the order fixed or how would one tell the diff between pid and tid? > > May I refer to the JEP: > > ------------ > Decorations > > Logging messages are decorated with information about the message. Here is a list of the possible decorations: > > ? time -- Current time and date in ISO-8601 format > ? uptime -- Time since the start of the JVM in seconds and milliseconds (e.g., 6.567s) > ? timemillis -- The same value as generated by System.currentTimeMillis() > ? uptimemillis -- Milliseconds since the JVM started > ? timenanos -- The same value as generated by System.nanoTime() > ? uptimenanos -- Nanoseconds since the JVM started > ? pid -- The process identifier > ? tid -- The thread identifier > ? level -- The level associated with the log message > ? tags -- The tag-set associated with the log message > Each output can be configured to use a custom set of decorators. The order of them is always the one above though. The decorations to be used can be configured by the user in runtime. Decorations will be prepended to the log message > > Example: [6.567s][info][gc,old] Old collection complete > > ------------ > > From volker.simonis at gmail.com Tue Sep 15 12:50:55 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 15 Sep 2015 14:50:55 +0200 Subject: Getting ready In-Reply-To: References: Message-ID: Your welcome! The problem is that the web pages at http://openjdk.java.net/ can only be edited by people from Oracle which have special permissions (not very scalable setup :( The Wiki pages can at least be edited by members of the corresponding groups :) On Tue, Sep 15, 2015 at 2:16 PM, ???? ??? wrote: > Thanks! Really appreciate for your help. I'll take a look at documentation > and provided links in order to start contributing. It will be great to > update the links, I think that it could confuse newbies like me :) > > > 2015-09-15 12:48 GMT+03:00 Volker Simonis : >> >> Hi Yuri, >> >> that page is terribly outdated :( >> >> You can find the current HS development repositories under: >> >> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/ >> >> for the JIT compiler team and: >> >> http://hg.openjdk.java.net/jdk9/hs-rt/hotspot/ >> >> for the GC and Runtime team. >> >> The consolidated changes can be found in: >> >> http://hg.openjdk.java.net/jdk9/hs/hotspot/ >> >> You may also want to have a look at the HotSpot Wiki which has a >> little more information: >> >> https://wiki.openjdk.java.net/display/HotSpot/Main >> >> Regards, >> Volker >> >> >> On Tue, Sep 15, 2015 at 9:36 AM, ???? ??? wrote: >> > Hello all, >> > >> > I'm going through group's documentation ( >> > http://openjdk.java.net/groups/hotspot/) and I'm not sure where should I >> > start as for help. Browsable source has broken link >> > http://hg.openjdk.java.net/jdk7/hotspot/hotspot/ >> > >> > Probably you have some advises for me? >> > >> > Thanks in advance! > > From kirk.pepperdine at gmail.com Tue Sep 15 13:06:55 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Tue, 15 Sep 2015 15:06:55 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> Message-ID: Hi Staffan, As long as the order is well known and remains fixed, all would be good without any of the labels. The potential problem is in the 5 time fields that maybe printed back to back or may be missing (not configured) each with the single initial u t u t u. Kind regards, Kirk > On Sep 15, 2015, at 2:46 PM, Staffan Larsen wrote: > > > Some way of differentiating the tid and pid decorations would be good. Even if you specify them in a different order on the command line, they will always show up in the same order which can be even more confusing. > > Maybe prefix them with ?p? and ?t?: > > [2015-09-15T10:08:40.407-0100][0.203s][p91539][t4867][info ][logging] Log configuration fully initialized. > > Or postfix: > > [2015-09-15T10:08:40.407-0100][0.203s][91539p][4867t][info ][logging] Log configuration fully initialized. > > Or label (but this is getting verbose): > > [2015-09-15T10:08:40.407-0100][0.203s][pid=91539][tid=4867][info ][logging] Log configuration fully initialized. > > Other ideas? > > > >> On 15 sep 2015, at 14:27, Staffan Larsen wrote: >> >> >>> On 15 sep 2015, at 14:20, Kirk Pepperdine wrote: >>> >>> >>>> >>>> Just for completeness: there can be more than 3 bracket pairs if you configure additional decorations: >>>> >>>> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >>>> [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info ][logging] Log configuration fully initialized. >>> >>> Are there a finite number of decorations? Is the order fixed or how would one tell the diff between pid and tid? >> >> May I refer to the JEP: >> >> ------------ >> Decorations >> >> Logging messages are decorated with information about the message. Here is a list of the possible decorations: >> >> ? time -- Current time and date in ISO-8601 format >> ? uptime -- Time since the start of the JVM in seconds and milliseconds (e.g., 6.567s) >> ? timemillis -- The same value as generated by System.currentTimeMillis() >> ? uptimemillis -- Milliseconds since the JVM started >> ? timenanos -- The same value as generated by System.nanoTime() >> ? uptimenanos -- Nanoseconds since the JVM started >> ? pid -- The process identifier >> ? tid -- The thread identifier >> ? level -- The level associated with the log message >> ? tags -- The tag-set associated with the log message >> Each output can be configured to use a custom set of decorators. The order of them is always the one above though. The decorations to be used can be configured by the user in runtime. Decorations will be prepended to the log message >> >> Example: [6.567s][info][gc,old] Old collection complete >> >> ------------ >> >> > From cnewland at chrisnewland.com Tue Sep 15 13:47:16 2015 From: cnewland at chrisnewland.com (Chris Newland) Date: Tue, 15 Sep 2015 14:47:16 +0100 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> Message-ID: <0fed0bdb35f81457858dcbcb09a18cc5.squirrel@excalibur.xssl.net> Rather than decorating the values, would it be so bad to print empty brackets [] when a log field is ommitted? Wastes 2 chars per ommitted field but reduces ambiguity and simplifies parsing? -Chris On Tue, September 15, 2015 13:46, Staffan Larsen wrote: > > Some way of differentiating the tid and pid decorations would be good. > Even if you specify them in a different order on the command line, they > will always show up in the same order which can be even more confusing. > > Maybe prefix them with ???p??? and ???t???: > > > [2015-09-15T10:08:40.407-0100][0.203s][p91539][t4867][info ][logging] > Log configuration fully initialized. > > > Or postfix: > > > [2015-09-15T10:08:40.407-0100][0.203s][91539p][4867t][info ][logging] > Log configuration fully initialized. > > > Or label (but this is getting verbose): > > > [2015-09-15T10:08:40.407-0100][0.203s][pid=91539][tid=4867][info > ][logging] Log configuration fully initialized. > > > Other ideas? > > > > >> On 15 sep 2015, at 14:27, Staffan Larsen >> wrote: >> >> >> >>> On 15 sep 2015, at 14:20, Kirk Pepperdine wrote: >>> >>> >>> >>>> >>>> Just for completeness: there can be more than 3 bracket pairs if >>>> you configure additional decorations: >>>> >>>> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >>>> [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info >>>> ][logging] Log configuration fully initialized. >>>> >>> >>> Are there a finite number of decorations? Is the order fixed or how >>> would one tell the diff between pid and tid? >> >> May I refer to the JEP: >> >> >> ------------ >> Decorations >> >> >> Logging messages are decorated with information about the message. Here >> is a list of the possible decorations: >> >> ??? time -- Current time and date in ISO-8601 format ??? uptime -- Time >> since the start of the JVM in seconds and milliseconds (e.g., 6.567s) >> ??? timemillis -- The same value as generated by >> System.currentTimeMillis() >> ??? uptimemillis -- Milliseconds since the JVM started ??? timenanos -- >> The same value as generated by System.nanoTime() >> ??? uptimenanos -- Nanoseconds since the JVM started ??? pid -- The >> process identifier ??? tid -- The thread identifier ??? level -- The >> level associated with the log message ??? tags -- The tag-set associated >> with the log message Each output can be configured to use a custom set >> of decorators. The order of them is always the one above though. The >> decorations to be used can be configured by the user in runtime. >> Decorations will be prepended to the log message >> >> >> Example: [6.567s][info][gc,old] Old collection complete >> >> >> ------------ >> >> >> > > From staffan.larsen at oracle.com Tue Sep 15 13:57:16 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Tue, 15 Sep 2015 15:57:16 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <0fed0bdb35f81457858dcbcb09a18cc5.squirrel@excalibur.xssl.net> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> <0fed0bdb35f81457858dcbcb09a18cc5.squirrel@excalibur.xssl.net> Message-ID: <1A9BDE02-6DEC-461E-836F-1E3E0FECAF5F@oracle.com> Like so: $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello [2015-09-15T10:08:40.407-0100][0.203s][][][][][91539][4867][info ][logging] Log configuration fully initialized. or with default decorations: $ java -Xlog:all Hello [][0.203s][][][][][][][info ][logging] Log configuration fully initialized. Looks a bit ugly to me. > On 15 sep 2015, at 15:47, Chris Newland wrote: > > Rather than decorating the values, would it be so bad to print empty > brackets [] when a log field is ommitted? > > Wastes 2 chars per ommitted field but reduces ambiguity and simplifies > parsing? > > -Chris > > On Tue, September 15, 2015 13:46, Staffan Larsen wrote: >> > >> Some way of differentiating the tid and pid decorations would be good. >> Even if you specify them in a different order on the command line, they >> will always show up in the same order which can be even more confusing. >> >> Maybe prefix them with ???p??? and ???t???: >> >> >> [2015-09-15T10:08:40.407-0100][0.203s][p91539][t4867][info ][logging] >> Log configuration fully initialized. >> >> >> Or postfix: >> >> >> [2015-09-15T10:08:40.407-0100][0.203s][91539p][4867t][info ][logging] >> Log configuration fully initialized. >> >> >> Or label (but this is getting verbose): >> >> >> [2015-09-15T10:08:40.407-0100][0.203s][pid=91539][tid=4867][info >> ][logging] Log configuration fully initialized. >> >> >> Other ideas? >> >> >> >> >>> On 15 sep 2015, at 14:27, Staffan Larsen >>> wrote: >>> >>> >>> >>>> On 15 sep 2015, at 14:20, Kirk Pepperdine wrote: >>>> >>>> >>>> >>>>> >>>>> Just for completeness: there can be more than 3 bracket pairs if >>>>> you configure additional decorations: >>>>> >>>>> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >>>>> [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info >>>>> ][logging] Log configuration fully initialized. >>>>> >>>> >>>> Are there a finite number of decorations? Is the order fixed or how >>>> would one tell the diff between pid and tid? >>> >>> May I refer to the JEP: >>> >>> >>> ------------ >>> Decorations >>> >>> >>> Logging messages are decorated with information about the message. Here >>> is a list of the possible decorations: >>> >>> ??? time -- Current time and date in ISO-8601 format ??? uptime -- Time >>> since the start of the JVM in seconds and milliseconds (e.g., 6.567s) >>> ??? timemillis -- The same value as generated by >>> System.currentTimeMillis() >>> ??? uptimemillis -- Milliseconds since the JVM started ??? timenanos -- >>> The same value as generated by System.nanoTime() >>> ??? uptimenanos -- Nanoseconds since the JVM started ??? pid -- The >>> process identifier ??? tid -- The thread identifier ??? level -- The >>> level associated with the log message ??? tags -- The tag-set associated >>> with the log message Each output can be configured to use a custom set >>> of decorators. The order of them is always the one above though. The >>> decorations to be used can be configured by the user in runtime. >>> Decorations will be prepended to the log message >>> >>> >>> Example: [6.567s][info][gc,old] Old collection complete >>> >>> >>> ------------ >>> >>> >>> >> >> > > From cnewland at chrisnewland.com Tue Sep 15 14:03:10 2015 From: cnewland at chrisnewland.com (Chris Newland) Date: Tue, 15 Sep 2015 15:03:10 +0100 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <1A9BDE02-6DEC-461E-836F-1E3E0FECAF5F@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> <0fed0bdb35f81457858dcbcb09a18cc5.squirrel@excalibur.xssl.net> <1A9BDE02-6DEC-461E-836F-1E3E0FECAF5F@oracle.com> Message-ID: <8d1b14ae3127d29dd27d444c25e7c820.squirrel@excalibur.xssl.net> Hi Staffan, I'd expect most logs to be read by tools so unambiguous beats ugly for me. [] could be replaced by a single char for neatness? .[0.203s]......[info ][logging] Log configuration fully initialized. -Chris On Tue, September 15, 2015 14:57, Staffan Larsen wrote: > Like so: > > > $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello > [2015-09-15T10:08:40.407-0100][0.203s][][][][][91539][4867][info > ][logging] Log configuration fully initialized. > > > or with default decorations: > > $ java -Xlog:all Hello > [][0.203s][][][][][][][info ][logging] Log configuration fully > initialized. > > Looks a bit ugly to me. > > > >> On 15 sep 2015, at 15:47, Chris Newland >> wrote: >> >> >> Rather than decorating the values, would it be so bad to print empty >> brackets [] when a log field is ommitted? >> >> Wastes 2 chars per ommitted field but reduces ambiguity and simplifies >> parsing? >> >> -Chris >> >> >> On Tue, September 15, 2015 13:46, Staffan Larsen wrote: >> >>> >> >>> Some way of differentiating the tid and pid decorations would be >>> good. Even if you specify them in a different order on the command >>> line, they will always show up in the same order which can be even >>> more confusing. >>> >>> Maybe prefix them with ???p??? and ???t???: >>> >>> >>> >>> [2015-09-15T10:08:40.407-0100][0.203s][p91539][t4867][info >>> ][logging] >>> Log configuration fully initialized. >>> >>> >>> >>> Or postfix: >>> >>> >>> >>> [2015-09-15T10:08:40.407-0100][0.203s][91539p][4867t][info >>> ][logging] >>> Log configuration fully initialized. >>> >>> >>> >>> Or label (but this is getting verbose): >>> >>> >>> >>> [2015-09-15T10:08:40.407-0100][0.203s][pid=91539][tid=4867][info >>> ][logging] Log configuration fully initialized. >>> >>> >>> >>> Other ideas? >>> >>> >>> >>> >>> >>>> On 15 sep 2015, at 14:27, Staffan Larsen >>>> >>>> wrote: >>>> >>>> >>>> >>>> >>>>> On 15 sep 2015, at 14:20, Kirk Pepperdine >>>>> wrote: >>>>> >>>>> >>>>> >>>>> >>>>>> >>>>>> Just for completeness: there can be more than 3 bracket pairs >>>>>> if you configure additional decorations: >>>>>> >>>>>> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >>>>>> [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info >>>>>> ][logging] Log configuration fully initialized. >>>>>> >>>>>> >>>>> >>>>> Are there a finite number of decorations? Is the order fixed or >>>>> how would one tell the diff between pid and tid? >>>> >>>> May I refer to the JEP: >>>> >>>> >>>> >>>> ------------ >>>> Decorations >>>> >>>> >>>> >>>> Logging messages are decorated with information about the message. >>>> Here >>>> is a list of the possible decorations: >>>> >>>> ??? time -- Current time and date in ISO-8601 format ??? uptime -- >>>> Time >>>> since the start of the JVM in seconds and milliseconds (e.g., >>>> 6.567s) >>>> ??? timemillis -- The same value as generated by >>>> System.currentTimeMillis() >>>> ??? uptimemillis -- Milliseconds since the JVM started ??? timenanos >>>> -- >>>> The same value as generated by System.nanoTime() >>>> ??? uptimenanos -- Nanoseconds since the JVM started ??? pid -- The >>>> process identifier ??? tid -- The thread identifier ??? level -- >>>> The >>>> level associated with the log message ??? tags -- The tag-set >>>> associated with the log message Each output can be configured to use >>>> a custom set of decorators. The order of them is always the one >>>> above though. The decorations to be used can be configured by the >>>> user in runtime. Decorations will be prepended to the log message >>>> >>>> >>>> >>>> Example: [6.567s][info][gc,old] Old collection complete >>>> >>>> >>>> >>>> ------------ >>>> >>>> >>>> >>>> >>> >>> >> >> > > From volker.simonis at gmail.com Tue Sep 15 14:07:15 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 15 Sep 2015 16:07:15 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> Message-ID: Hi Christian, can you please merge the following changes into your code: http://cr.openjdk.java.net/~simonis/webrevs/2015/JVMCI_hs_ppc/ They fix the build on our ppc64 platforms (Linux and AIX). Of course there's still a lot to do in order to implement the full interface for PowerPC (currently only the two options tests pass and all the runtime tests fail as expected), but this is a first step :) My webrev contains only one shared change in jvmciRuntime.cpp which is necessary to fix a warning because unreachable code introduced by the CHECK_NULL macro in a return statement. Finally, I also want to ask if it is expected that the current C1/C2 JIT compilers and/or the standard JDK class will eventually depend on the JVMCI library or will the JVMCI remain an optional component like for example the SA agent. Thank you and best regards, Volker PS: are there any JVMCI documents/presentation available other than the JEP description? On Mon, Sep 14, 2015 at 9:24 AM, Christian Thalinger wrote: > The JEP itself can be found here: > > https://bugs.openjdk.java.net/browse/JDK-8062493 > > Here are the webrevs: > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ > > The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: > > https://bugs.openjdk.java.net/browse/JDK-8134994 > > SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. > > The integration will happen under the bug number: > > https://bugs.openjdk.java.net/browse/JDK-8136421 > From christian.thalinger at oracle.com Tue Sep 15 16:26:29 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 15 Sep 2015 06:26:29 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> Message-ID: <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> > On Sep 15, 2015, at 4:07 AM, Volker Simonis wrote: > > Hi Christian, > > can you please merge the following changes into your code: > > http://cr.openjdk.java.net/~simonis/webrevs/2015/JVMCI_hs_ppc/ > > They fix the build on our ppc64 platforms (Linux and AIX). http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/93deccb356cc > Of course > there's still a lot to do in order to implement the full interface for > PowerPC (currently only the two options tests pass and all the runtime > tests fail as expected), but this is a first step :) Sure :-) > > My webrev contains only one shared change in jvmciRuntime.cpp which is > necessary to fix a warning because unreachable code introduced by the > CHECK_NULL macro in a return statement. Ahh, thanks for catching that one. Pushed: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/3ae34e1ac57e > > Finally, I also want to ask if it is expected that the current C1/C2 > JIT compilers and/or the standard JDK class will eventually depend on > the JVMCI library or will the JVMCI remain an optional component like > for example the SA agent. It?s very unlikely that we will retrofit C1 or C2 to the JVMCI. For JDK 9 the JVMCI is an experimental feature and at this point we we don?t know if and when it will become a product feature. > > Thank you and best regards, > Volker > > PS: are there any JVMCI documents/presentation available other than > the JEP description? There is an internal JVMCI Design Document. Let me see if I can put this somewhere publicly available. > > > On Mon, Sep 14, 2015 at 9:24 AM, Christian Thalinger > wrote: >> The JEP itself can be found here: >> >> https://bugs.openjdk.java.net/browse/JDK-8062493 >> >> Here are the webrevs: >> >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >> >> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >> >> https://bugs.openjdk.java.net/browse/JDK-8134994 >> >> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >> >> The integration will happen under the bug number: >> >> https://bugs.openjdk.java.net/browse/JDK-8136421 >> From vladimir.kozlov at oracle.com Tue Sep 15 17:05:22 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 15 Sep 2015 10:05:22 -0700 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> Message-ID: <55F84FD2.80203@oracle.com> I updated JVMCI/webrev.hotspot/ in place with Volker's changes. Thanks, Vladimir On 9/15/15 9:26 AM, Christian Thalinger wrote: > >> On Sep 15, 2015, at 4:07 AM, Volker Simonis wrote: >> >> Hi Christian, >> >> can you please merge the following changes into your code: >> >> http://cr.openjdk.java.net/~simonis/webrevs/2015/JVMCI_hs_ppc/ >> >> They fix the build on our ppc64 platforms (Linux and AIX). > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/93deccb356cc > >> Of course >> there's still a lot to do in order to implement the full interface for >> PowerPC (currently only the two options tests pass and all the runtime >> tests fail as expected), but this is a first step :) > > Sure :-) > >> >> My webrev contains only one shared change in jvmciRuntime.cpp which is >> necessary to fix a warning because unreachable code introduced by the >> CHECK_NULL macro in a return statement. > > Ahh, thanks for catching that one. Pushed: > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/3ae34e1ac57e > >> >> Finally, I also want to ask if it is expected that the current C1/C2 >> JIT compilers and/or the standard JDK class will eventually depend on >> the JVMCI library or will the JVMCI remain an optional component like >> for example the SA agent. > > It?s very unlikely that we will retrofit C1 or C2 to the JVMCI. For JDK 9 the JVMCI is an experimental feature and at this point we we don?t know if and when it will become a product feature. > >> >> Thank you and best regards, >> Volker >> >> PS: are there any JVMCI documents/presentation available other than >> the JEP description? > > There is an internal JVMCI Design Document. Let me see if I can put this somewhere publicly available. > >> >> >> On Mon, Sep 14, 2015 at 9:24 AM, Christian Thalinger >> wrote: >>> The JEP itself can be found here: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>> >>> Here are the webrevs: >>> >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>> >>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>> >>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>> >>> The integration will happen under the bug number: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>> > From christian.thalinger at oracle.com Tue Sep 15 17:33:39 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 15 Sep 2015 07:33:39 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55F84FD2.80203@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> <55F84FD2.80203@oracle.com> Message-ID: <87E3E7CF-75EF-4FE6-B854-FFFD8B6D6355@oracle.com> One thing we missed is that the JVMCI only works on 64-bit x86. I just pushed a change to exclude it in 32-bit builds: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/4ce8a718a4c2 > On Sep 15, 2015, at 7:05 AM, Vladimir Kozlov wrote: > > I updated JVMCI/webrev.hotspot/ in place with Volker's changes. > > Thanks, > Vladimir > > On 9/15/15 9:26 AM, Christian Thalinger wrote: >> >>> On Sep 15, 2015, at 4:07 AM, Volker Simonis wrote: >>> >>> Hi Christian, >>> >>> can you please merge the following changes into your code: >>> >>> http://cr.openjdk.java.net/~simonis/webrevs/2015/JVMCI_hs_ppc/ >>> >>> They fix the build on our ppc64 platforms (Linux and AIX). >> >> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/93deccb356cc >> >>> Of course >>> there's still a lot to do in order to implement the full interface for >>> PowerPC (currently only the two options tests pass and all the runtime >>> tests fail as expected), but this is a first step :) >> >> Sure :-) >> >>> >>> My webrev contains only one shared change in jvmciRuntime.cpp which is >>> necessary to fix a warning because unreachable code introduced by the >>> CHECK_NULL macro in a return statement. >> >> Ahh, thanks for catching that one. Pushed: >> >> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/3ae34e1ac57e >> >>> >>> Finally, I also want to ask if it is expected that the current C1/C2 >>> JIT compilers and/or the standard JDK class will eventually depend on >>> the JVMCI library or will the JVMCI remain an optional component like >>> for example the SA agent. >> >> It?s very unlikely that we will retrofit C1 or C2 to the JVMCI. For JDK 9 the JVMCI is an experimental feature and at this point we we don?t know if and when it will become a product feature. >> >>> >>> Thank you and best regards, >>> Volker >>> >>> PS: are there any JVMCI documents/presentation available other than >>> the JEP description? >> >> There is an internal JVMCI Design Document. Let me see if I can put this somewhere publicly available. >> >>> >>> >>> On Mon, Sep 14, 2015 at 9:24 AM, Christian Thalinger >>> wrote: >>>> The JEP itself can be found here: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>> >>>> Here are the webrevs: >>>> >>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>> >>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>> >>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>> >>>> The integration will happen under the bug number: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>> >> From christian.thalinger at oracle.com Tue Sep 15 20:16:13 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 15 Sep 2015 10:16:13 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <87E3E7CF-75EF-4FE6-B854-FFFD8B6D6355@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> <55F84FD2.80203@oracle.com> <87E3E7CF-75EF-4FE6-B854-FFFD8B6D6355@oracle.com> Message-ID: <584BEA32-689A-4C85-9864-A47D80F80798@oracle.com> Here is another piece for AArch64 similar to the PPC changes: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/016cced79b40 Could anyone from RedHat verify this builds on AArch64? > On Sep 15, 2015, at 7:33 AM, Christian Thalinger wrote: > > One thing we missed is that the JVMCI only works on 64-bit x86. I just pushed a change to exclude it in 32-bit builds: > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/4ce8a718a4c2 > >> On Sep 15, 2015, at 7:05 AM, Vladimir Kozlov wrote: >> >> I updated JVMCI/webrev.hotspot/ in place with Volker's changes. >> >> Thanks, >> Vladimir >> >> On 9/15/15 9:26 AM, Christian Thalinger wrote: >>> >>>> On Sep 15, 2015, at 4:07 AM, Volker Simonis wrote: >>>> >>>> Hi Christian, >>>> >>>> can you please merge the following changes into your code: >>>> >>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/JVMCI_hs_ppc/ >>>> >>>> They fix the build on our ppc64 platforms (Linux and AIX). >>> >>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/93deccb356cc >>> >>>> Of course >>>> there's still a lot to do in order to implement the full interface for >>>> PowerPC (currently only the two options tests pass and all the runtime >>>> tests fail as expected), but this is a first step :) >>> >>> Sure :-) >>> >>>> >>>> My webrev contains only one shared change in jvmciRuntime.cpp which is >>>> necessary to fix a warning because unreachable code introduced by the >>>> CHECK_NULL macro in a return statement. >>> >>> Ahh, thanks for catching that one. Pushed: >>> >>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/3ae34e1ac57e >>> >>>> >>>> Finally, I also want to ask if it is expected that the current C1/C2 >>>> JIT compilers and/or the standard JDK class will eventually depend on >>>> the JVMCI library or will the JVMCI remain an optional component like >>>> for example the SA agent. >>> >>> It?s very unlikely that we will retrofit C1 or C2 to the JVMCI. For JDK 9 the JVMCI is an experimental feature and at this point we we don?t know if and when it will become a product feature. >>> >>>> >>>> Thank you and best regards, >>>> Volker >>>> >>>> PS: are there any JVMCI documents/presentation available other than >>>> the JEP description? >>> >>> There is an internal JVMCI Design Document. Let me see if I can put this somewhere publicly available. >>> >>>> >>>> >>>> On Mon, Sep 14, 2015 at 9:24 AM, Christian Thalinger >>>> wrote: >>>>> The JEP itself can be found here: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>> >>>>> Here are the webrevs: >>>>> >>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>> >>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>> >>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>> >>>>> The integration will happen under the bug number: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>> >>> > From vladimir.kozlov at oracle.com Tue Sep 15 23:44:31 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 15 Sep 2015 16:44:31 -0700 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <584BEA32-689A-4C85-9864-A47D80F80798@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> <55F84FD2.80203@oracle.com> <87E3E7CF-75EF-4FE6-B854-FFFD8B6D6355@oracle.com> <584BEA32-689A-4C85-9864-A47D80F80798@oracle.com> Message-ID: <55F8AD5F.1040206@oracle.com> I updated hotspot webrev with these changes. Vladimir On 9/15/15 1:16 PM, Christian Thalinger wrote: > Here is another piece for AArch64 similar to the PPC changes: > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/016cced79b40 > > Could anyone from RedHat verify this builds on AArch64? > >> On Sep 15, 2015, at 7:33 AM, Christian Thalinger wrote: >> >> One thing we missed is that the JVMCI only works on 64-bit x86. I just pushed a change to exclude it in 32-bit builds: >> >> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/4ce8a718a4c2 >> >>> On Sep 15, 2015, at 7:05 AM, Vladimir Kozlov wrote: >>> >>> I updated JVMCI/webrev.hotspot/ in place with Volker's changes. >>> >>> Thanks, >>> Vladimir >>> >>> On 9/15/15 9:26 AM, Christian Thalinger wrote: >>>> >>>>> On Sep 15, 2015, at 4:07 AM, Volker Simonis wrote: >>>>> >>>>> Hi Christian, >>>>> >>>>> can you please merge the following changes into your code: >>>>> >>>>> http://cr.openjdk.java.net/~simonis/webrevs/2015/JVMCI_hs_ppc/ >>>>> >>>>> They fix the build on our ppc64 platforms (Linux and AIX). >>>> >>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/93deccb356cc >>>> >>>>> Of course >>>>> there's still a lot to do in order to implement the full interface for >>>>> PowerPC (currently only the two options tests pass and all the runtime >>>>> tests fail as expected), but this is a first step :) >>>> >>>> Sure :-) >>>> >>>>> >>>>> My webrev contains only one shared change in jvmciRuntime.cpp which is >>>>> necessary to fix a warning because unreachable code introduced by the >>>>> CHECK_NULL macro in a return statement. >>>> >>>> Ahh, thanks for catching that one. Pushed: >>>> >>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/3ae34e1ac57e >>>> >>>>> >>>>> Finally, I also want to ask if it is expected that the current C1/C2 >>>>> JIT compilers and/or the standard JDK class will eventually depend on >>>>> the JVMCI library or will the JVMCI remain an optional component like >>>>> for example the SA agent. >>>> >>>> It?s very unlikely that we will retrofit C1 or C2 to the JVMCI. For JDK 9 the JVMCI is an experimental feature and at this point we we don?t know if and when it will become a product feature. >>>> >>>>> >>>>> Thank you and best regards, >>>>> Volker >>>>> >>>>> PS: are there any JVMCI documents/presentation available other than >>>>> the JEP description? >>>> >>>> There is an internal JVMCI Design Document. Let me see if I can put this somewhere publicly available. >>>> >>>>> >>>>> >>>>> On Mon, Sep 14, 2015 at 9:24 AM, Christian Thalinger >>>>> wrote: >>>>>> The JEP itself can be found here: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>> >>>>>> Here are the webrevs: >>>>>> >>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>> >>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>> >>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>> >>>>>> The integration will happen under the bug number: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>> >>>> >> > From magnus.ihse.bursie at oracle.com Wed Sep 16 12:57:00 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 16 Sep 2015 14:57:00 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> Message-ID: <55F9671C.4000208@oracle.com> On 2015-09-14 09:24, Christian Thalinger wrote: > The JEP itself can be found here: > > https://bugs.openjdk.java.net/browse/JDK-8062493 > > Here are the webrevs: > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ > > The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: > > https://bugs.openjdk.java.net/browse/JDK-8134994 > > SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. > > The integration will happen under the bug number: > > https://bugs.openjdk.java.net/browse/JDK-8136421 > Hi Christian, (Adding build-dev since this review includes some major build changes.) The makefile changes looks good in general to me. I have not reviewed the source code changes. Some comments: * modules.xml: Make sure you get sign-off from a Jigsaw developer for modifying this file. * hotspot/make/linux/makefiles/gcc.make: Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. /Magnus From volker.simonis at gmail.com Wed Sep 16 13:29:16 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 16 Sep 2015 15:29:16 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <49D334C6-516B-4589-811C-4D2C67DF0045@oracle.com> Message-ID: Hi Christian, thanks for including my fixes. Making the "JVMCI Design Document" available (maybe from within the JEP entry in the bug system) would be very much appreciated. Regards, Volker On Tue, Sep 15, 2015 at 6:26 PM, Christian Thalinger wrote: > > On Sep 15, 2015, at 4:07 AM, Volker Simonis > wrote: > > Hi Christian, > > can you please merge the following changes into your code: > > http://cr.openjdk.java.net/~simonis/webrevs/2015/JVMCI_hs_ppc/ > > They fix the build on our ppc64 platforms (Linux and AIX). > > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/93deccb356cc > > Of course > there's still a lot to do in order to implement the full interface for > PowerPC (currently only the two options tests pass and all the runtime > tests fail as expected), but this is a first step :) > > > Sure :-) > > > My webrev contains only one shared change in jvmciRuntime.cpp which is > necessary to fix a warning because unreachable code introduced by the > CHECK_NULL macro in a return statement. > > > Ahh, thanks for catching that one. Pushed: > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/3ae34e1ac57e > > > Finally, I also want to ask if it is expected that the current C1/C2 > JIT compilers and/or the standard JDK class will eventually depend on > the JVMCI library or will the JVMCI remain an optional component like > for example the SA agent. > > > It?s very unlikely that we will retrofit C1 or C2 to the JVMCI. For JDK 9 > the JVMCI is an experimental feature and at this point we we don?t know if > and when it will become a product feature. > > > Thank you and best regards, > Volker > > PS: are there any JVMCI documents/presentation available other than > the JEP description? > > > There is an internal JVMCI Design Document. Let me see if I can put this > somewhere publicly available. > > > > On Mon, Sep 14, 2015 at 9:24 AM, Christian Thalinger > wrote: > > The JEP itself can be found here: > > https://bugs.openjdk.java.net/browse/JDK-8062493 > > > Here are the webrevs: > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ > > > The change has already undergone a few iterations of internal review by > different people of different teams. Most comments and suggestions were > handled accordingly. Although there is one open item we agreed we will > address after the integration of JEP 243 and that work is captured in RFE: > > https://bugs.openjdk.java.net/browse/JDK-8134994 > > > SQE is still working on the tests and all test tasks can be seen as > sub-tasks of the JEP. > > The integration will happen under the bug number: > > https://bugs.openjdk.java.net/browse/JDK-8136421 > > > From christian.thalinger at oracle.com Wed Sep 16 16:52:01 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 16 Sep 2015 06:52:01 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55F9671C.4000208@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> Message-ID: > On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: > > On 2015-09-14 09:24, Christian Thalinger wrote: >> The JEP itself can be found here: >> >> https://bugs.openjdk.java.net/browse/JDK-8062493 >> >> Here are the webrevs: >> >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >> >> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >> >> https://bugs.openjdk.java.net/browse/JDK-8134994 >> >> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >> >> The integration will happen under the bug number: >> >> https://bugs.openjdk.java.net/browse/JDK-8136421 >> > Hi Christian, > > (Adding build-dev since this review includes some major build changes.) > > The makefile changes looks good in general to me. I have not reviewed the source code changes. > > Some comments: > > * modules.xml: > Make sure you get sign-off from a Jigsaw developer for modifying this file. I?ve asked Alan to take a look. > > * hotspot/make/linux/makefiles/gcc.make: > Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: ifeq ($(USE_CLANG), true) > > * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: > I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. Thanks for the heads up. > > Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. Ok, thanks. > > /Magnus From Alan.Bateman at oracle.com Wed Sep 16 16:52:37 2015 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 16 Sep 2015 17:52:37 +0100 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55F9671C.4000208@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> Message-ID: <55F99E55.9080206@oracle.com> On 16/09/2015 13:57, Magnus Ihse Bursie wrote: > : > > * modules.xml: > Make sure you get sign-off from a Jigsaw developer for modifying this > file. The update to modules.xml looks fine. -Alan. From coleen.phillimore at oracle.com Wed Sep 16 19:59:07 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 16 Sep 2015 15:59:07 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F7E6B5.2040502@oracle.com> References: <55ED9210.6030904@oracle.com> <55F7E6B5.2040502@oracle.com> Message-ID: <55F9CA0B.2030100@oracle.com> Hi Marcus, I had to have someone tell me how the templates work so that the log_is_enabled(Debug, etc) statements are efficient. My initial concern was that it would be expensive to test for logging in place of some runtime Trace flags which evaluate to constant 'false' in product mode. If this still turns out to be a problem, we may ask for an RFE to add a flag_is_enabled(Develop, ) or similar macro that evaluates to a constant false in product mode. Also, I think it might be helpful to have more than one Develop level if we want certain logging flags to not be available in product mode, ie DevelopInfo, DevelopDebug and DevelopTrace. But we can file RFEs for things we may need while doing the runtime logging. In http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.hpp.html Can you add to the comment at 103 that explains the magic that instantiates _tagset to be the static variable that holds the settings for each combination of tags found in the source code? So that is_level is only a static test. And here: http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.cpp.html The constructor for LogTagSet is a static constructor that links all the static combinations of tags settings together so that they can be initialized with the values for the logging options on the command line. I don't need to see the comments. I think this change is ready for integration. Thanks, Coleen On 9/15/15 5:36 AM, Marcus Larsson wrote: > Hi, > > Here is the updated version of the patch, including the feedback and > suggestions from Coleen, David and Ioi. > > Incremental webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ > > Full webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ > > Changes include: > * New NMT tag 'mtLogging' used by UL > * Added storage allocation classes for all new classes > * Added Log::puts(const char*) for plain string messages of any > length > * Fixed broken printf-attribute on Log::write functions > * Moved VM startup timestamp to after os::init > * Formatting and code style changes > > Thanks, > Marcus > > > On 2015-09-07 15:33, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch adding the unified logging >> framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. >> Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which >> contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log >> call has a corresponding LogTagSet instance, which keeps a track of >> all the outputs it should write the log message to (and their >> levels). Having tags as template arguments allows mapping directly >> from a set of tags to the LogTagSet instance, which means that the >> overhead for disabled logging should be low. Currently each log >> message can be tagged with up to 5 tags, but this can be increased if >> ever required (and with C++11's variadic templates the limit can be >> removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by >> command line arguments (-Xlog) or by DCMD. Both methods will in turn >> use the LogConfiguration class to perform the parsing & >> configuration. This configuration includes iterating over all >> LogTagSet instances and updating them accordingly. The >> LogTagLevelExpression class is used to represent the selection of >> tags and levels for a given configuration request (the >> "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances >> of this class is kept in LogTagSet to track what decorators to use >> (this is the union of all decorators used by its outputs). Each log >> call will create a LogDecorations instance (note: different classes), >> which will contain the actual decoration strings for the log message. >> These decorations are used for each output the tagset is set to log >> on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with >> argument. (This allows GC logging to prefix messages of certain >> tagsets with GCId.) Prefixes are implemented using template >> specializations based on the specified tags, with the >> general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log >> outputs. LogFileStreamOutput corresponds to FILE* stream based log >> outputs. LogFileOutput builds on this and adds the file management >> and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will >> be added at a later stage. >> >> Thanks, >> Marcus > From magnus.ihse.bursie at oracle.com Wed Sep 16 20:11:52 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 16 Sep 2015 22:11:52 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> Message-ID: <55F9CD08.8010406@oracle.com> On 2015-09-16 18:52, Christian Thalinger wrote: >> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >> >> On 2015-09-14 09:24, Christian Thalinger wrote: >>> The JEP itself can be found here: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>> >>> Here are the webrevs: >>> >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>> >>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>> >>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>> >>> The integration will happen under the bug number: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>> >> Hi Christian, >> >> (Adding build-dev since this review includes some major build changes.) >> >> The makefile changes looks good in general to me. I have not reviewed the source code changes. >> >> Some comments: >> >> * modules.xml: >> Make sure you get sign-off from a Jigsaw developer for modifying this file. > I?ve asked Alan to take a look. > >> * hotspot/make/linux/makefiles/gcc.make: >> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? > We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: > > ifeq ($(USE_CLANG), true) I'm not sure I understand why that's relevant..? Isn't it just as important to try to submit warning-free code when compiling with clang as with any other compiler? Or is clang just being anal about the code in question? /Magnus > >> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. > Thanks for the heads up. > >> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. > Ok, thanks. > >> /Magnus From christian.thalinger at oracle.com Wed Sep 16 20:25:30 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 16 Sep 2015 10:25:30 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55F9CD08.8010406@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <55F9CD08.8010406@oracle.com> Message-ID: <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> > On Sep 16, 2015, at 10:11 AM, Magnus Ihse Bursie wrote: > > On 2015-09-16 18:52, Christian Thalinger wrote: >>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>> >>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>> The JEP itself can be found here: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>> >>>> Here are the webrevs: >>>> >>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>> >>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>> >>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>> >>>> The integration will happen under the bug number: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>> >>> Hi Christian, >>> >>> (Adding build-dev since this review includes some major build changes.) >>> >>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>> >>> Some comments: >>> >>> * modules.xml: >>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >> I?ve asked Alan to take a look. >> >>> * hotspot/make/linux/makefiles/gcc.make: >>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >> >> ifeq ($(USE_CLANG), true) > > I'm not sure I understand why that's relevant..? Isn't it just as important to try to submit warning-free code when compiling with clang as with any other compiler? Or is clang just being anal about the code in question? I don?t have a Clang compiler at hand but Clang is anal about everything. Do you want that suppression to be removed? > > /Magnus > >> >>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >> Thanks for the heads up. >> >>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >> Ok, thanks. >> >>> /Magnus From christian.thalinger at oracle.com Wed Sep 16 21:28:04 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 16 Sep 2015 11:28:04 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> Message-ID: <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> > On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: > > >> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >> >> On 2015-09-14 09:24, Christian Thalinger wrote: >>> The JEP itself can be found here: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>> >>> Here are the webrevs: >>> >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>> >>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>> >>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>> >>> The integration will happen under the bug number: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>> >> Hi Christian, >> >> (Adding build-dev since this review includes some major build changes.) >> >> The makefile changes looks good in general to me. I have not reviewed the source code changes. >> >> Some comments: >> >> * modules.xml: >> Make sure you get sign-off from a Jigsaw developer for modifying this file. > > I?ve asked Alan to take a look. > >> >> * hotspot/make/linux/makefiles/gcc.make: >> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? > > We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: > > ifeq ($(USE_CLANG), true) > >> >> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. > > Thanks for the heads up. Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >> >> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. > > Ok, thanks. > >> >> /Magnus > From kirk at kodewerk.com Wed Sep 16 21:51:43 2015 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Wed, 16 Sep 2015 23:51:43 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F9CA0B.2030100@oracle.com> References: <55ED9210.6030904@oracle.com> <55F7E6B5.2040502@oracle.com> <55F9CA0B.2030100@oracle.com> Message-ID: <52003E29-009C-4364-809B-87FB7471A02C@kodewerk.com> Hi Coleen, My initial concerns with the JEP and this concern remains is that logging levels should be removed from the specification and that they be replaced with tags that make sense to the work that you are doing. Instead of seeing you define the tags you?d like, we are witnessing a proliferation of logging levels. I do apologize on bring this point up but I believe this JEP could be improved to avoid some of the problems that I keep running into with the existing logging frameworks that this work seems to want to repeat. Kind regards, Kirk Pepperdine > On Sep 16, 2015, at 9:59 PM, Coleen Phillimore wrote: > > Hi Marcus, > > I had to have someone tell me how the templates work so that the log_is_enabled(Debug, etc) statements are efficient. My initial concern was that it would be expensive to test for logging in place of some runtime Trace flags which evaluate to constant 'false' in product mode. > > If this still turns out to be a problem, we may ask for an RFE to add a flag_is_enabled(Develop, ) or similar macro that evaluates to a constant false in product mode. Also, I think it might be helpful to have more than one Develop level if we want certain logging flags to not be available in product mode, ie DevelopInfo, DevelopDebug and DevelopTrace. But we can file RFEs for things we may need while doing the runtime logging. > > In http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.hpp.html > > Can you add to the comment at 103 that explains the magic that instantiates _tagset to be the static variable that holds the settings for each combination of tags found in the source code? So that is_level is only a static test. > > And here: > > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.cpp.html > > The constructor for LogTagSet is a static constructor that links all the static combinations of tags settings together so that they can be initialized with the values for the logging options on the command line. > > I don't need to see the comments. I think this change is ready for integration. > > Thanks, > Coleen > > On 9/15/15 5:36 AM, Marcus Larsson wrote: >> Hi, >> >> Here is the updated version of the patch, including the feedback and suggestions from Coleen, David and Ioi. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ >> >> Full webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ >> >> Changes include: >> * New NMT tag 'mtLogging' used by UL >> * Added storage allocation classes for all new classes >> * Added Log::puts(const char*) for plain string messages of any length >> * Fixed broken printf-attribute on Log::write functions >> * Moved VM startup timestamp to after os::init >> * Formatting and code style changes >> >> Thanks, >> Marcus >> >> >> On 2015-09-07 15:33, Marcus Larsson wrote: >>> Hi, >>> >>> Please review the following patch adding the unified logging framework to hotspot. >>> >>> JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>> >>> See the JEP description for a general overview of the new feature. Below are some notes on the implementation. >>> >>> The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. >>> >>> Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). >>> >>> The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). >>> >>> The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. >>> >>> The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. >>> >>> The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. >>> >>> A simple jtreg test is included in this patch. Additional tests will be added at a later stage. >>> >>> Thanks, >>> Marcus >> > From coleen.phillimore at oracle.com Wed Sep 16 22:18:32 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 16 Sep 2015 18:18:32 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <52003E29-009C-4364-809B-87FB7471A02C@kodewerk.com> References: <55ED9210.6030904@oracle.com> <55F7E6B5.2040502@oracle.com> <55F9CA0B.2030100@oracle.com> <52003E29-009C-4364-809B-87FB7471A02C@kodewerk.com> Message-ID: <55F9EAB8.2090208@oracle.com> Hi Kirk, I think Max answered that we had similar concern. We'll have lots of logging tags for the runtime actions that need to be logged and maybe a sub-tag design would work better. But given the current state of the runtime code (I can't speak for GC but you'll see how GC has been adapted to this framework soon), the levels do make sense for us also. Take the case of PrintVtables - we have some code under PrintVtables and some other code under PrintVtables && Verbose. Having the levels defined (there are only 3) allows us granularity to get this output or a more verbose output by command line. We hate having to add Verbose and/or Wizard mode to get more logging information. This unified logging approach makes more sense in this context. For the most part, I see this logging as a step forward that cleans up the code that we have, but doesn't significantly change the functionality as we're adapting it. We are planning on deprecating many (most?) of the nonproduct command line options, which helps with my concern with the proliferation of options that we have to spend time and support. I think there will be ways to improve this and we can add RFE's for what we find. Again, I think this is a step in the right direction for the product. I realize that the GC log changes will cause you pain, and I'm not crazy about blanks in the log lines, but I think this will evolve to something that turns out very nice. Thanks, Coleen On 9/16/15 5:51 PM, Kirk Pepperdine wrote: > Hi Coleen, > > My initial concerns with the JEP and this concern remains is that logging levels should be removed from the specification and that they be replaced with tags that make sense to the work that you are doing. Instead of seeing you define the tags you?d like, we are witnessing a proliferation of logging levels. > > I do apologize on bring this point up but I believe this JEP could be improved to avoid some of the problems that I keep running into with the existing logging frameworks that this work seems to want to repeat. > > Kind regards, > Kirk Pepperdine > >> On Sep 16, 2015, at 9:59 PM, Coleen Phillimore wrote: >> >> Hi Marcus, >> >> I had to have someone tell me how the templates work so that the log_is_enabled(Debug, etc) statements are efficient. My initial concern was that it would be expensive to test for logging in place of some runtime Trace flags which evaluate to constant 'false' in product mode. >> >> If this still turns out to be a problem, we may ask for an RFE to add a flag_is_enabled(Develop, ) or similar macro that evaluates to a constant false in product mode. Also, I think it might be helpful to have more than one Develop level if we want certain logging flags to not be available in product mode, ie DevelopInfo, DevelopDebug and DevelopTrace. But we can file RFEs for things we may need while doing the runtime logging. >> >> In http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.hpp.html >> >> Can you add to the comment at 103 that explains the magic that instantiates _tagset to be the static variable that holds the settings for each combination of tags found in the source code? So that is_level is only a static test. >> >> And here: >> >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.cpp.html >> >> The constructor for LogTagSet is a static constructor that links all the static combinations of tags settings together so that they can be initialized with the values for the logging options on the command line. >> >> I don't need to see the comments. I think this change is ready for integration. >> >> Thanks, >> Coleen >> >> On 9/15/15 5:36 AM, Marcus Larsson wrote: >>> Hi, >>> >>> Here is the updated version of the patch, including the feedback and suggestions from Coleen, David and Ioi. >>> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ >>> >>> Changes include: >>> * New NMT tag 'mtLogging' used by UL >>> * Added storage allocation classes for all new classes >>> * Added Log::puts(const char*) for plain string messages of any length >>> * Fixed broken printf-attribute on Log::write functions >>> * Moved VM startup timestamp to after os::init >>> * Formatting and code style changes >>> >>> Thanks, >>> Marcus >>> >>> >>> On 2015-09-07 15:33, Marcus Larsson wrote: >>>> Hi, >>>> >>>> Please review the following patch adding the unified logging framework to hotspot. >>>> >>>> JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>>> >>>> See the JEP description for a general overview of the new feature. Below are some notes on the implementation. >>>> >>>> The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. >>>> >>>> Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). >>>> >>>> The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). >>>> >>>> The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. >>>> >>>> The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. >>>> >>>> The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. >>>> >>>> A simple jtreg test is included in this patch. Additional tests will be added at a later stage. >>>> >>>> Thanks, >>>> Marcus From vladimir.kozlov at oracle.com Wed Sep 16 22:43:26 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 16 Sep 2015 15:43:26 -0700 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> Message-ID: <55F9F08E.8010506@oracle.com> I updated top and hotspot webrev with latest (build) changes. Vladimir On 9/16/15 2:28 PM, Christian Thalinger wrote: > >> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >> >> >>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>> >>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>> The JEP itself can be found here: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>> >>>> Here are the webrevs: >>>> >>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>> >>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>> >>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>> >>>> The integration will happen under the bug number: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>> >>> Hi Christian, >>> >>> (Adding build-dev since this review includes some major build changes.) >>> >>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>> >>> Some comments: >>> >>> * modules.xml: >>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >> >> I?ve asked Alan to take a look. >> >>> >>> * hotspot/make/linux/makefiles/gcc.make: >>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >> >> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >> >> ifeq ($(USE_CLANG), true) >> >>> >>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >> >> Thanks for the heads up. > > Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: > > http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >> >>> >>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >> >> Ok, thanks. >> >>> >>> /Magnus >> > From christian.thalinger at oracle.com Thu Sep 17 01:05:18 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 16 Sep 2015 15:05:18 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55F9F08E.8010506@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> Message-ID: <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> I would like to add this change: diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { Abstract_VM_Version::vm_release(), Abstract_VM_Version::vm_info_string(), TieredCompilation ? ", tiered" : "", +#if INCLUDE_JVMCI + EnableJVMCI ? ", jvmci" : "", + UseJVMCICompiler ? ", jvmci compiler" : "", +#endif UseCompressedOops ? ", compressed oops" : "", gc_mode(), Abstract_VM_Version::vm_platform_string() It would be useful to see in the crash logs if this experimental feature was turned on. > On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: > > I updated top and hotspot webrev with latest (build) changes. > > Vladimir > > On 9/16/15 2:28 PM, Christian Thalinger wrote: >> >>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>> >>> >>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>> >>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>> The JEP itself can be found here: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>> >>>>> Here are the webrevs: >>>>> >>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>> >>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>> >>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>> >>>>> The integration will happen under the bug number: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>> >>>> Hi Christian, >>>> >>>> (Adding build-dev since this review includes some major build changes.) >>>> >>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>> >>>> Some comments: >>>> >>>> * modules.xml: >>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>> >>> I?ve asked Alan to take a look. >>> >>>> >>>> * hotspot/make/linux/makefiles/gcc.make: >>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>> >>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>> >>> ifeq ($(USE_CLANG), true) >>> >>>> >>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>> >>> Thanks for the heads up. >> >> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >> >> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >> >>> >>>> >>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>> >>> Ok, thanks. >>> >>>> >>>> /Magnus From magnus.ihse.bursie at oracle.com Thu Sep 17 07:24:51 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 17 Sep 2015 09:24:51 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <55F9CD08.8010406@oracle.com> <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> Message-ID: <55FA6AC3.2010609@oracle.com> On 2015-09-16 22:25, Christian Thalinger wrote: > >> On Sep 16, 2015, at 10:11 AM, Magnus Ihse Bursie >> > > wrote: >> >> On 2015-09-16 18:52, Christian Thalinger wrote: >>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie >>>> >>> > wrote: >>>> >>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>> The JEP itself can be found here: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>> >>>>> >>>>> Here are the webrevs: >>>>> >>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>> >>>>> >>>> > >>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>> >>>>> >>>> > >>>>> >>>>> The change has already undergone a few iterations of internal >>>>> review by different people of different teams. Most comments and >>>>> suggestions were handled accordingly. Although there is one open >>>>> item we agreed we will address after the integration of JEP 243 >>>>> and that work is captured in RFE: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>> >>>>> >>>>> SQE is still working on the tests and all test tasks can be seen >>>>> as sub-tasks of the JEP. >>>>> >>>>> The integration will happen under the bug number: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>> >>>>> >>>> Hi Christian, >>>> >>>> (Adding build-dev since this review includes some major build changes.) >>>> >>>> The makefile changes looks good in general to me. I have not >>>> reviewed the source code changes. >>>> >>>> Some comments: >>>> >>>> * modules.xml: >>>> Make sure you get sign-off from a Jigsaw developer for modifying >>>> this file. >>> I?ve asked Alan to take a look. >>> >>>> * hotspot/make/linux/makefiles/gcc.make: >>>> Seems unfortunate to have to disable a new warning >>>> (undefined-bool-conversion) for newly incorporated code. Is it not >>>> possible to fix the code emitting this warning instead? >>> We had this question before. It?s not obvious because you can?t see >>> it in the regular diff views but this is under: >>> >>> ifeq ($(USE_CLANG), true) >> >> I'm not sure I understand why that's relevant..? Isn't it just as >> important to try to submit warning-free code when compiling with >> clang as with any other compiler? Or is clang just being anal about >> the code in question? > > I don?t have a Clang compiler at hand but Clang is anal about > everything. Do you want that suppression to be removed? It's more a hotspot code quality issue, not a build system issue, so I won't insist. I just wanted to point out that this change will start hiding a new kind of warning for all files in hotspot. Unless there was a compelling reason, I would personally rather see an effort to fix the code in question. But if no-one from Hotspot agrees on this, I'll drop it. /Magnus From magnus.ihse.bursie at oracle.com Thu Sep 17 08:50:29 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 17 Sep 2015 10:50:29 +0200 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <1441874805.3458.11.camel@redhat.com> References: <55F0FF9D.8030309@oracle.com> <55F104AF.5070009@oracle.com> <1441874805.3458.11.camel@redhat.com> Message-ID: <55FA7ED5.7030406@oracle.com> I'm not sure this code does what you think it does. At least when building from the top-level makefile, ZERO_LIBARCH will always be set, so the "unknown" case will not be reached. Also, I'd like to hijacking this thread slightly, sorry about that. :) When rewriting the Hotspot makefiles, I've been wondering if it would not make more sense to consider "zero" to be a different architecture, rather than a different "flavor" (like minimal1), as it is treated right now. This patch seems to align with the treatment of zero as an architecture. Just as an open question, do you think that would make sense? /Magnus On 2015-09-10 10:46, Severin Gehwolf wrote: > On Thu, 2015-09-10 at 14:18 +1000, David Holmes wrote: >> Thanks Coleen! Looks good. >> >> David >> >> On 10/09/2015 1:57 PM, Coleen Phillimore wrote: >>> Summary: add zero architectures for default cases >>> Reviewed-by: coleenp, dholmes >>> Contributed-by: Matthias Klose >>> >>> Tested on linux zero and linux x64. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8135298/ >>> bug link https://bugs.openjdk.java.net/browse/JDK-8135298 >>> >>> Thanks, >>> Coleen > Thanks, Coleen, for getting this pushed! > > Cheers, > Severin > From sgehwolf at redhat.com Thu Sep 17 12:48:45 2015 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 17 Sep 2015 14:48:45 +0200 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <55FA7ED5.7030406@oracle.com> References: <55F0FF9D.8030309@oracle.com> <55F104AF.5070009@oracle.com> <1441874805.3458.11.camel@redhat.com> <55FA7ED5.7030406@oracle.com> Message-ID: <1442494125.13753.21.camel@redhat.com> Hi, On Thu, 2015-09-17 at 10:50 +0200, Magnus Ihse Bursie wrote: > I'm not sure this code does what you think it does. At least when > building from the top-level makefile, ZERO_LIBARCH will always be > set, > so the "unknown" case will not be reached. CC'ing Matthias as we're not really building on any platform not listed in platform.m4. Looking at common/autoconf/platform.m4 (line 20 onwards) where VAR_CPU seems to be defined, then used as OPENJDK_TARGET_CPU and then as OPENJDK_TARGET_CPU_LEGACY_LIB and yet again as ZERO_LIBARCH. It may be true for all the CPUs listed in platform.m4. I can only speculate, but some distributions still build on more platforms than listed there (Debian?). So if some distribution patches that code to support a specific arch, what happens then? Has this been considered when you wrote the above? > Also, I'd like to hijacking this thread slightly, sorry about that. > :) > > When rewriting the Hotspot makefiles, I've been wondering if it > would > not make more sense to consider "zero" to be a different > architecture, > rather than a different "flavor" (like minimal1), as it is treated > right > now. This patch seems to align with the treatment of zero as an > architecture. Just as an open question, do you think that would make > sense? A Zero (interpreter only) JVM has the goal to be (easily) built on many (any?) architectures. There can be Zero on x86_64, on arm32, etc. I don't think it makes sense to treat Zero as an architecture itself. To me it being a variant/flavor kind of build is correct (depending on what the definition of a variant/flavor build is exactly). Cheers, Severin > /Magnus From magnus.ihse.bursie at oracle.com Thu Sep 17 13:06:02 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 17 Sep 2015 15:06:02 +0200 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <1442494125.13753.21.camel@redhat.com> References: <55F0FF9D.8030309@oracle.com> <55F104AF.5070009@oracle.com> <1441874805.3458.11.camel@redhat.com> <55FA7ED5.7030406@oracle.com> <1442494125.13753.21.camel@redhat.com> Message-ID: <55FABABA.7070207@oracle.com> On 2015-09-17 14:48, Severin Gehwolf wrote: > Hi, > > On Thu, 2015-09-17 at 10:50 +0200, Magnus Ihse Bursie wrote: >> I'm not sure this code does what you think it does. At least when >> building from the top-level makefile, ZERO_LIBARCH will always be >> set, >> so the "unknown" case will not be reached. > CC'ing Matthias as we're not really building on any platform not > listed in platform.m4. > > Looking at common/autoconf/platform.m4 (line 20 onwards) where VAR_CPU > seems to be defined, then used as OPENJDK_TARGET_CPU and then as > OPENJDK_TARGET_CPU_LEGACY_LIB and yet again as ZERO_LIBARCH. It may be > true for all the CPUs listed in platform.m4. I can only speculate, but > some distributions still build on more platforms than listed there > (Debian?). So if some distribution patches that code to support a > specific arch, what happens then? Has this been considered when you > wrote the above? I can't really tell how the code behaves when additional patches I have not seen are laid on top of it. :-) However, I *can* see that in hotspot-spec.gmk ZERO_LIBARCH is *always* defined to have a value, so unless this is patched around as well, the "unknown" case will never be reached. Otoh, I'm not sure what that would mean either. The code is not likely to just happen to start running on an unknown architecture without some nontrivial effort. I just thought the code would do the same but more clearly if the fallback case just used OPENJDK_TARGET_CPU_LEGACY_LIB, instead of talking about zero and "unknown". > >> Also, I'd like to hijacking this thread slightly, sorry about that. >> :) >> >> When rewriting the Hotspot makefiles, I've been wondering if it >> would >> not make more sense to consider "zero" to be a different >> architecture, >> rather than a different "flavor" (like minimal1), as it is treated >> right >> now. This patch seems to align with the treatment of zero as an >> architecture. Just as an open question, do you think that would make >> sense? > A Zero (interpreter only) JVM has the goal to be (easily) built on > many (any?) architectures. There can be Zero on x86_64, on arm32, etc. > I don't think it makes sense to treat Zero as an architecture itself. > To me it being a variant/flavor kind of build is correct (depending on > what the definition of a variant/flavor build is exactly). Ok, great! Thanks for your input. /Magnus > > Cheers, > Severin > >> /Magnus From gilles.m.duboscq at oracle.com Thu Sep 17 13:57:19 2015 From: gilles.m.duboscq at oracle.com (Gilles Duboscq) Date: Thu, 17 Sep 2015 15:57:19 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55FA6AC3.2010609@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <55F9CD08.8010406@oracle.com> <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> <55FA6AC3.2010609@oracle.com> Message-ID: <55FAC6BF.5070404@oracle.com> This warning (undefined-bool-conversion) comes from existing code, not from new code contained in this patch. The warning appears is in adlc: /src/share/vm/adlc/filebuff.cpp:109:8: error: 'this' pointer cannot be null in well-defined C++ code; pointer may be assumed to always convert to true [-Werror,-Wundefined-bool-conversion] We added -Wundefined-bool-conversion during jvmci development because it was making our builds fail but it's not related to jvmci at all. Gilles On 17/09/15 09:24, Magnus Ihse Bursie wrote: > On 2015-09-16 22:25, Christian Thalinger wrote: >> >>> On Sep 16, 2015, at 10:11 AM, Magnus Ihse Bursie >>> >> > wrote: >>> >>> On 2015-09-16 18:52, Christian Thalinger wrote: >>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie >>>>> >>>> > wrote: >>>>> >>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>> The JEP itself can be found here: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>> >>>>>> >>>>>> Here are the webrevs: >>>>>> >>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>> >>>>>> >>>>> > >>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>> >>>>>> >>>>> > >>>>>> >>>>>> The change has already undergone a few iterations of internal >>>>>> review by different people of different teams. Most comments and >>>>>> suggestions were handled accordingly. Although there is one open >>>>>> item we agreed we will address after the integration of JEP 243 >>>>>> and that work is captured in RFE: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>> >>>>>> >>>>>> SQE is still working on the tests and all test tasks can be seen >>>>>> as sub-tasks of the JEP. >>>>>> >>>>>> The integration will happen under the bug number: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>> >>>>>> >>>>> Hi Christian, >>>>> >>>>> (Adding build-dev since this review includes some major build >>>>> changes.) >>>>> >>>>> The makefile changes looks good in general to me. I have not >>>>> reviewed the source code changes. >>>>> >>>>> Some comments: >>>>> >>>>> * modules.xml: >>>>> Make sure you get sign-off from a Jigsaw developer for modifying >>>>> this file. >>>> I?ve asked Alan to take a look. >>>> >>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>> Seems unfortunate to have to disable a new warning >>>>> (undefined-bool-conversion) for newly incorporated code. Is it not >>>>> possible to fix the code emitting this warning instead? >>>> We had this question before. It?s not obvious because you can?t see >>>> it in the regular diff views but this is under: >>>> >>>> ifeq ($(USE_CLANG), true) >>> >>> I'm not sure I understand why that's relevant..? Isn't it just as >>> important to try to submit warning-free code when compiling with >>> clang as with any other compiler? Or is clang just being anal about >>> the code in question? >> >> I don?t have a Clang compiler at hand but Clang is anal about >> everything. Do you want that suppression to be removed? > > It's more a hotspot code quality issue, not a build system issue, so I > won't insist. I just wanted to point out that this change will start > hiding a new kind of warning for all files in hotspot. Unless there was > a compelling reason, I would personally rather see an effort to fix the > code in question. But if no-one from Hotspot agrees on this, I'll drop it. > > /Magnus > From volker.simonis at gmail.com Thu Sep 17 16:01:38 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Thu, 17 Sep 2015 18:01:38 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55FAC6BF.5070404@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <55F9CD08.8010406@oracle.com> <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> <55FA6AC3.2010609@oracle.com> <55FAC6BF.5070404@oracle.com> Message-ID: If this comes only from adlc, you could fix for adlc only by adding WARNINGS_ARE_ERRORS += -Wno-undefined-bool-conversion to make/linux/makefiles/adlc.make instead of make/linux/makefiles/gcc.make I do however not understand why this could happen during jvmci development if you didn't touch any adlc file. Maybe you used another compiler (newer/older than 4.8.2) during jvmci development ? Regards, Volker On Thu, Sep 17, 2015 at 3:57 PM, Gilles Duboscq wrote: > This warning (undefined-bool-conversion) comes from existing code, not > from new code contained in this patch. > The warning appears is in adlc: > /src/share/vm/adlc/filebuff.cpp:109:8: error: 'this' pointer cannot be > null in well-defined C++ code; pointer may be assumed to always convert > to true [-Werror,-Wundefined-bool-conversion] > We added -Wundefined-bool-conversion during jvmci development because it > was making our builds fail but it's not related to jvmci at all. > > Gilles > > On 17/09/15 09:24, Magnus Ihse Bursie wrote: >> On 2015-09-16 22:25, Christian Thalinger wrote: >>> >>>> On Sep 16, 2015, at 10:11 AM, Magnus Ihse Bursie >>>> >>> > wrote: >>>> >>>> On 2015-09-16 18:52, Christian Thalinger wrote: >>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie >>>>>> >>>>> > wrote: >>>>>> >>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>> The JEP itself can be found here: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>> >>>>>>> >>>>>>> Here are the webrevs: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>> >>>>>>> >>>>>> > >>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>> >>>>>>> >>>>>> > >>>>>>> >>>>>>> The change has already undergone a few iterations of internal >>>>>>> review by different people of different teams. Most comments and >>>>>>> suggestions were handled accordingly. Although there is one open >>>>>>> item we agreed we will address after the integration of JEP 243 >>>>>>> and that work is captured in RFE: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>> >>>>>>> >>>>>>> SQE is still working on the tests and all test tasks can be seen >>>>>>> as sub-tasks of the JEP. >>>>>>> >>>>>>> The integration will happen under the bug number: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>> >>>>>>> >>>>>> Hi Christian, >>>>>> >>>>>> (Adding build-dev since this review includes some major build >>>>>> changes.) >>>>>> >>>>>> The makefile changes looks good in general to me. I have not >>>>>> reviewed the source code changes. >>>>>> >>>>>> Some comments: >>>>>> >>>>>> * modules.xml: >>>>>> Make sure you get sign-off from a Jigsaw developer for modifying >>>>>> this file. >>>>> I?ve asked Alan to take a look. >>>>> >>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>> Seems unfortunate to have to disable a new warning >>>>>> (undefined-bool-conversion) for newly incorporated code. Is it not >>>>>> possible to fix the code emitting this warning instead? >>>>> We had this question before. It?s not obvious because you can?t see >>>>> it in the regular diff views but this is under: >>>>> >>>>> ifeq ($(USE_CLANG), true) >>>> >>>> I'm not sure I understand why that's relevant..? Isn't it just as >>>> important to try to submit warning-free code when compiling with >>>> clang as with any other compiler? Or is clang just being anal about >>>> the code in question? >>> >>> I don?t have a Clang compiler at hand but Clang is anal about >>> everything. Do you want that suppression to be removed? >> >> It's more a hotspot code quality issue, not a build system issue, so I >> won't insist. I just wanted to point out that this change will start >> hiding a new kind of warning for all files in hotspot. Unless there was >> a compelling reason, I would personally rather see an effort to fix the >> code in question. But if no-one from Hotspot agrees on this, I'll drop it. >> >> /Magnus >> From ioi.lam at oracle.com Thu Sep 17 19:05:05 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 17 Sep 2015 12:05:05 -0700 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <8d1b14ae3127d29dd27d444c25e7c820.squirrel@excalibur.xssl.net> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> <0fed0bdb35f81457858dcbcb09a18cc5.squirrel@excalibur.xssl.net> <1A9BDE02-6DEC-461E-836F-1E3E0FECAF5F@oracle.com> <8d1b14ae3127d29dd27d444c25e7c820.squirrel@excalibur.xssl.net> Message-ID: <55FB0EE1.6030708@oracle.com> Or, could we change the [] to <>? That way it's easier to parse by regexp. [2015-09-15T10:08:40.407-0100][0.203s] regex is: \[([^\[]*)\]\[([^\[]*)\] <2015-09-15T10:08:40.407-0100><0.203s> regex is: <([^>]*)><([^>]*)> Thanks - Ioi On 9/15/15 7:03 AM, Chris Newland wrote: > Hi Staffan, > > I'd expect most logs to be read by tools so unambiguous beats ugly for me. > > [] could be replaced by a single char for neatness? > > .[0.203s]......[info ][logging] Log configuration fully initialized. > > -Chris > > On Tue, September 15, 2015 14:57, Staffan Larsen wrote: >> Like so: >> >> >> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >> [2015-09-15T10:08:40.407-0100][0.203s][][][][][91539][4867][info >> ][logging] Log configuration fully initialized. >> >> >> or with default decorations: >> >> $ java -Xlog:all Hello >> [][0.203s][][][][][][][info ][logging] Log configuration fully >> initialized. >> >> Looks a bit ugly to me. >> >> >> >>> On 15 sep 2015, at 15:47, Chris Newland >>> wrote: >>> >>> >>> Rather than decorating the values, would it be so bad to print empty >>> brackets [] when a log field is ommitted? >>> >>> Wastes 2 chars per ommitted field but reduces ambiguity and simplifies >>> parsing? >>> >>> -Chris >>> >>> >>> On Tue, September 15, 2015 13:46, Staffan Larsen wrote: >>> >>>> Some way of differentiating the tid and pid decorations would be >>>> good. Even if you specify them in a different order on the command >>>> line, they will always show up in the same order which can be even >>>> more confusing. >>>> >>>> Maybe prefix them with ???p??? and ???t???: >>>> >>>> >>>> >>>> [2015-09-15T10:08:40.407-0100][0.203s][p91539][t4867][info >>>> ][logging] >>>> Log configuration fully initialized. >>>> >>>> >>>> >>>> Or postfix: >>>> >>>> >>>> >>>> [2015-09-15T10:08:40.407-0100][0.203s][91539p][4867t][info >>>> ][logging] >>>> Log configuration fully initialized. >>>> >>>> >>>> >>>> Or label (but this is getting verbose): >>>> >>>> >>>> >>>> [2015-09-15T10:08:40.407-0100][0.203s][pid=91539][tid=4867][info >>>> ][logging] Log configuration fully initialized. >>>> >>>> >>>> >>>> Other ideas? >>>> >>>> >>>> >>>> >>>> >>>>> On 15 sep 2015, at 14:27, Staffan Larsen >>>>> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> >>>>>> On 15 sep 2015, at 14:20, Kirk Pepperdine >>>>>> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Just for completeness: there can be more than 3 bracket pairs >>>>>>> if you configure additional decorations: >>>>>>> >>>>>>> $ java -Xlog:all::uptime,time,pid,tid,level,tags Hello >>>>>>> [2015-09-15T10:08:40.407-0100][0.203s][91539][4867][info >>>>>>> ][logging] Log configuration fully initialized. >>>>>>> >>>>>>> >>>>>> Are there a finite number of decorations? Is the order fixed or >>>>>> how would one tell the diff between pid and tid? >>>>> May I refer to the JEP: >>>>> >>>>> >>>>> >>>>> ------------ >>>>> Decorations >>>>> >>>>> >>>>> >>>>> Logging messages are decorated with information about the message. >>>>> Here >>>>> is a list of the possible decorations: >>>>> >>>>> ??? time -- Current time and date in ISO-8601 format ??? uptime -- >>>>> Time >>>>> since the start of the JVM in seconds and milliseconds (e.g., >>>>> 6.567s) >>>>> ??? timemillis -- The same value as generated by >>>>> System.currentTimeMillis() >>>>> ??? uptimemillis -- Milliseconds since the JVM started ??? timenanos >>>>> -- >>>>> The same value as generated by System.nanoTime() >>>>> ??? uptimenanos -- Nanoseconds since the JVM started ??? pid -- The >>>>> process identifier ??? tid -- The thread identifier ??? level -- >>>>> The >>>>> level associated with the log message ??? tags -- The tag-set >>>>> associated with the log message Each output can be configured to use >>>>> a custom set of decorators. The order of them is always the one >>>>> above though. The decorations to be used can be configured by the >>>>> user in runtime. Decorations will be prepended to the log message >>>>> >>>>> >>>>> >>>>> Example: [6.567s][info][gc,old] Old collection complete >>>>> >>>>> >>>>> >>>>> ------------ >>>>> >>>>> >>>>> >>>>> >>>> >>> >> > From christian.thalinger at oracle.com Thu Sep 17 19:51:09 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 17 Sep 2015 09:51:09 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55FAC6BF.5070404@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <55F9CD08.8010406@oracle.com> <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> <55FA6AC3.2010609@oracle.com> <55FAC6BF.5070404@oracle.com> Message-ID: > On Sep 17, 2015, at 3:57 AM, Gilles Duboscq wrote: > > This warning (undefined-bool-conversion) comes from existing code, not > from new code contained in this patch. > The warning appears is in adlc: > /src/share/vm/adlc/filebuff.cpp:109:8: error: 'this' pointer cannot be > null in well-defined C++ code; pointer may be assumed to always convert > to true [-Werror,-Wundefined-bool-conversion] I think all these got fixed in 9. In particular the code you are referring to was removed. Could you try to compile graal-jvmci-9 without the warning disabled? > We added -Wundefined-bool-conversion during jvmci development because it > was making our builds fail but it's not related to jvmci at all. > > Gilles > > On 17/09/15 09:24, Magnus Ihse Bursie wrote: >> On 2015-09-16 22:25, Christian Thalinger wrote: >>> >>>> On Sep 16, 2015, at 10:11 AM, Magnus Ihse Bursie >>>> >>> > wrote: >>>> >>>> On 2015-09-16 18:52, Christian Thalinger wrote: >>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie >>>>>> >>>>> > wrote: >>>>>> >>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>> The JEP itself can be found here: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>> >>>>>>> >>>>>>> Here are the webrevs: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>> >>>>>>> >>>>>> > >>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>> >>>>>>> >>>>>> > >>>>>>> >>>>>>> The change has already undergone a few iterations of internal >>>>>>> review by different people of different teams. Most comments and >>>>>>> suggestions were handled accordingly. Although there is one open >>>>>>> item we agreed we will address after the integration of JEP 243 >>>>>>> and that work is captured in RFE: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>> >>>>>>> >>>>>>> SQE is still working on the tests and all test tasks can be seen >>>>>>> as sub-tasks of the JEP. >>>>>>> >>>>>>> The integration will happen under the bug number: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>> >>>>>>> >>>>>> Hi Christian, >>>>>> >>>>>> (Adding build-dev since this review includes some major build >>>>>> changes.) >>>>>> >>>>>> The makefile changes looks good in general to me. I have not >>>>>> reviewed the source code changes. >>>>>> >>>>>> Some comments: >>>>>> >>>>>> * modules.xml: >>>>>> Make sure you get sign-off from a Jigsaw developer for modifying >>>>>> this file. >>>>> I?ve asked Alan to take a look. >>>>> >>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>> Seems unfortunate to have to disable a new warning >>>>>> (undefined-bool-conversion) for newly incorporated code. Is it not >>>>>> possible to fix the code emitting this warning instead? >>>>> We had this question before. It?s not obvious because you can?t see >>>>> it in the regular diff views but this is under: >>>>> >>>>> ifeq ($(USE_CLANG), true) >>>> >>>> I'm not sure I understand why that's relevant..? Isn't it just as >>>> important to try to submit warning-free code when compiling with >>>> clang as with any other compiler? Or is clang just being anal about >>>> the code in question? >>> >>> I don?t have a Clang compiler at hand but Clang is anal about >>> everything. Do you want that suppression to be removed? >> >> It's more a hotspot code quality issue, not a build system issue, so I >> won't insist. I just wanted to point out that this change will start >> hiding a new kind of warning for all files in hotspot. Unless there was >> a compelling reason, I would personally rather see an effort to fix the >> code in question. But if no-one from Hotspot agrees on this, I'll drop it. >> >> /Magnus >> From christian.thalinger at oracle.com Thu Sep 17 19:51:36 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 17 Sep 2015 09:51:36 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <55F9CD08.8010406@oracle.com> <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> <55FA6AC3.2010609@oracle.com> <55FAC6BF.5070404@oracle.com> Message-ID: > On Sep 17, 2015, at 6:01 AM, Volker Simonis wrote: > > If this comes only from adlc, you could fix for adlc only by adding > > WARNINGS_ARE_ERRORS += -Wno-undefined-bool-conversion > > to make/linux/makefiles/adlc.make instead of make/linux/makefiles/gcc.make > > I do however not understand why this could happen during jvmci > development if you didn't touch any adlc file. Maybe you used another > compiler (newer/older than 4.8.2) during jvmci development ? graal-jvmci-8 is still on 8u45. > > Regards, > Volker > > On Thu, Sep 17, 2015 at 3:57 PM, Gilles Duboscq > wrote: >> This warning (undefined-bool-conversion) comes from existing code, not >> from new code contained in this patch. >> The warning appears is in adlc: >> /src/share/vm/adlc/filebuff.cpp:109:8: error: 'this' pointer cannot be >> null in well-defined C++ code; pointer may be assumed to always convert >> to true [-Werror,-Wundefined-bool-conversion] >> We added -Wundefined-bool-conversion during jvmci development because it >> was making our builds fail but it's not related to jvmci at all. >> >> Gilles >> >> On 17/09/15 09:24, Magnus Ihse Bursie wrote: >>> On 2015-09-16 22:25, Christian Thalinger wrote: >>>> >>>>> On Sep 16, 2015, at 10:11 AM, Magnus Ihse Bursie >>>>> >>>> > wrote: >>>>> >>>>> On 2015-09-16 18:52, Christian Thalinger wrote: >>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie >>>>>>> >>>>>> > wrote: >>>>>>> >>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>> The JEP itself can be found here: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>> >>>>>>>> >>>>>>>> Here are the webrevs: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>> >>>>>>>> >>>>>>> > >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>> >>>>>>>> >>>>>>> > >>>>>>>> >>>>>>>> The change has already undergone a few iterations of internal >>>>>>>> review by different people of different teams. Most comments and >>>>>>>> suggestions were handled accordingly. Although there is one open >>>>>>>> item we agreed we will address after the integration of JEP 243 >>>>>>>> and that work is captured in RFE: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>> >>>>>>>> >>>>>>>> SQE is still working on the tests and all test tasks can be seen >>>>>>>> as sub-tasks of the JEP. >>>>>>>> >>>>>>>> The integration will happen under the bug number: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>> >>>>>>>> >>>>>>> Hi Christian, >>>>>>> >>>>>>> (Adding build-dev since this review includes some major build >>>>>>> changes.) >>>>>>> >>>>>>> The makefile changes looks good in general to me. I have not >>>>>>> reviewed the source code changes. >>>>>>> >>>>>>> Some comments: >>>>>>> >>>>>>> * modules.xml: >>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying >>>>>>> this file. >>>>>> I?ve asked Alan to take a look. >>>>>> >>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>> Seems unfortunate to have to disable a new warning >>>>>>> (undefined-bool-conversion) for newly incorporated code. Is it not >>>>>>> possible to fix the code emitting this warning instead? >>>>>> We had this question before. It?s not obvious because you can?t see >>>>>> it in the regular diff views but this is under: >>>>>> >>>>>> ifeq ($(USE_CLANG), true) >>>>> >>>>> I'm not sure I understand why that's relevant..? Isn't it just as >>>>> important to try to submit warning-free code when compiling with >>>>> clang as with any other compiler? Or is clang just being anal about >>>>> the code in question? >>>> >>>> I don?t have a Clang compiler at hand but Clang is anal about >>>> everything. Do you want that suppression to be removed? >>> >>> It's more a hotspot code quality issue, not a build system issue, so I >>> won't insist. I just wanted to point out that this change will start >>> hiding a new kind of warning for all files in hotspot. Unless there was >>> a compelling reason, I would personally rather see an effort to fix the >>> code in question. But if no-one from Hotspot agrees on this, I'll drop it. >>> >>> /Magnus >>> From christian.thalinger at oracle.com Thu Sep 17 20:32:58 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 17 Sep 2015 10:32:58 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> Message-ID: Since there are no objections I?m going to push this? http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf > On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: > > I would like to add this change: > > diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp > --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 > +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 > @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { > Abstract_VM_Version::vm_release(), > Abstract_VM_Version::vm_info_string(), > TieredCompilation ? ", tiered" : "", > +#if INCLUDE_JVMCI > + EnableJVMCI ? ", jvmci" : "", > + UseJVMCICompiler ? ", jvmci compiler" : "", > +#endif > UseCompressedOops ? ", compressed oops" : "", > gc_mode(), > Abstract_VM_Version::vm_platform_string() > > It would be useful to see in the crash logs if this experimental feature was turned on. > >> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >> >> I updated top and hotspot webrev with latest (build) changes. >> >> Vladimir >> >> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>> >>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>> >>>> >>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>> >>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>> The JEP itself can be found here: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>> >>>>>> Here are the webrevs: >>>>>> >>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>> >>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>> >>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>> >>>>>> The integration will happen under the bug number: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>> >>>>> Hi Christian, >>>>> >>>>> (Adding build-dev since this review includes some major build changes.) >>>>> >>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>> >>>>> Some comments: >>>>> >>>>> * modules.xml: >>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>> >>>> I?ve asked Alan to take a look. >>>> >>>>> >>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>> >>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>> >>>> ifeq ($(USE_CLANG), true) >>>> >>>>> >>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>> >>>> Thanks for the heads up. >>> >>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>> >>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>> >>>> >>>>> >>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>> >>>> Ok, thanks. >>>> >>>>> >>>>> /Magnus > From ioi.lam at oracle.com Thu Sep 17 22:44:20 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 17 Sep 2015 15:44:20 -0700 Subject: RFR (S) - 8065155 Refactor Hotspot mapfiles Message-ID: <55FB4244.4080509@oracle.com> Please review a small fix: http://cr.openjdk.java.net/~iklam/8065155-refactor-hotspot-mapfiles.00/ Bug: Refactor Hotspot mapfiles https://bugs.openjdk.java.net/browse/JDK-8065155 Summary of fix: After this fix, when you're adding a new JVM_XXX or JNI_XXX function, you no longer need to edit *9* files!! I checked the mapfiles of all platforms that are in the jdk9/hs-rt repo: [a] linux, solaris, bsd, bsd-darwin all have an identical common part that contains many JNI_ and JVM_ functions. [b] the common parts of linux, solaris and bsd are sorted in the same order. bsd-darwin is sorted differently but the contents are the same. [c] aix is slightly differently than the other platforms, probably due to code rot. So the fix is to move the common part into a separate file. I changed the makefiles of all platforms to concatenate the common part into the main mapfile. As part of the fix, I also addressed: JDK-8134783 - libjvm.so is not rebuilt after mapfile-ext is modified Note that I have not tested AIX since I don't have a build environment. However, it should be no worse than before. Tests: JPRT (passed) RBT (will do it before pushing) Thanks - Ioi From coleen.phillimore at oracle.com Thu Sep 17 22:54:08 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 17 Sep 2015 18:54:08 -0400 Subject: RFR (S) - 8065155 Refactor Hotspot mapfiles In-Reply-To: <55FB4244.4080509@oracle.com> References: <55FB4244.4080509@oracle.com> Message-ID: <55FB4490.5060104@oracle.com> Hooray, this is great. Thank you! I'm not really a good reviewer of makefiles however. Coleen On 9/17/15 6:44 PM, Ioi Lam wrote: > Please review a small fix: > > http://cr.openjdk.java.net/~iklam/8065155-refactor-hotspot-mapfiles.00/ > > Bug: Refactor Hotspot mapfiles > > https://bugs.openjdk.java.net/browse/JDK-8065155 > > Summary of fix: > > After this fix, when you're adding a new JVM_XXX or JNI_XXX function, > you no longer need to edit *9* files!! > > I checked the mapfiles of all platforms that are in the jdk9/hs-rt > repo: > > [a] linux, solaris, bsd, bsd-darwin all have an identical common > part that > contains many JNI_ and JVM_ functions. > [b] the common parts of linux, solaris and bsd are sorted in the > same order. > bsd-darwin is sorted differently but the contents are the same. > [c] aix is slightly differently than the other platforms, probably > due to > code rot. > > So the fix is to move the common part into a separate file. I > changed the makefiles of all platforms to concatenate the common > part into > the main mapfile. > > As part of the fix, I also addressed: > JDK-8134783 - libjvm.so is not rebuilt after mapfile-ext is > modified > > Note that I have not tested AIX since I don't have a build > environment. However, > it should be no worse than before. > > Tests: > > JPRT (passed) > RBT (will do it before pushing) > > Thanks > - Ioi From magnus.ihse.bursie at oracle.com Fri Sep 18 07:06:03 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 18 Sep 2015 09:06:03 +0200 Subject: RFR (S) - 8065155 Refactor Hotspot mapfiles In-Reply-To: <55FB4244.4080509@oracle.com> References: <55FB4244.4080509@oracle.com> Message-ID: <55FBB7DB.9000101@oracle.com> On 2015-09-18 00:44, Ioi Lam wrote: > Please review a small fix: > > http://cr.openjdk.java.net/~iklam/8065155-refactor-hotspot-mapfiles.00/ Looks good to me. /Magnus > > Bug: Refactor Hotspot mapfiles > > https://bugs.openjdk.java.net/browse/JDK-8065155 > > Summary of fix: > > After this fix, when you're adding a new JVM_XXX or JNI_XXX function, > you no longer need to edit *9* files!! > > I checked the mapfiles of all platforms that are in the jdk9/hs-rt > repo: > > [a] linux, solaris, bsd, bsd-darwin all have an identical common > part that > contains many JNI_ and JVM_ functions. > [b] the common parts of linux, solaris and bsd are sorted in the > same order. > bsd-darwin is sorted differently but the contents are the same. > [c] aix is slightly differently than the other platforms, probably > due to > code rot. > > So the fix is to move the common part into a separate file. I > changed the makefiles of all platforms to concatenate the common > part into > the main mapfile. > > As part of the fix, I also addressed: > JDK-8134783 - libjvm.so is not rebuilt after mapfile-ext is > modified > > Note that I have not tested AIX since I don't have a build > environment. However, > it should be no worse than before. > > Tests: > > JPRT (passed) > RBT (will do it before pushing) > > Thanks > - Ioi From volker.simonis at gmail.com Fri Sep 18 08:20:17 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 18 Sep 2015 10:20:17 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> Message-ID: For the top-level change I always get a strange error when importing it: patch failed, unable to continue (try -v) patch failed, rejects left in working dir errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch It is because of the strange path syntax of the last hunk in the patch file: --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 @@ -254,6 +254,14 @@ jdk.jfr + jdk.internal.jvmci.hotspot + jdk.jfr + + + jdk.internal.jvmci.hotspot.events + jdk.jfr + + sun.misc java.corba java.desktop Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". If I remove the redundant './' from the path, everything works fine. For some reason only the diffs for modules.xml has this strange path syntax in the patch. Regards, Volker On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger wrote: > Since there are no objections I?m going to push this? > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf > >> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >> >> I would like to add this change: >> >> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >> Abstract_VM_Version::vm_release(), >> Abstract_VM_Version::vm_info_string(), >> TieredCompilation ? ", tiered" : "", >> +#if INCLUDE_JVMCI >> + EnableJVMCI ? ", jvmci" : "", >> + UseJVMCICompiler ? ", jvmci compiler" : "", >> +#endif >> UseCompressedOops ? ", compressed oops" : "", >> gc_mode(), >> Abstract_VM_Version::vm_platform_string() >> >> It would be useful to see in the crash logs if this experimental feature was turned on. >> >>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>> >>> I updated top and hotspot webrev with latest (build) changes. >>> >>> Vladimir >>> >>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>> >>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>> >>>>> >>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>> >>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>> The JEP itself can be found here: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>> >>>>>>> Here are the webrevs: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>> >>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>> >>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>> >>>>>>> The integration will happen under the bug number: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>> >>>>>> Hi Christian, >>>>>> >>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>> >>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>> >>>>>> Some comments: >>>>>> >>>>>> * modules.xml: >>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>> >>>>> I?ve asked Alan to take a look. >>>>> >>>>>> >>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>> >>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>> >>>>> ifeq ($(USE_CLANG), true) >>>>> >>>>>> >>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>> >>>>> Thanks for the heads up. >>>> >>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>> >>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>> >>>>> >>>>>> >>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>> >>>>> Ok, thanks. >>>>> >>>>>> >>>>>> /Magnus >> > From volker.simonis at gmail.com Fri Sep 18 09:00:05 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 18 Sep 2015 11:00:05 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> Message-ID: To which repository will you finally push these changes? The current webrevs are against jdk9/jdk9/hotspot but I doubt that they will be integrated there. Unfortunately the patches don't apply to jdk9/dev/hotspot anymore because of '8135067: Preparatory refactorings for compiler control' and "8134626: Misc cleanups after generation array removal": 3 out of 18 hunks FAILED -- saving rejects to file src/share/vm/compiler/compileBroker.cpp.rej 1 out of 5 hunks FAILED -- saving rejects to file src/share/vm/compiler/compileBroker.hpp.rej 6 out of 40 hunks FAILED -- saving rejects to file src/share/vm/runtime/vmStructs.cpp.rej Will you update the patches and if yes against which repositories? If I want to work on the ppc64 implementation, which repository should I use? Thank you and best regards, Volker On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis wrote: > For the top-level change I always get a strange error when importing it: > > patch failed, unable to continue (try -v) > patch failed, rejects left in working dir > errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch > > It is because of the strange path syntax of the last hunk in the patch file: > > --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 > +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 > @@ -254,6 +254,14 @@ > jdk.jfr > > > + jdk.internal.jvmci.hotspot > + jdk.jfr > + > + > + jdk.internal.jvmci.hotspot.events > + jdk.jfr > + > + > sun.misc > java.corba > java.desktop > > > Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". > If I remove the redundant './' from the path, everything works fine. > For some reason only the diffs for modules.xml has this strange path > syntax in the patch. > > Regards, > Volker > > > On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger > wrote: >> Since there are no objections I?m going to push this? >> >> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >> >>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>> >>> I would like to add this change: >>> >>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>> Abstract_VM_Version::vm_release(), >>> Abstract_VM_Version::vm_info_string(), >>> TieredCompilation ? ", tiered" : "", >>> +#if INCLUDE_JVMCI >>> + EnableJVMCI ? ", jvmci" : "", >>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>> +#endif >>> UseCompressedOops ? ", compressed oops" : "", >>> gc_mode(), >>> Abstract_VM_Version::vm_platform_string() >>> >>> It would be useful to see in the crash logs if this experimental feature was turned on. >>> >>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>> >>>> I updated top and hotspot webrev with latest (build) changes. >>>> >>>> Vladimir >>>> >>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>> >>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>> >>>>>> >>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>> >>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>> The JEP itself can be found here: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>> >>>>>>>> Here are the webrevs: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>> >>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>> >>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>> >>>>>>>> The integration will happen under the bug number: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>> >>>>>>> Hi Christian, >>>>>>> >>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>> >>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>> >>>>>>> Some comments: >>>>>>> >>>>>>> * modules.xml: >>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>> >>>>>> I?ve asked Alan to take a look. >>>>>> >>>>>>> >>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>> >>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>> >>>>>> ifeq ($(USE_CLANG), true) >>>>>> >>>>>>> >>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>> >>>>>> Thanks for the heads up. >>>>> >>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>> >>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>> >>>>>> >>>>>>> >>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>> >>>>>> Ok, thanks. >>>>>> >>>>>>> >>>>>>> /Magnus >>> >> From bertrand.delsart at oracle.com Fri Sep 18 09:11:14 2015 From: bertrand.delsart at oracle.com (Bertrand Delsart) Date: Fri, 18 Sep 2015 11:11:14 +0200 Subject: RFR (S) - 8065155 Refactor Hotspot mapfiles In-Reply-To: <55FB4244.4080509@oracle.com> References: <55FB4244.4080509@oracle.com> Message-ID: <55FBD532.1060105@oracle.com> Looks good. Thanks, Bertrand (not a Reviewer). On 18/09/2015 00:44, Ioi Lam wrote: > Please review a small fix: > > http://cr.openjdk.java.net/~iklam/8065155-refactor-hotspot-mapfiles.00/ > > Bug: Refactor Hotspot mapfiles > > https://bugs.openjdk.java.net/browse/JDK-8065155 > > Summary of fix: > > After this fix, when you're adding a new JVM_XXX or JNI_XXX function, > you no longer need to edit *9* files!! > > I checked the mapfiles of all platforms that are in the jdk9/hs-rt > repo: > > [a] linux, solaris, bsd, bsd-darwin all have an identical common > part that > contains many JNI_ and JVM_ functions. > [b] the common parts of linux, solaris and bsd are sorted in the > same order. > bsd-darwin is sorted differently but the contents are the same. > [c] aix is slightly differently than the other platforms, probably > due to > code rot. > > So the fix is to move the common part into a separate file. I > changed the makefiles of all platforms to concatenate the common > part into > the main mapfile. > > As part of the fix, I also addressed: > JDK-8134783 - libjvm.so is not rebuilt after mapfile-ext is > modified > > Note that I have not tested AIX since I don't have a build > environment. However, > it should be no worse than before. > > Tests: > > JPRT (passed) > RBT (will do it before pushing) > > Thanks > - Ioi -- Bertrand Delsart, Grenoble Engineering Center Oracle, 180 av. de l'Europe, ZIRST de Montbonnot 38330 Montbonnot Saint Martin, FRANCE bertrand.delsart at oracle.com Phone : +33 4 76 18 81 23 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From gilles.m.duboscq at oracle.com Fri Sep 18 09:35:28 2015 From: gilles.m.duboscq at oracle.com (Gilles Duboscq) Date: Fri, 18 Sep 2015 11:35:28 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <55F9CD08.8010406@oracle.com> <981500C6-EA6B-4BCE-972B-4981B5C1382F@oracle.com> <55FA6AC3.2010609@oracle.com> <55FAC6BF.5070404@oracle.com> Message-ID: <55FBDAE0.8080202@oracle.com> This only affects clang, and yes, we probably use a newer version. For example, this warning already triggers for me in clang 3.5. If this change is a problem, we can file a separate bug since it's not really related to the jvmci changes. Gilles On 17/09/15 18:01, Volker Simonis wrote: > If this comes only from adlc, you could fix for adlc only by adding > > WARNINGS_ARE_ERRORS += -Wno-undefined-bool-conversion > > to make/linux/makefiles/adlc.make instead of make/linux/makefiles/gcc.make > > I do however not understand why this could happen during jvmci > development if you didn't touch any adlc file. Maybe you used another > compiler (newer/older than 4.8.2) during jvmci development ? > > Regards, > Volker > > On Thu, Sep 17, 2015 at 3:57 PM, Gilles Duboscq > wrote: >> This warning (undefined-bool-conversion) comes from existing code, not >> from new code contained in this patch. >> The warning appears is in adlc: >> /src/share/vm/adlc/filebuff.cpp:109:8: error: 'this' pointer cannot be >> null in well-defined C++ code; pointer may be assumed to always convert >> to true [-Werror,-Wundefined-bool-conversion] >> We added -Wundefined-bool-conversion during jvmci development because it >> was making our builds fail but it's not related to jvmci at all. >> >> Gilles >> >> On 17/09/15 09:24, Magnus Ihse Bursie wrote: >>> On 2015-09-16 22:25, Christian Thalinger wrote: >>>> >>>>> On Sep 16, 2015, at 10:11 AM, Magnus Ihse Bursie >>>>> >>>> > wrote: >>>>> >>>>> On 2015-09-16 18:52, Christian Thalinger wrote: >>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie >>>>>>> >>>>>> > wrote: >>>>>>> >>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>> The JEP itself can be found here: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>> >>>>>>>> >>>>>>>> Here are the webrevs: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>> >>>>>>>> >>>>>>> > >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>> >>>>>>>> >>>>>>> > >>>>>>>> >>>>>>>> The change has already undergone a few iterations of internal >>>>>>>> review by different people of different teams. Most comments and >>>>>>>> suggestions were handled accordingly. Although there is one open >>>>>>>> item we agreed we will address after the integration of JEP 243 >>>>>>>> and that work is captured in RFE: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>> >>>>>>>> >>>>>>>> SQE is still working on the tests and all test tasks can be seen >>>>>>>> as sub-tasks of the JEP. >>>>>>>> >>>>>>>> The integration will happen under the bug number: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>> >>>>>>>> >>>>>>> Hi Christian, >>>>>>> >>>>>>> (Adding build-dev since this review includes some major build >>>>>>> changes.) >>>>>>> >>>>>>> The makefile changes looks good in general to me. I have not >>>>>>> reviewed the source code changes. >>>>>>> >>>>>>> Some comments: >>>>>>> >>>>>>> * modules.xml: >>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying >>>>>>> this file. >>>>>> I?ve asked Alan to take a look. >>>>>> >>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>> Seems unfortunate to have to disable a new warning >>>>>>> (undefined-bool-conversion) for newly incorporated code. Is it not >>>>>>> possible to fix the code emitting this warning instead? >>>>>> We had this question before. It?s not obvious because you can?t see >>>>>> it in the regular diff views but this is under: >>>>>> >>>>>> ifeq ($(USE_CLANG), true) >>>>> >>>>> I'm not sure I understand why that's relevant..? Isn't it just as >>>>> important to try to submit warning-free code when compiling with >>>>> clang as with any other compiler? Or is clang just being anal about >>>>> the code in question? >>>> >>>> I don?t have a Clang compiler at hand but Clang is anal about >>>> everything. Do you want that suppression to be removed? >>> >>> It's more a hotspot code quality issue, not a build system issue, so I >>> won't insist. I just wanted to point out that this change will start >>> hiding a new kind of warning for all files in hotspot. Unless there was >>> a compelling reason, I would personally rather see an effort to fix the >>> code in question. But if no-one from Hotspot agrees on this, I'll drop it. >>> >>> /Magnus >>> From kirk at kodewerk.com Fri Sep 18 10:38:33 2015 From: kirk at kodewerk.com (Kirk Pepperdine) Date: Fri, 18 Sep 2015 13:38:33 +0300 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55FB0EE1.6030708@oracle.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> <0fed0bdb35f81457858dcbcb09a18cc5.squirrel@excalibur.xssl.net> <1A9BDE02-6DEC-461E-836F-1E3E0FECAF5F@oracle.com> <8d1b14ae3127d29dd27d444c25e7c820.squirrel@excalibur.xssl.net> <55FB0EE1.6030708@oracle.com> Message-ID: <6B7DE017-84F8-485E-BA97-FE6DB8601360@kodewerk.com> > On Sep 17, 2015, at 10:05 PM, Ioi Lam wrote: > > Or, could we change the [] to <>? That way it's easier to parse by regexp. > > [2015-09-15T10:08:40.407-0100][0.203s] > regex is: \[([^\[]*)\]\[([^\[]*)\] > > <2015-09-15T10:08:40.407-0100><0.203s> > regex is: <([^>]*)><([^>]*)> +1 It?s been traditional to use ?[? but moving to ? References: <55ED9210.6030904@oracle.com> <55F7E6B5.2040502@oracle.com> <55F9CA0B.2030100@oracle.com> <52003E29-009C-4364-809B-87FB7471A02C@kodewerk.com> <55F9EAB8.2090208@oracle.com> Message-ID: Hi Coleen, Thanks for responding. I am not concerned about the initial pain that will come with a move to UL. I also think that overall it is a move in the right direction. My concerns are that the mistakes that have been made in the Java level logging frameworks don?t get propagated in the JVM. As for GC logging, UL will already break everyone?s tooling so a great opportunity to go whole hog on revamping GC logging. Regards, Kirk > On Sep 17, 2015, at 1:18 AM, Coleen Phillimore wrote: > > > Hi Kirk, > > I think Max answered that we had similar concern. We'll have lots of logging tags for the runtime actions that need to be logged and maybe a sub-tag design would work better. But given the current state of the runtime code (I can't speak for GC but you'll see how GC has been adapted to this framework soon), the levels do make sense for us also. > > Take the case of PrintVtables - we have some code under PrintVtables and some other code under PrintVtables && Verbose. > > Having the levels defined (there are only 3) allows us granularity to get this output or a more verbose output by command line. We hate having to add Verbose and/or Wizard mode to get more logging information. This unified logging approach makes more sense in this context. > > For the most part, I see this logging as a step forward that cleans up the code that we have, but doesn't significantly change the functionality as we're adapting it. We are planning on deprecating many (most?) of the nonproduct command line options, which helps with my concern with the proliferation of options that we have to spend time and support. > > I think there will be ways to improve this and we can add RFE's for what we find. Again, I think this is a step in the right direction for the product. I realize that the GC log changes will cause you pain, and I'm not crazy about blanks in the log lines, but I think this will evolve to something that turns out very nice. > > Thanks, > Coleen > > > On 9/16/15 5:51 PM, Kirk Pepperdine wrote: >> Hi Coleen, >> >> My initial concerns with the JEP and this concern remains is that logging levels should be removed from the specification and that they be replaced with tags that make sense to the work that you are doing. Instead of seeing you define the tags you?d like, we are witnessing a proliferation of logging levels. >> >> I do apologize on bring this point up but I believe this JEP could be improved to avoid some of the problems that I keep running into with the existing logging frameworks that this work seems to want to repeat. >> >> Kind regards, >> Kirk Pepperdine >> >>> On Sep 16, 2015, at 9:59 PM, Coleen Phillimore wrote: >>> >>> Hi Marcus, >>> >>> I had to have someone tell me how the templates work so that the log_is_enabled(Debug, etc) statements are efficient. My initial concern was that it would be expensive to test for logging in place of some runtime Trace flags which evaluate to constant 'false' in product mode. >>> >>> If this still turns out to be a problem, we may ask for an RFE to add a flag_is_enabled(Develop, ) or similar macro that evaluates to a constant false in product mode. Also, I think it might be helpful to have more than one Develop level if we want certain logging flags to not be available in product mode, ie DevelopInfo, DevelopDebug and DevelopTrace. But we can file RFEs for things we may need while doing the runtime logging. >>> >>> In http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.hpp.html >>> >>> Can you add to the comment at 103 that explains the magic that instantiates _tagset to be the static variable that holds the settings for each combination of tags found in the source code? So that is_level is only a static test. >>> >>> And here: >>> >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/src/share/vm/logging/logTagSet.cpp.html >>> >>> The constructor for LogTagSet is a static constructor that links all the static combinations of tags settings together so that they can be initialized with the values for the logging options on the command line. >>> >>> I don't need to see the comments. I think this change is ready for integration. >>> >>> Thanks, >>> Coleen >>> >>> On 9/15/15 5:36 AM, Marcus Larsson wrote: >>>> Hi, >>>> >>>> Here is the updated version of the patch, including the feedback and suggestions from Coleen, David and Ioi. >>>> >>>> Incremental webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ >>>> >>>> Full webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ >>>> >>>> Changes include: >>>> * New NMT tag 'mtLogging' used by UL >>>> * Added storage allocation classes for all new classes >>>> * Added Log::puts(const char*) for plain string messages of any length >>>> * Fixed broken printf-attribute on Log::write functions >>>> * Moved VM startup timestamp to after os::init >>>> * Formatting and code style changes >>>> >>>> Thanks, >>>> Marcus >>>> >>>> >>>> On 2015-09-07 15:33, Marcus Larsson wrote: >>>>> Hi, >>>>> >>>>> Please review the following patch adding the unified logging framework to hotspot. >>>>> >>>>> JEP: >>>>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>>>> >>>>> See the JEP description for a general overview of the new feature. Below are some notes on the implementation. >>>>> >>>>> The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. >>>>> >>>>> Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). >>>>> >>>>> The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). >>>>> >>>>> The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. >>>>> >>>>> The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. >>>>> >>>>> The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. >>>>> >>>>> A simple jtreg test is included in this patch. Additional tests will be added at a later stage. >>>>> >>>>> Thanks, >>>>> Marcus > From roland.westrelin at oracle.com Fri Sep 18 11:14:50 2015 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Fri, 18 Sep 2015 13:14:50 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> Message-ID: <4457C7EC-0672-41C4-B812-97F8EDB8E148@oracle.com> > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ sharedRuntime.cpp 539 if (!((NativeInstruction*)pc)->is_safepoint_poll()) { 540 tty->print_cr("bad pc: %p", pc); 541 Disassembler::decode(cb); 542 assert(false, "Only polling locations are used for safepoint"); 543 } Shouldn?t that be under #idef ASSERT? I looked at the c++ files except src/share/vm/gc stuff. I didn?t go over the java files or makefile changes. What I looked at is ok. Roland. From aph at redhat.com Fri Sep 18 13:07:25 2015 From: aph at redhat.com (Andrew Haley) Date: Fri, 18 Sep 2015 14:07:25 +0100 Subject: 8136165: AARCH64: Tidy up compiled native calls In-Reply-To: <55F1BB94.4080408@redhat.com> References: <55F1AA57.70307@redhat.com> <55F1BB94.4080408@redhat.com> Message-ID: <55FC0C8D.6050508@redhat.com> New patch at http://cr.openjdk.java.net/~aph/8136165-2 Thanks, Andrew. From erik.joelsson at oracle.com Fri Sep 18 16:41:25 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 18 Sep 2015 09:41:25 -0700 Subject: [8u] Request for Approval and Review: JDK-8136691: 8u65/8u66 b14 Solaris builds failed on Linking libverify.so Message-ID: <55FC3EB5.2080905@oracle.com> Hello, Please approve and review this fix for 8u. There is a discrepancy between the Solaris and Linux makefiles for Hotspot, where a source file is excluded for a certain configuration on Linux but not on Solaris. This causes the build to fail on Solaris in that configuration. Bug: https://bugs.openjdk.java.net/browse/JDK-8136691 Patch: diff --git a/make/solaris/makefiles/trace.make b/make/solaris/makefiles/trace.make --- a/make/solaris/makefiles/trace.make +++ b/make/solaris/makefiles/trace.make @@ -56,8 +56,12 @@ ifeq ($(HAS_ALT_SRC), true) TraceGeneratedNames += \ traceRequestables.hpp \ - traceEventControl.hpp \ - traceProducer.cpp + traceEventControl.hpp + +ifneq ($(INCLUDE_TRACE), false) + TraceGeneratedNames += traceProducer.cpp +endif + endif TraceGeneratedFiles = $(TraceGeneratedNames:%=$(TraceOutDir)/%) /Erik From christian.thalinger at oracle.com Fri Sep 18 16:58:38 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 18 Sep 2015 06:58:38 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> Message-ID: <25D9D308-0FBA-47A6-A30D-D5A672AB1AE3@oracle.com> > On Sep 17, 2015, at 10:20 PM, Volker Simonis wrote: > > For the top-level change I always get a strange error when importing it: > > patch failed, unable to continue (try -v) > patch failed, rejects left in working dir > errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch > > It is because of the strange path syntax of the last hunk in the patch file: > > --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 > +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 > @@ -254,6 +254,14 @@ > jdk.jfr > > > + jdk.internal.jvmci.hotspot > + jdk.jfr > + > + > + jdk.internal.jvmci.hotspot.events > + jdk.jfr > + > + > sun.misc > java.corba > java.desktop > > > Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". > If I remove the redundant './' from the path, everything works fine. > For some reason only the diffs for modules.xml has this strange path > syntax in the patch. That?s odd. Vladimir created the webrevs. Maybe he did something different with the top-level one. > > Regards, > Volker > > > On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger > wrote: >> Since there are no objections I?m going to push this? >> >> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >> >>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>> >>> I would like to add this change: >>> >>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>> Abstract_VM_Version::vm_release(), >>> Abstract_VM_Version::vm_info_string(), >>> TieredCompilation ? ", tiered" : "", >>> +#if INCLUDE_JVMCI >>> + EnableJVMCI ? ", jvmci" : "", >>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>> +#endif >>> UseCompressedOops ? ", compressed oops" : "", >>> gc_mode(), >>> Abstract_VM_Version::vm_platform_string() >>> >>> It would be useful to see in the crash logs if this experimental feature was turned on. >>> >>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>> >>>> I updated top and hotspot webrev with latest (build) changes. >>>> >>>> Vladimir >>>> >>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>> >>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>> >>>>>> >>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>> >>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>> The JEP itself can be found here: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>> >>>>>>>> Here are the webrevs: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>> >>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>> >>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>> >>>>>>>> The integration will happen under the bug number: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>> >>>>>>> Hi Christian, >>>>>>> >>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>> >>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>> >>>>>>> Some comments: >>>>>>> >>>>>>> * modules.xml: >>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>> >>>>>> I?ve asked Alan to take a look. >>>>>> >>>>>>> >>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>> >>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>> >>>>>> ifeq ($(USE_CLANG), true) >>>>>> >>>>>>> >>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>> >>>>>> Thanks for the heads up. >>>>> >>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>> >>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>> >>>>>> >>>>>>> >>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>> >>>>>> Ok, thanks. >>>>>> >>>>>>> >>>>>>> /Magnus >>> >> From sean.coffey at oracle.com Fri Sep 18 16:59:00 2015 From: sean.coffey at oracle.com (=?UTF-8?B?U2XDoW4gQ29mZmV5?=) Date: Fri, 18 Sep 2015 17:59:00 +0100 Subject: [8u] Request for Approval and Review: JDK-8136691: 8u65/8u66 b14 Solaris builds failed on Linking libverify.so In-Reply-To: <55FC3EB5.2080905@oracle.com> References: <55FC3EB5.2080905@oracle.com> Message-ID: <55FC42D4.3000703@oracle.com> Approved but subject to review. Please add the noreg-build label. Add the 9-na label if it's not applicable to JDK 9. If it is applicable to JDK 9, create a backport record so that it doesn't get overlooked. Regards, Sean. On 18/09/15 17:41, Erik Joelsson wrote: > Hello, > > Please approve and review this fix for 8u. There is a discrepancy > between the Solaris and Linux makefiles for Hotspot, where a source > file is excluded for a certain configuration on Linux but not on > Solaris. This causes the build to fail on Solaris in that configuration. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8136691 > Patch: > diff --git a/make/solaris/makefiles/trace.make > b/make/solaris/makefiles/trace.make > --- a/make/solaris/makefiles/trace.make > +++ b/make/solaris/makefiles/trace.make > @@ -56,8 +56,12 @@ > ifeq ($(HAS_ALT_SRC), true) > TraceGeneratedNames += \ > traceRequestables.hpp \ > - traceEventControl.hpp \ > - traceProducer.cpp > + traceEventControl.hpp > + > +ifneq ($(INCLUDE_TRACE), false) > + TraceGeneratedNames += traceProducer.cpp > +endif > + > endif > > TraceGeneratedFiles = $(TraceGeneratedNames:%=$(TraceOutDir)/%) > > > /Erik From christian.thalinger at oracle.com Fri Sep 18 17:00:43 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 18 Sep 2015 07:00:43 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> Message-ID: <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> > On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: > > To which repository will you finally push these changes? > > The current webrevs are against jdk9/jdk9/hotspot but I doubt that > they will be integrated there. Good point. > > Unfortunately the patches don't apply to jdk9/dev/hotspot anymore > because of '8135067: Preparatory refactorings for compiler control' > and "8134626: Misc cleanups after generation array removal": > > 3 out of 18 hunks FAILED -- saving rejects to file > src/share/vm/compiler/compileBroker.cpp.rej > 1 out of 5 hunks FAILED -- saving rejects to file > src/share/vm/compiler/compileBroker.hpp.rej > 6 out of 40 hunks FAILED -- saving rejects to file > src/share/vm/runtime/vmStructs.cpp.rej > > Will you update the patches and if yes against which repositories? Yes. Let me update the graal-jvmci-9 repository to hs-comp. > > If I want to work on the ppc64 implementation, which repository should I use? graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? > > Thank you and best regards, > Volker > > > On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis > wrote: >> For the top-level change I always get a strange error when importing it: >> >> patch failed, unable to continue (try -v) >> patch failed, rejects left in working dir >> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >> >> It is because of the strange path syntax of the last hunk in the patch file: >> >> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >> @@ -254,6 +254,14 @@ >> jdk.jfr >> >> >> + jdk.internal.jvmci.hotspot >> + jdk.jfr >> + >> + >> + jdk.internal.jvmci.hotspot.events >> + jdk.jfr >> + >> + >> sun.misc >> java.corba >> java.desktop >> >> >> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >> If I remove the redundant './' from the path, everything works fine. >> For some reason only the diffs for modules.xml has this strange path >> syntax in the patch. >> >> Regards, >> Volker >> >> >> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >> wrote: >>> Since there are no objections I?m going to push this? >>> >>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>> >>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>> >>>> I would like to add this change: >>>> >>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>> Abstract_VM_Version::vm_release(), >>>> Abstract_VM_Version::vm_info_string(), >>>> TieredCompilation ? ", tiered" : "", >>>> +#if INCLUDE_JVMCI >>>> + EnableJVMCI ? ", jvmci" : "", >>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>> +#endif >>>> UseCompressedOops ? ", compressed oops" : "", >>>> gc_mode(), >>>> Abstract_VM_Version::vm_platform_string() >>>> >>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>> >>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>> >>>>> I updated top and hotspot webrev with latest (build) changes. >>>>> >>>>> Vladimir >>>>> >>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>> >>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>> >>>>>>> >>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>> >>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>> The JEP itself can be found here: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>> >>>>>>>>> Here are the webrevs: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>> >>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>> >>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>> >>>>>>>>> The integration will happen under the bug number: >>>>>>>>> >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>> >>>>>>>> Hi Christian, >>>>>>>> >>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>> >>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>> >>>>>>>> Some comments: >>>>>>>> >>>>>>>> * modules.xml: >>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>> >>>>>>> I?ve asked Alan to take a look. >>>>>>> >>>>>>>> >>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>> >>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>> >>>>>>> ifeq ($(USE_CLANG), true) >>>>>>> >>>>>>>> >>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>> >>>>>>> Thanks for the heads up. >>>>>> >>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>> >>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>> >>>>>>> >>>>>>>> >>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>> >>>>>>> Ok, thanks. >>>>>>> >>>>>>>> >>>>>>>> /Magnus >>>> >>> From christian.thalinger at oracle.com Fri Sep 18 17:02:33 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 18 Sep 2015 07:02:33 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <4457C7EC-0672-41C4-B812-97F8EDB8E148@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <4457C7EC-0672-41C4-B812-97F8EDB8E148@oracle.com> Message-ID: > On Sep 18, 2015, at 1:14 AM, Roland Westrelin wrote: > >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ > > sharedRuntime.cpp > > 539 if (!((NativeInstruction*)pc)->is_safepoint_poll()) { > 540 tty->print_cr("bad pc: %p", pc); > 541 Disassembler::decode(cb); > 542 assert(false, "Only polling locations are used for safepoint"); > 543 } > > Shouldn?t that be under #idef ASSERT? Either that or make the assert a fatal. > > I looked at the c++ files except src/share/vm/gc stuff. I didn?t go over the java files or makefile changes. What I looked at is ok. Thank you. > > Roland. From tim.bell at oracle.com Fri Sep 18 18:16:17 2015 From: tim.bell at oracle.com (Tim Bell) Date: Fri, 18 Sep 2015 11:16:17 -0700 Subject: [8u] Request for Approval and Review: JDK-8136691: 8u65/8u66 b14 Solaris builds failed on Linking libverify.so In-Reply-To: <55FC42D4.3000703@oracle.com> References: <55FC3EB5.2080905@oracle.com> <55FC42D4.3000703@oracle.com> Message-ID: <55FC54F1.6030901@oracle.com> Erik- The code review part looks good. Tim On 09/18/15 09:59, Se?n Coffey wrote: > Approved but subject to review. Please add the noreg-build label. Add > the 9-na label if it's not applicable to JDK 9. If it is applicable to > JDK 9, create a backport record so that it doesn't get overlooked. > > Regards, > Sean. > > On 18/09/15 17:41, Erik Joelsson wrote: >> Hello, >> >> Please approve and review this fix for 8u. There is a discrepancy >> between the Solaris and Linux makefiles for Hotspot, where a source >> file is excluded for a certain configuration on Linux but not on >> Solaris. This causes the build to fail on Solaris in that configuration. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8136691 >> Patch: >> diff --git a/make/solaris/makefiles/trace.make >> b/make/solaris/makefiles/trace.make >> --- a/make/solaris/makefiles/trace.make >> +++ b/make/solaris/makefiles/trace.make >> @@ -56,8 +56,12 @@ >> ifeq ($(HAS_ALT_SRC), true) >> TraceGeneratedNames += \ >> traceRequestables.hpp \ >> - traceEventControl.hpp \ >> - traceProducer.cpp >> + traceEventControl.hpp >> + >> +ifneq ($(INCLUDE_TRACE), false) >> + TraceGeneratedNames += traceProducer.cpp >> +endif >> + >> endif >> >> TraceGeneratedFiles = $(TraceGeneratedNames:%=$(TraceOutDir)/%) >> >> >> /Erik > From volker.simonis at gmail.com Fri Sep 18 18:46:39 2015 From: volker.simonis at gmail.com (Volker Simonis) Date: Fri, 18 Sep 2015 20:46:39 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> Message-ID: On Fri, Sep 18, 2015 at 7:00 PM, Christian Thalinger wrote: > >> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >> >> To which repository will you finally push these changes? >> >> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >> they will be integrated there. > > Good point. > >> >> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >> because of '8135067: Preparatory refactorings for compiler control' >> and "8134626: Misc cleanups after generation array removal": >> >> 3 out of 18 hunks FAILED -- saving rejects to file >> src/share/vm/compiler/compileBroker.cpp.rej >> 1 out of 5 hunks FAILED -- saving rejects to file >> src/share/vm/compiler/compileBroker.hpp.rej >> 6 out of 40 hunks FAILED -- saving rejects to file >> src/share/vm/runtime/vmStructs.cpp.rej >> >> Will you update the patches and if yes against which repositories? > > Yes. Let me update the graal-jvmci-9 repository to hs-comp. > >> >> If I want to work on the ppc64 implementation, which repository should I use? > > graal-jvmci-9. Yes, I already figured out that for now, until nothing has been pushed to jdk9 graal-jvmci-9 is probably the best way to go. > Are you working on this for fun or do you want to have that integrated with this JEP? > Isn't it always fun :) But seriously, I currently can not estimate the time it would take me to do the implementation and I also can not oversee the benefit we would have from it until we don't have a Graal back-end for ppc64. The "JVMCI Design Document" could certainly help here :) Thank you and best regards, Volker >> >> Thank you and best regards, >> Volker >> >> >> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >> wrote: >>> For the top-level change I always get a strange error when importing it: >>> >>> patch failed, unable to continue (try -v) >>> patch failed, rejects left in working dir >>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>> >>> It is because of the strange path syntax of the last hunk in the patch file: >>> >>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>> @@ -254,6 +254,14 @@ >>> jdk.jfr >>> >>> >>> + jdk.internal.jvmci.hotspot >>> + jdk.jfr >>> + >>> + >>> + jdk.internal.jvmci.hotspot.events >>> + jdk.jfr >>> + >>> + >>> sun.misc >>> java.corba >>> java.desktop >>> >>> >>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>> If I remove the redundant './' from the path, everything works fine. >>> For some reason only the diffs for modules.xml has this strange path >>> syntax in the patch. >>> >>> Regards, >>> Volker >>> >>> >>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>> wrote: >>>> Since there are no objections I?m going to push this? >>>> >>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>> >>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>> >>>>> I would like to add this change: >>>>> >>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>> Abstract_VM_Version::vm_release(), >>>>> Abstract_VM_Version::vm_info_string(), >>>>> TieredCompilation ? ", tiered" : "", >>>>> +#if INCLUDE_JVMCI >>>>> + EnableJVMCI ? ", jvmci" : "", >>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>> +#endif >>>>> UseCompressedOops ? ", compressed oops" : "", >>>>> gc_mode(), >>>>> Abstract_VM_Version::vm_platform_string() >>>>> >>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>> >>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>> >>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>> >>>>>> Vladimir >>>>>> >>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>> >>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>> >>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>> The JEP itself can be found here: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>> >>>>>>>>>> Here are the webrevs: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>> >>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>> >>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>> >>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>> >>>>>>>>> Hi Christian, >>>>>>>>> >>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>> >>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>> >>>>>>>>> Some comments: >>>>>>>>> >>>>>>>>> * modules.xml: >>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>> >>>>>>>> I?ve asked Alan to take a look. >>>>>>>> >>>>>>>>> >>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>> >>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>> >>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>> >>>>>>>>> >>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>> >>>>>>>> Thanks for the heads up. >>>>>>> >>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>> >>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>> >>>>>>>> Ok, thanks. >>>>>>>> >>>>>>>>> >>>>>>>>> /Magnus >>>>> >>>> > From christian.thalinger at oracle.com Fri Sep 18 19:09:57 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 18 Sep 2015 09:09:57 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> Message-ID: <6081B18D-1594-41A4-AC90-36DFFC644D6C@oracle.com> > On Sep 18, 2015, at 8:46 AM, Volker Simonis wrote: > > On Fri, Sep 18, 2015 at 7:00 PM, Christian Thalinger > > wrote: >> >>> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >>> >>> To which repository will you finally push these changes? >>> >>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>> they will be integrated there. >> >> Good point. >> >>> >>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>> because of '8135067: Preparatory refactorings for compiler control' >>> and "8134626: Misc cleanups after generation array removal": >>> >>> 3 out of 18 hunks FAILED -- saving rejects to file >>> src/share/vm/compiler/compileBroker.cpp.rej >>> 1 out of 5 hunks FAILED -- saving rejects to file >>> src/share/vm/compiler/compileBroker.hpp.rej >>> 6 out of 40 hunks FAILED -- saving rejects to file >>> src/share/vm/runtime/vmStructs.cpp.rej >>> >>> Will you update the patches and if yes against which repositories? >> >> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >> >>> >>> If I want to work on the ppc64 implementation, which repository should I use? >> >> graal-jvmci-9. > > Yes, I already figured out that for now, until nothing has been pushed > to jdk9 graal-jvmci-9 is probably the best way to go. > >> Are you working on this for fun or do you want to have that integrated with this JEP? >> > > Isn't it always fun :) Of course :-) > > But seriously, I currently can not estimate the time it would take me > to do the implementation and I also can not oversee the benefit we > would have from it until we don't have a Graal back-end for ppc64. Correct. The benefit you would have is you can run (all?) JVMCI regression tests and make sure it doesn?t break on PPC64 and if someone really would go ahead and write a PPC64 backend for Graal you don?t have to make any changes to JDK 9, just to Graal. The most tricky part would be the CodeInstaller. > The > "JVMCI Design Document" could certainly help here :) It?s not as detailed as you think. Btw. can I count you as a reviewer? > > Thank you and best regards, > Volker > >>> >>> Thank you and best regards, >>> Volker >>> >>> >>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>> wrote: >>>> For the top-level change I always get a strange error when importing it: >>>> >>>> patch failed, unable to continue (try -v) >>>> patch failed, rejects left in working dir >>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>> >>>> It is because of the strange path syntax of the last hunk in the patch file: >>>> >>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>> @@ -254,6 +254,14 @@ >>>> jdk.jfr >>>> >>>> >>>> + jdk.internal.jvmci.hotspot >>>> + jdk.jfr >>>> + >>>> + >>>> + jdk.internal.jvmci.hotspot.events >>>> + jdk.jfr >>>> + >>>> + >>>> sun.misc >>>> java.corba >>>> java.desktop >>>> >>>> >>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>> If I remove the redundant './' from the path, everything works fine. >>>> For some reason only the diffs for modules.xml has this strange path >>>> syntax in the patch. >>>> >>>> Regards, >>>> Volker >>>> >>>> >>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>> wrote: >>>>> Since there are no objections I?m going to push this? >>>>> >>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>> >>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>> >>>>>> I would like to add this change: >>>>>> >>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>> Abstract_VM_Version::vm_release(), >>>>>> Abstract_VM_Version::vm_info_string(), >>>>>> TieredCompilation ? ", tiered" : "", >>>>>> +#if INCLUDE_JVMCI >>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>> +#endif >>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>> gc_mode(), >>>>>> Abstract_VM_Version::vm_platform_string() >>>>>> >>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>> >>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>> >>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>> >>>>>>> Vladimir >>>>>>> >>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>> >>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>> >>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>> >>>>>>>>>>> Here are the webrevs: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>> >>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>> >>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>> >>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>> >>>>>>>>>> Hi Christian, >>>>>>>>>> >>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>> >>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>> >>>>>>>>>> Some comments: >>>>>>>>>> >>>>>>>>>> * modules.xml: >>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>> >>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>> >>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>> >>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>> >>>>>>>>>> >>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>> >>>>>>>>> Thanks for the heads up. >>>>>>>> >>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>> >>>>>>>>> Ok, thanks. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> /Magnus From coleen.phillimore at oracle.com Fri Sep 18 22:14:14 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 18 Sep 2015 18:14:14 -0400 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> Message-ID: <55FC8CB6.2030204@oracle.com> I'm adopting Kim's '----' delimiters (only minimally). ---- http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/sparc/vm/cppInterpreter_sparc.cpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/cppInterpreter_x86.cpp.udiff.html Even though this doesn't build, you should change *- void InterpreterGenerator::lock_method(void) {* to CppInterpreterGenerator::lock_method(void) since you've already changed the line. ---- eg. http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/interp_masm_x86.cpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/vm_version_x86.cpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/compiler/compileBroker.cpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/oops/methodData.hpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/sharedRuntime.cpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.cpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.hpp.udiff.html Can you go through and add more #endif // INCLUDE_JVMTI comments? I hate raw #endif preprocessor directives. They make the code really hard to read. Above is a sample, but there are more. If it's a few lines separated, it's not too bad but some of these are long patches of code inserted in the middle of functions. ---- http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp.udiff.html *+ map->set_callee_saved(YMMHI_STACK_OFFSET( 0), xmm0->as_VMReg()->next()->next()->next()->next());* What? Are 4 next() calls really enough? I'm glad I don't work on the compiler. This is really odd looking. Maybe there could be a function in VMReg->4th_next(). ---- http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/templateTable_x86.cpp.udiff.html http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/sparc/vm/templateTable_sparc.cpp.udiff.html profiled_called_method could be JVMCI_ONLY(). Is there always a profile_called_method() before call_from_interpreter() ? maybe the method should be called in there instead at the callers. Or make it unconditional and have !JVMCI version not do anything. Just to cut down on all this conditional code. ---- http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/classfile/* Very nice refactoring!! Thank you. ---- http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/code/relocInfo.hpp.udiff.html Why is the copyright updated to 2014? Do you want my sed script? ---- http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/oops/method.cpp.udiff.html In most of the code WizardMode prints *more* information not less, but this prints less. This change is going to be broken with Unified Logging. ---- http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.hpp.udiff.html I thought you were only going to add 3 words to thread? (If you reordered the fields, it would be). Having to declare this many fields inline to any class suggests it should be refactored into a different class, and a pointer to that class in the global thread class. I've made this comment before, I know. It seems all the blue in this could be a new class. That's all the comments I have. These are minor. I'm not going to look at the 42000 lines of new code again. Thank you for making the suggested changes in the code preview. Thanks, Coleen On 9/14/15 3:24 AM, Christian Thalinger wrote: > The JEP itself can be found here: > > https://bugs.openjdk.java.net/browse/JDK-8062493 > > Here are the webrevs: > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ > > The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: > > https://bugs.openjdk.java.net/browse/JDK-8134994 > > SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. > > The integration will happen under the bug number: > > https://bugs.openjdk.java.net/browse/JDK-8136421 > From christian.thalinger at oracle.com Sat Sep 19 00:19:23 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 18 Sep 2015 14:19:23 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> Message-ID: > On Sep 18, 2015, at 7:00 AM, Christian Thalinger wrote: > > >> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >> >> To which repository will you finally push these changes? >> >> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >> they will be integrated there. > > Good point. > >> >> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >> because of '8135067: Preparatory refactorings for compiler control' >> and "8134626: Misc cleanups after generation array removal": >> >> 3 out of 18 hunks FAILED -- saving rejects to file >> src/share/vm/compiler/compileBroker.cpp.rej >> 1 out of 5 hunks FAILED -- saving rejects to file >> src/share/vm/compiler/compileBroker.hpp.rej >> 6 out of 40 hunks FAILED -- saving rejects to file >> src/share/vm/runtime/vmStructs.cpp.rej >> >> Will you update the patches and if yes against which repositories? > > Yes. Let me update the graal-jvmci-9 repository to hs-comp. Done. Now I have to create new webrevs? > >> >> If I want to work on the ppc64 implementation, which repository should I use? > > graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? > >> >> Thank you and best regards, >> Volker >> >> >> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >> wrote: >>> For the top-level change I always get a strange error when importing it: >>> >>> patch failed, unable to continue (try -v) >>> patch failed, rejects left in working dir >>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>> >>> It is because of the strange path syntax of the last hunk in the patch file: >>> >>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>> @@ -254,6 +254,14 @@ >>> jdk.jfr >>> >>> >>> + jdk.internal.jvmci.hotspot >>> + jdk.jfr >>> + >>> + >>> + jdk.internal.jvmci.hotspot.events >>> + jdk.jfr >>> + >>> + >>> sun.misc >>> java.corba >>> java.desktop >>> >>> >>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>> If I remove the redundant './' from the path, everything works fine. >>> For some reason only the diffs for modules.xml has this strange path >>> syntax in the patch. >>> >>> Regards, >>> Volker >>> >>> >>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>> wrote: >>>> Since there are no objections I?m going to push this? >>>> >>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>> >>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>> >>>>> I would like to add this change: >>>>> >>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>> Abstract_VM_Version::vm_release(), >>>>> Abstract_VM_Version::vm_info_string(), >>>>> TieredCompilation ? ", tiered" : "", >>>>> +#if INCLUDE_JVMCI >>>>> + EnableJVMCI ? ", jvmci" : "", >>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>> +#endif >>>>> UseCompressedOops ? ", compressed oops" : "", >>>>> gc_mode(), >>>>> Abstract_VM_Version::vm_platform_string() >>>>> >>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>> >>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>> >>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>> >>>>>> Vladimir >>>>>> >>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>> >>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>> >>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>> The JEP itself can be found here: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>> >>>>>>>>>> Here are the webrevs: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>> >>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>> >>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>> >>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>> >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>> >>>>>>>>> Hi Christian, >>>>>>>>> >>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>> >>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>> >>>>>>>>> Some comments: >>>>>>>>> >>>>>>>>> * modules.xml: >>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>> >>>>>>>> I?ve asked Alan to take a look. >>>>>>>> >>>>>>>>> >>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>> >>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>> >>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>> >>>>>>>>> >>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>> >>>>>>>> Thanks for the heads up. >>>>>>> >>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>> >>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>> >>>>>>>> Ok, thanks. >>>>>>>> >>>>>>>>> >>>>>>>>> /Magnus >>>>> >>>> > From aph at redhat.com Sat Sep 19 09:42:31 2015 From: aph at redhat.com (Andrew Haley) Date: Sat, 19 Sep 2015 10:42:31 +0100 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> Message-ID: <55FD2E07.6020401@redhat.com> On 14/09/15 08:24, Christian Thalinger wrote: +#if defined(COMPILER2) || INCLUDE_JVMCI + if (save_vectors) { + assert(ymmhi_offset != -1, "save area must exist"); + map->set_callee_saved(YMMHI_STACK_OFFSET( 0), xmm0->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET( 16), xmm1->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET( 32), xmm2->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET( 48), xmm3->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET( 64), xmm4->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET( 80), xmm5->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET( 96), xmm6->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(112), xmm7->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(128), xmm8->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(144), xmm9->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(160), xmm10->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(176), xmm11->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(192), xmm12->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(208), xmm13->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(224), xmm14->as_VMReg()->next()->next()->next()->next()); + map->set_callee_saved(YMMHI_STACK_OFFSET(240), xmm15->as_VMReg()->next()->next()->next()->next()); + } +#endif + Um, really? There is plenty of very odd code in HotSpot, but this is something else again. Andrew. From erik.joelsson at oracle.com Mon Sep 21 16:38:01 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 21 Sep 2015 09:38:01 -0700 Subject: [8u] Request for Approval and Review: JDK-8136691: 8u65/8u66 b14 Solaris builds failed on Linking libverify.so In-Reply-To: <55FC54F1.6030901@oracle.com> References: <55FC3EB5.2080905@oracle.com> <55FC42D4.3000703@oracle.com> <55FC54F1.6030901@oracle.com> Message-ID: <56003269.3040707@oracle.com> Thanks Tim, Since this is Hotspot I assume I need another reviewer. Any chance I could get one? /Erik On 2015-09-18 11:16, Tim Bell wrote: > Erik- > > The code review part looks good. > > Tim > > On 09/18/15 09:59, Se?n Coffey wrote: >> Approved but subject to review. Please add the noreg-build label. Add >> the 9-na label if it's not applicable to JDK 9. If it is applicable >> to JDK 9, create a backport record so that it doesn't get overlooked. >> >> Regards, >> Sean. >> >> On 18/09/15 17:41, Erik Joelsson wrote: >>> Hello, >>> >>> Please approve and review this fix for 8u. There is a discrepancy >>> between the Solaris and Linux makefiles for Hotspot, where a source >>> file is excluded for a certain configuration on Linux but not on >>> Solaris. This causes the build to fail on Solaris in that >>> configuration. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8136691 >>> Patch: >>> diff --git a/make/solaris/makefiles/trace.make >>> b/make/solaris/makefiles/trace.make >>> --- a/make/solaris/makefiles/trace.make >>> +++ b/make/solaris/makefiles/trace.make >>> @@ -56,8 +56,12 @@ >>> ifeq ($(HAS_ALT_SRC), true) >>> TraceGeneratedNames += \ >>> traceRequestables.hpp \ >>> - traceEventControl.hpp \ >>> - traceProducer.cpp >>> + traceEventControl.hpp >>> + >>> +ifneq ($(INCLUDE_TRACE), false) >>> + TraceGeneratedNames += traceProducer.cpp >>> +endif >>> + >>> endif >>> >>> TraceGeneratedFiles = $(TraceGeneratedNames:%=$(TraceOutDir)/%) >>> >>> >>> /Erik >> > From tomasz.wojtowicz at intel.com Fri Sep 18 21:59:01 2015 From: tomasz.wojtowicz at intel.com (Wojtowicz, Tomasz) Date: Fri, 18 Sep 2015 21:59:01 +0000 Subject: RFR (S): 8136672: editor configuration for IDEs Message-ID: <3616187E21868C40AD1B36D41D29F4C136905BEE@FMSMSX106.amr.corp.intel.com> Review Title: editor configuration for IDEs Review ID: #8136672 Diff: http://cr.openjdk.java.net/~mcberg/8136672/webrev.01 Description: In order to avoid jcheck errors like following ones: "Tests found: src/cpu/x86/vm/macroAssembler_x86.cpp:8686: Trailing whitespace src/cpu/x86/vm/macroAssembler_x86.hpp:1330: Trailing whitespace src/cpu/x86/vm/stubGenerator_x86_32.cpp:3001: Trailing whitespace src/cpu/x86/vm/stubGenerator_x86_64.cpp:3968: Trailing whitespace src/cpu/x86/vm/stubRoutines_x86.cpp:136: Trailing whitespace src/share/vm/interpreter/interpreter.cpp:594: Carriage return (^M) src/share/vm/interpreter/templateInterpreter.cpp:426: Carriage return (^M) src/cpu/x86/vm/crc32c.h:31: Trailing whitespace" I propose to introduce an .editorconfig file which would encapsulate common layout required settings. EditorConfig (http://editorconfig.org/) provides plugins for most popular editors/IDEs and make sure that every modification adheres to the stated rules: hotspot/.editorconfig file contents ??????????????? root = true ??????????????? ??????????????? [*] ??????????????? indent_style = space ??????????????? indent_size = 2 ??????????????? end_of_line = lf ??????????????? charset = utf-8 ??????????????? trim_trailing_whitespace = true ??????????????? insert_final_newline = true Link: https://bugs.openjdk.java.net/browse/JDK-8136672 Author: Tomasz, Wojtowicz -- Thank you, Tomek From christian.thalinger at oracle.com Mon Sep 21 17:25:31 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 21 Sep 2015 07:25:31 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <4457C7EC-0672-41C4-B812-97F8EDB8E148@oracle.com> Message-ID: <4298967A-2136-40DD-9DD8-A4A44BC4FC8E@oracle.com> > On Sep 18, 2015, at 7:02 AM, Christian Thalinger wrote: > > >> On Sep 18, 2015, at 1:14 AM, Roland Westrelin wrote: >> >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >> >> sharedRuntime.cpp >> >> 539 if (!((NativeInstruction*)pc)->is_safepoint_poll()) { >> 540 tty->print_cr("bad pc: %p", pc); >> 541 Disassembler::decode(cb); >> 542 assert(false, "Only polling locations are used for safepoint"); >> 543 } >> >> Shouldn?t that be under #idef ASSERT? > > Either that or make the assert a fatal. Or both :-) http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/7274d7abdd37 > >> >> I looked at the c++ files except src/share/vm/gc stuff. I didn?t go over the java files or makefile changes. What I looked at is ok. > > Thank you. > >> >> Roland. > From christian.thalinger at oracle.com Mon Sep 21 18:35:27 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 21 Sep 2015 08:35:27 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55FC8CB6.2030204@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55FC8CB6.2030204@oracle.com> Message-ID: <529B4819-0325-41A0-81F0-EA85A0EBE908@oracle.com> > On Sep 18, 2015, at 12:14 PM, Coleen Phillimore wrote: > > I'm adopting Kim's '----' delimiters (only minimally). > > ---- > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/sparc/vm/cppInterpreter_sparc.cpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/cppInterpreter_x86.cpp.udiff.html > > Even though this doesn't build, you should change > > *- void InterpreterGenerator::lock_method(void) {* > > > to CppInterpreterGenerator::lock_method(void) since you've already changed the line. Done. > > ---- > > eg. http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/interp_masm_x86.cpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/vm_version_x86.cpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/compiler/compileBroker.cpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/oops/methodData.hpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/sharedRuntime.cpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.cpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.hpp.udiff.html > > Can you go through and add more #endif // INCLUDE_JVMTI comments? I hate raw #endif preprocessor directives. They make the code really hard to read. Above is a sample, but there are more. If it's a few lines separated, it's not too bad but some of these are long patches of code inserted in the middle of functions. I?ll try? > > ---- > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp.udiff.html > > *+ map->set_callee_saved(YMMHI_STACK_OFFSET( 0), xmm0->as_VMReg()->next()->next()->next()->next());* > > > What? Are 4 next() calls really enough? I'm glad I don't work on the compiler. This is really odd looking. Maybe there could be a function in VMReg->4th_next(). > > ---- > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/x86/vm/templateTable_x86.cpp.udiff.html > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/cpu/sparc/vm/templateTable_sparc.cpp.udiff.html > > profiled_called_method could be JVMCI_ONLY(). Is there always a profile_called_method() before call_from_interpreter() ? maybe the method should be called in there instead at the callers. Or make it unconditional and have !JVMCI version not do anything. Just to cut down on all this conditional code. I?ve added NOT_JVMCI_RETURN and used that. > > ---- > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/classfile/* > > Very nice refactoring!! Thank you. > > ---- > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/code/relocInfo.hpp.udiff.html > > Why is the copyright updated to 2014? Do you want my sed script? Maybe the last change done to that file was in 2014. Not sure. > > ---- > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/oops/method.cpp.udiff.html > > In most of the code WizardMode prints *more* information not less, but this prints less. This change is going to be broken with Unified Logging. I made that change. It does not print less but in a more low-level way as it prints signatures the way they are stored in the constant pool (with L and /). Wizard-style. > > ---- > > http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.hpp.udiff.html > > I thought you were only going to add 3 words to thread? (If you reordered the fields, it would be). I reordered the fields a bit. > > Having to declare this many fields inline to any class suggests it should be refactored into a different class, and a pointer to that class in the global thread class. I've made this comment before, I know. It seems all the blue in this could be a new class. We can make this change later if we want to. > > That's all the comments I have. These are minor. I'm not going to look at the 42000 lines of new code again. Thank you for making the suggested changes in the code preview. This should address all your review comments: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/7e53ffed78bc Thanks for the review. > > Thanks, > Coleen > > On 9/14/15 3:24 AM, Christian Thalinger wrote: >> The JEP itself can be found here: >> >> https://bugs.openjdk.java.net/browse/JDK-8062493 >> >> Here are the webrevs: >> >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >> >> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >> >> https://bugs.openjdk.java.net/browse/JDK-8134994 >> >> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >> >> The integration will happen under the bug number: >> >> https://bugs.openjdk.java.net/browse/JDK-8136421 >> > From mikael.vidstedt at oracle.com Mon Sep 21 19:38:22 2015 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Mon, 21 Sep 2015 12:38:22 -0700 Subject: [8u] Request for Approval and Review: JDK-8136691: 8u65/8u66 b14 Solaris builds failed on Linking libverify.so In-Reply-To: <56003269.3040707@oracle.com> References: <55FC3EB5.2080905@oracle.com> <55FC42D4.3000703@oracle.com> <55FC54F1.6030901@oracle.com> <56003269.3040707@oracle.com> Message-ID: <56005CAE.5030902@oracle.com> Looks good! Cheers, Mikael On 2015-09-21 09:38, Erik Joelsson wrote: > Thanks Tim, > > Since this is Hotspot I assume I need another reviewer. Any chance I > could get one? > > /Erik > > On 2015-09-18 11:16, Tim Bell wrote: >> Erik- >> >> The code review part looks good. >> >> Tim >> >> On 09/18/15 09:59, Se?n Coffey wrote: >>> Approved but subject to review. Please add the noreg-build label. >>> Add the 9-na label if it's not applicable to JDK 9. If it is >>> applicable to JDK 9, create a backport record so that it doesn't get >>> overlooked. >>> >>> Regards, >>> Sean. >>> >>> On 18/09/15 17:41, Erik Joelsson wrote: >>>> Hello, >>>> >>>> Please approve and review this fix for 8u. There is a discrepancy >>>> between the Solaris and Linux makefiles for Hotspot, where a source >>>> file is excluded for a certain configuration on Linux but not on >>>> Solaris. This causes the build to fail on Solaris in that >>>> configuration. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8136691 >>>> Patch: >>>> diff --git a/make/solaris/makefiles/trace.make >>>> b/make/solaris/makefiles/trace.make >>>> --- a/make/solaris/makefiles/trace.make >>>> +++ b/make/solaris/makefiles/trace.make >>>> @@ -56,8 +56,12 @@ >>>> ifeq ($(HAS_ALT_SRC), true) >>>> TraceGeneratedNames += \ >>>> traceRequestables.hpp \ >>>> - traceEventControl.hpp \ >>>> - traceProducer.cpp >>>> + traceEventControl.hpp >>>> + >>>> +ifneq ($(INCLUDE_TRACE), false) >>>> + TraceGeneratedNames += traceProducer.cpp >>>> +endif >>>> + >>>> endif >>>> >>>> TraceGeneratedFiles = $(TraceGeneratedNames:%=$(TraceOutDir)/%) >>>> >>>> >>>> /Erik >>> >> > From jesper.wilhelmsson at oracle.com Mon Sep 21 19:46:14 2015 From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson) Date: Mon, 21 Sep 2015 21:46:14 +0200 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F6D878.1070903@oracle.com> References: <55F219CE.3010006@oracle.com> <55F319B8.1000803@oracle.com> <55F32A81.8000307@oracle.com> <55F3E484.70107@oracle.com> <55F6D878.1070903@oracle.com> Message-ID: <56005E86.2080205@oracle.com> Looks good! /Jesper Den 14/9/15 kl. 16:23, skrev gerard ziemski: > Thank you. I have no more comments - reviewed. > > > cheers > > > On 09/12/2015 03:38 AM, sangheon.kim wrote: >> Hi Gerard, >> >> On 09/11/2015 12:24 PM, sangheon.kim wrote: >>> Hi Gerard, >>> >>> Thank you for looking at this. >>> >>> On 09/11/2015 11:13 AM, gerard ziemski wrote: >>>> hi Sangheon, >>>> >>>> #1 test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>> >>>> Please change the comment to: >>>> >>>> + /* >>>> + * Exclude below options as their maximum value would consume too >>>> much memory >>>> + * and would affect other tests that run in parallel. >>>> + */ >>> Okay, I will fix as you suggested. >>> >>>> >>>> >>>> #2 What tests did you run? Did you run >>>> test/runtime/CommandLine/OptionsValidation on all platforms (including >>>> embedded)? >>> No. >>> I ran tests under test/runtime/CommandLine/OptionsValidation (especially >>> TestOptionsWithRanges.java) for all platforms >>> except embedded. >>> Let me back after testing on embedded. >> I ran for embedded (linux-arm64, linux-armvh, linux-armvfpsflt, >> linux-armvfphflt, linux-armsflt) and all of them PASSED >> for test/runtime/CommandLine/OptionsValidation. >> >> Thanks, >> Sangheon >> >> >>> >>> Thanks, >>> Sangheon >>> >>> >>>> >>>> >>>> cheers >>>> >>>> >>>> On 09/10/2015 07:01 PM, sangheon.kim wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch for command-line validation for GC flags. >>>>> This REDO patch is adding ranges and implementing constraint functions for >>>>> GC flags. >>>>> >>>>> Original CR of JDK-8078555 was backout as it made a test failure from >>>>> 'TestOptionsWithRanges.java'. >>>>> And also there were some discussion of OOM handling. >>>>> >>>>> Most parts are same as JDK-8078555 except below: >>>>> 1. Changed 'range' for some flags. >>>>> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags make >>>>> this test unstable as it tries to allocate >>>>> huge amount of memory. >>>>> >>>>> And below are the suggestion note for JDK-8078555: >>>>> 1. Exponential notation for 'double' type variable parse: Previously there >>>>> were some discussion for maximum value for >>>>> double type flags from code review of JDK-8059557 and JDK-8112746. And Kim >>>>> and I decided not to add upper limit unless >>>>> there are problems with DBL_MAX. And as 255 is the maximum length that can >>>>> be passed via command-line, we introduced >>>>> exponential notation to avoid this limit. ( arguments.cpp ) >>>>> 2. These GC flags ranges are not ideal ranges but ranges which don't make >>>>> problem with current source code. >>>>> If one flag makes some problem but hard to find good range, I added >>>>> some ranges. >>>>> 3. There are some constraint functions to avoid overflow. >>>>> 4. Test applications are changed: as some of them assumed to be ParallelGC >>>>> or to check it's output messages. >>>>> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >>>>> >>>>> CR: >>>>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>>>> >>>>> Testing: >>>>> JPRT, UTE(vm.quick-pcl) and >>>>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >>>>> >>>>> Thanks, >>>>> Sangheon >>>>> >>>>> >>> >> >> From kim.barrett at oracle.com Mon Sep 21 21:59:13 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 21 Sep 2015 17:59:13 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F219CE.3010006@oracle.com> References: <55F219CE.3010006@oracle.com> Message-ID: <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> On Sep 10, 2015, at 8:01 PM, sangheon.kim wrote: > > Hi all, > > Please review this patch for command-line validation for GC flags. > This REDO patch is adding ranges and implementing constraint functions for GC flags. > > Original CR of JDK-8078555 was backout as it made a test failure from 'TestOptionsWithRanges.java'. > And also there were some discussion of OOM handling. > > Most parts are same as JDK-8078555 except below: > 1. Changed 'range' for some flags. > 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags make this test unstable as it tries to allocate huge amount of memory. > > And below are the suggestion note for JDK-8078555: > 1. Exponential notation for 'double' type variable parse: Previously there were some discussion for maximum value for double type flags from code review of JDK-8059557 and JDK-8112746. And Kim and I decided not to add upper limit unless there are problems with DBL_MAX. And as 255 is the maximum length that can be passed via command-line, we introduced exponential notation to avoid this limit. ( arguments.cpp ) > 2. These GC flags ranges are not ideal ranges but ranges which don't make problem with current source code. > If one flag makes some problem but hard to find good range, I added some ranges. > 3. There are some constraint functions to avoid overflow. > 4. Test applications are changed: as some of them assumed to be ParallelGC or to check it's output messages. > 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8134995 > > Webrev: > http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ > http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 > > Testing: > JPRT, UTE(vm.quick-pcl) and test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1_globals.hpp 142 product(intx, G1ConcRefinementServiceIntervalMillis, 300, \ ... 145 range(0, max_jint) \ Why is this being changed from max_intx to max_jint? ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1_globals.hpp 169 develop(intx, G1RSetRegionEntriesBase, 256, \ 173 product(intx, G1RSetRegionEntries, 0, \ 179 develop(intx, G1RSetSparseRegionEntriesBase, 4, \ 184 product(intx, G1RSetSparseRegionEntries, 0, \ 240 product(uintx, G1ConcRefinementThreads, 0, \ All of these have their max range value changed to be divided by 4. What's this about? What is this magic "4"? ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1_globals.hpp 240 product(uintx, G1ConcRefinementThreads, 0, \ ... 243 range(0, (max_jint-1)/4) \ I know we discussed the problem of thread counts. I'd suggested perhaps basing the upper bound on the number of processors. (Some care might be needed for uniprocessor systems.) I couldn't find any followup on that suggestion though. Such an approach would probably let us remove this change: test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java 58 allOptionsAsMap.remove("G1ConcRefinementThreads"); Note also that this test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java 52 allOptionsAsMap.remove("CICompilerCount"); is another example of an impossibly huge thread count. ------------------------------------------------------------------------------ From christian.thalinger at oracle.com Mon Sep 21 22:24:23 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 21 Sep 2015 12:24:23 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> Message-ID: <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> > On Sep 18, 2015, at 2:19 PM, Christian Thalinger wrote: > > >> On Sep 18, 2015, at 7:00 AM, Christian Thalinger wrote: >> >> >>> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >>> >>> To which repository will you finally push these changes? >>> >>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>> they will be integrated there. >> >> Good point. >> >>> >>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>> because of '8135067: Preparatory refactorings for compiler control' >>> and "8134626: Misc cleanups after generation array removal": >>> >>> 3 out of 18 hunks FAILED -- saving rejects to file >>> src/share/vm/compiler/compileBroker.cpp.rej >>> 1 out of 5 hunks FAILED -- saving rejects to file >>> src/share/vm/compiler/compileBroker.hpp.rej >>> 6 out of 40 hunks FAILED -- saving rejects to file >>> src/share/vm/runtime/vmStructs.cpp.rej >>> >>> Will you update the patches and if yes against which repositories? >> >> Yes. Let me update the graal-jvmci-9 repository to hs-comp. > > Done. Now I have to create new webrevs? Here are new webrevs against hs-comp: http://cr.openjdk.java.net/~twisti/8136421/webrev/ http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ > >> >>> >>> If I want to work on the ppc64 implementation, which repository should I use? >> >> graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? >> >>> >>> Thank you and best regards, >>> Volker >>> >>> >>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>> wrote: >>>> For the top-level change I always get a strange error when importing it: >>>> >>>> patch failed, unable to continue (try -v) >>>> patch failed, rejects left in working dir >>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>> >>>> It is because of the strange path syntax of the last hunk in the patch file: >>>> >>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>> @@ -254,6 +254,14 @@ >>>> jdk.jfr >>>> >>>> >>>> + jdk.internal.jvmci.hotspot >>>> + jdk.jfr >>>> + >>>> + >>>> + jdk.internal.jvmci.hotspot.events >>>> + jdk.jfr >>>> + >>>> + >>>> sun.misc >>>> java.corba >>>> java.desktop >>>> >>>> >>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>> If I remove the redundant './' from the path, everything works fine. >>>> For some reason only the diffs for modules.xml has this strange path >>>> syntax in the patch. >>>> >>>> Regards, >>>> Volker >>>> >>>> >>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>> wrote: >>>>> Since there are no objections I?m going to push this? >>>>> >>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>> >>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>> >>>>>> I would like to add this change: >>>>>> >>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>> Abstract_VM_Version::vm_release(), >>>>>> Abstract_VM_Version::vm_info_string(), >>>>>> TieredCompilation ? ", tiered" : "", >>>>>> +#if INCLUDE_JVMCI >>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>> +#endif >>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>> gc_mode(), >>>>>> Abstract_VM_Version::vm_platform_string() >>>>>> >>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>> >>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>> >>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>> >>>>>>> Vladimir >>>>>>> >>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>> >>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>> >>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>> >>>>>>>>>>> Here are the webrevs: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>> >>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>> >>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>> >>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>> >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>> >>>>>>>>>> Hi Christian, >>>>>>>>>> >>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>> >>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>> >>>>>>>>>> Some comments: >>>>>>>>>> >>>>>>>>>> * modules.xml: >>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>> >>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>> >>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>> >>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>> >>>>>>>>>> >>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>> >>>>>>>>> Thanks for the heads up. >>>>>>>> >>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>> >>>>>>>>> Ok, thanks. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> /Magnus >>>>>> >>>>> >> > From sangheon.kim at oracle.com Mon Sep 21 22:39:59 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Mon, 21 Sep 2015 15:39:59 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <56005E86.2080205@oracle.com> References: <55F219CE.3010006@oracle.com> <55F319B8.1000803@oracle.com> <55F32A81.8000307@oracle.com> <55F3E484.70107@oracle.com> <55F6D878.1070903@oracle.com> <56005E86.2080205@oracle.com> Message-ID: <5600873F.1060409@oracle.com> Hi Jesper, Thanks for reviewing this. Sangheon On 09/21/2015 12:46 PM, Jesper Wilhelmsson wrote: > Looks good! > /Jesper > > Den 14/9/15 kl. 16:23, skrev gerard ziemski: >> Thank you. I have no more comments - reviewed. >> >> >> cheers >> >> >> On 09/12/2015 03:38 AM, sangheon.kim wrote: >>> Hi Gerard, >>> >>> On 09/11/2015 12:24 PM, sangheon.kim wrote: >>>> Hi Gerard, >>>> >>>> Thank you for looking at this. >>>> >>>> On 09/11/2015 11:13 AM, gerard ziemski wrote: >>>>> hi Sangheon, >>>>> >>>>> #1 >>>>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>> >>>>> Please change the comment to: >>>>> >>>>> + /* >>>>> + * Exclude below options as their maximum value would >>>>> consume too >>>>> much memory >>>>> + * and would affect other tests that run in parallel. >>>>> + */ >>>> Okay, I will fix as you suggested. >>>> >>>>> >>>>> >>>>> #2 What tests did you run? Did you run >>>>> test/runtime/CommandLine/OptionsValidation on all platforms >>>>> (including >>>>> embedded)? >>>> No. >>>> I ran tests under test/runtime/CommandLine/OptionsValidation >>>> (especially >>>> TestOptionsWithRanges.java) for all platforms >>>> except embedded. >>>> Let me back after testing on embedded. >>> I ran for embedded (linux-arm64, linux-armvh, linux-armvfpsflt, >>> linux-armvfphflt, linux-armsflt) and all of them PASSED >>> for test/runtime/CommandLine/OptionsValidation. >>> >>> Thanks, >>> Sangheon >>> >>> >>>> >>>> Thanks, >>>> Sangheon >>>> >>>> >>>>> >>>>> >>>>> cheers >>>>> >>>>> >>>>> On 09/10/2015 07:01 PM, sangheon.kim wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this patch for command-line validation for GC flags. >>>>>> This REDO patch is adding ranges and implementing constraint >>>>>> functions for >>>>>> GC flags. >>>>>> >>>>>> Original CR of JDK-8078555 was backout as it made a test failure >>>>>> from >>>>>> 'TestOptionsWithRanges.java'. >>>>>> And also there were some discussion of OOM handling. >>>>>> >>>>>> Most parts are same as JDK-8078555 except below: >>>>>> 1. Changed 'range' for some flags. >>>>>> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These >>>>>> flags make >>>>>> this test unstable as it tries to allocate >>>>>> huge amount of memory. >>>>>> >>>>>> And below are the suggestion note for JDK-8078555: >>>>>> 1. Exponential notation for 'double' type variable parse: >>>>>> Previously there >>>>>> were some discussion for maximum value for >>>>>> double type flags from code review of JDK-8059557 and >>>>>> JDK-8112746. And Kim >>>>>> and I decided not to add upper limit unless >>>>>> there are problems with DBL_MAX. And as 255 is the maximum length >>>>>> that can >>>>>> be passed via command-line, we introduced >>>>>> exponential notation to avoid this limit. ( arguments.cpp ) >>>>>> 2. These GC flags ranges are not ideal ranges but ranges which >>>>>> don't make >>>>>> problem with current source code. >>>>>> If one flag makes some problem but hard to find good range, >>>>>> I added >>>>>> some ranges. >>>>>> 3. There are some constraint functions to avoid overflow. >>>>>> 4. Test applications are changed: as some of them assumed to be >>>>>> ParallelGC >>>>>> or to check it's output messages. >>>>>> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >>>>>> >>>>>> CR: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>>>>> >>>>>> Testing: >>>>>> JPRT, UTE(vm.quick-pcl) and >>>>>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Sangheon >>>>>> >>>>>> >>>> >>> >>> From sangheon.kim at oracle.com Mon Sep 21 23:05:31 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Mon, 21 Sep 2015 16:05:31 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> Message-ID: <56008D3B.5010008@oracle.com> Hi Kim, Thanks for looking at this. On 09/21/2015 02:59 PM, Kim Barrett wrote: > On Sep 10, 2015, at 8:01 PM, sangheon.kim wrote: >> Hi all, >> >> Please review this patch for command-line validation for GC flags. >> This REDO patch is adding ranges and implementing constraint functions for GC flags. >> >> Original CR of JDK-8078555 was backout as it made a test failure from 'TestOptionsWithRanges.java'. >> And also there were some discussion of OOM handling. >> >> Most parts are same as JDK-8078555 except below: >> 1. Changed 'range' for some flags. >> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags make this test unstable as it tries to allocate huge amount of memory. >> >> And below are the suggestion note for JDK-8078555: >> 1. Exponential notation for 'double' type variable parse: Previously there were some discussion for maximum value for double type flags from code review of JDK-8059557 and JDK-8112746. And Kim and I decided not to add upper limit unless there are problems with DBL_MAX. And as 255 is the maximum length that can be passed via command-line, we introduced exponential notation to avoid this limit. ( arguments.cpp ) >> 2. These GC flags ranges are not ideal ranges but ranges which don't make problem with current source code. >> If one flag makes some problem but hard to find good range, I added some ranges. >> 3. There are some constraint functions to avoid overflow. >> 4. Test applications are changed: as some of them assumed to be ParallelGC or to check it's output messages. >> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8134995 >> >> Webrev: >> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >> >> Testing: >> JPRT, UTE(vm.quick-pcl) and test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1_globals.hpp > 142 product(intx, G1ConcRefinementServiceIntervalMillis, 300, \ > ... > 145 range(0, max_jint) \ > > Why is this being changed from max_intx to max_jint? This option is used for 'long' type parameter and it is 32bit for Windows. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1_globals.hpp > 169 develop(intx, G1RSetRegionEntriesBase, 256, \ > 173 product(intx, G1RSetRegionEntries, 0, \ > 179 develop(intx, G1RSetSparseRegionEntriesBase, 4, \ > 184 product(intx, G1RSetSparseRegionEntries, 0, \ > 240 product(uintx, G1ConcRefinementThreads, 0, \ > > All of these have their max range value changed to be divided by 4. > What's this about? What is this magic "4"? 'G1RSetRegionEntries', 'G1RSetSparseRegionEntries' and 'G1ConcRefinementThreads' are finally multiplied by pointer size. So we need to divide by 4 for 32bit to avoid overflow. We don't need for 64bit but I thought it will be enough for 64bit. 'G1RSetRegionEntriesBase' and 'G1RSetSparseRegionEntriesBase' changed to have same range as 'G1RSetRegionEntries' and 'G1RSetSparseRegionEntries'. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1_globals.hpp > 240 product(uintx, G1ConcRefinementThreads, 0, \ > ... > 243 range(0, (max_jint-1)/4) \ > > I know we discussed the problem of thread counts. I'd suggested > perhaps basing the upper bound on the number of processors. (Some > care might be needed for uniprocessor systems.) I couldn't find any > followup on that suggestion though. Firstly I agreed to your opinion of using the number of processors if we need to give upper bound. But from the email thread that we discussed, I ended not to handle OOM from our range/constraint validation framework. Dmitry told us that testing framework is okay for OOM(only for exit value of 1) and Gerard also agreed not to handle it from validation. Finally, I think the only thing that we need for this option is just excluding this test. If we don't exclude tests that are using too many resources, it would make problem to other tests that running in parallel. So it would be better to exclude these tests. Thanks, Sangheon > Such an approach would probably > let us remove this change: > > test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java > 58 allOptionsAsMap.remove("G1ConcRefinementThreads"); > > Note also that this > test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java > 52 allOptionsAsMap.remove("CICompilerCount"); > is another example of an impossibly huge thread count. > > ------------------------------------------------------------------------------ > From igor.veresov at oracle.com Mon Sep 21 23:48:54 2015 From: igor.veresov at oracle.com (Igor Veresov) Date: Mon, 21 Sep 2015 16:48:54 -0700 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> Message-ID: I?ve seen it all before, but I went through it one more time in its entirety and it looks good. igor > On Sep 21, 2015, at 3:24 PM, Christian Thalinger wrote: > >> > > Here are new webrevs against hs-comp: > > http://cr.openjdk.java.net/~twisti/8136421/webrev/ > > http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ > > From christian.thalinger at oracle.com Mon Sep 21 23:52:22 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 21 Sep 2015 13:52:22 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> Message-ID: Thank you, Igor. > On Sep 21, 2015, at 1:48 PM, Igor Veresov wrote: > > I?ve seen it all before, but I went through it one more time in its entirety and it looks good. > > igor > >> On Sep 21, 2015, at 3:24 PM, Christian Thalinger wrote: >> >>> >> >> Here are new webrevs against hs-comp: >> >> http://cr.openjdk.java.net/~twisti/8136421/webrev/ > >> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ > >> From christian.tornqvist at oracle.com Tue Sep 22 00:36:02 2015 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Mon, 21 Sep 2015 20:36:02 -0400 Subject: RFR(S): 8027565 - Enable /d2Zi+ when building with Visual Studio 2013 Message-ID: <01e701d0f4ce$a97d8400$fc788c00$@oracle.com> Hi everyone, Please review this small change that enables generation of enhanced debug symbols for optimized code on Windows. The switch is available as /d2Zi+ prior to Update 3, when it was officially added as /Zo. Both /Zo and /d2Zi+ works with VS2013 Update 3 and later. More information about this switch can be found at https://msdn.microsoft.com/en-us/library/dn785163.aspx I've verified the change by building locally with VS2013 Update 4 and with JPRT that has the older VS2013 (non-update) version. Webrev: http://cr.openjdk.java.net/~ctornqvi/webrev/8027565/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8027565 Thanks, Christian From magnus.ihse.bursie at oracle.com Tue Sep 22 09:05:36 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 22 Sep 2015 11:05:36 +0200 Subject: RFR(S): 8027565 - Enable /d2Zi+ when building with Visual Studio 2013 In-Reply-To: <01e701d0f4ce$a97d8400$fc788c00$@oracle.com> References: <01e701d0f4ce$a97d8400$fc788c00$@oracle.com> Message-ID: <560119E0.5090902@oracle.com> On 2015-09-22 02:36, Christian Tornqvist wrote: > Hi everyone, > > > Please review this small change that enables generation of enhanced debug > symbols for optimized code on Windows. The switch is available as /d2Zi+ > prior to Update 3, when it was officially added as /Zo. Both /Zo and /d2Zi+ > works with VS2013 Update 3 and later. More information about this switch can > be found at https://msdn.microsoft.com/en-us/library/dn785163.aspx Christian, When was the /d2Zi+ switch introduced? /Magnus > > > > I've verified the change by building locally with VS2013 Update 4 and with > JPRT that has the older VS2013 (non-update) version. > > > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8027565/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8027565 > > > > Thanks, > > Christian > From vladimir.kozlov at oracle.com Tue Sep 22 11:09:49 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 22 Sep 2015 19:09:49 +0800 Subject: RFR (S): 8136672: editor configuration for IDEs In-Reply-To: <3616187E21868C40AD1B36D41D29F4C136905BEE@FMSMSX106.amr.corp.intel.com> References: <3616187E21868C40AD1B36D41D29F4C136905BEE@FMSMSX106.amr.corp.intel.com> Message-ID: <560136FD.4080909@oracle.com> I am fine with this addition. Ask for sponsor. Thanks, Vladimir On 9/19/15 5:59 AM, Wojtowicz, Tomasz wrote: > Review Title: editor configuration for IDEs > Review ID: #8136672 > Diff: http://cr.openjdk.java.net/~mcberg/8136672/webrev.01 > Description: In order to avoid jcheck errors like following ones: > "Tests found: > > src/cpu/x86/vm/macroAssembler_x86.cpp:8686: Trailing whitespace > src/cpu/x86/vm/macroAssembler_x86.hpp:1330: Trailing whitespace > src/cpu/x86/vm/stubGenerator_x86_32.cpp:3001: Trailing whitespace > src/cpu/x86/vm/stubGenerator_x86_64.cpp:3968: Trailing whitespace > src/cpu/x86/vm/stubRoutines_x86.cpp:136: Trailing whitespace > src/share/vm/interpreter/interpreter.cpp:594: Carriage return (^M) > src/share/vm/interpreter/templateInterpreter.cpp:426: Carriage return > (^M) > src/cpu/x86/vm/crc32c.h:31: Trailing whitespace" > > I propose to introduce an .editorconfig file which would encapsulate common layout required settings. EditorConfig (http://editorconfig.org/) provides plugins for most popular editors/IDEs and make sure that every modification adheres to the stated rules: > hotspot/.editorconfig file contents > root = true > > [*] > indent_style = space > indent_size = 2 > end_of_line = lf > charset = utf-8 > trim_trailing_whitespace = true > insert_final_newline = true > Link: https://bugs.openjdk.java.net/browse/JDK-8136672 > Author: Tomasz, Wojtowicz > > -- > Thank you, > > Tomek > From roland.westrelin at oracle.com Tue Sep 22 11:12:12 2015 From: roland.westrelin at oracle.com (Roland Westrelin) Date: Tue, 22 Sep 2015 13:12:12 +0200 Subject: 8136165: AARCH64: Tidy up compiled native calls In-Reply-To: <55FC0C8D.6050508@redhat.com> References: <55F1AA57.70307@redhat.com> <55F1BB94.4080408@redhat.com> <55FC0C8D.6050508@redhat.com> Message-ID: <57FE21CF-5FBE-4BBF-8065-30DA33130DC7@oracle.com> > New patch at http://cr.openjdk.java.net/~aph/8136165-2 That looks ok to me but it would be better if someone working on the AARCH64 port took a look as well. Roland. From vladimir.kozlov at oracle.com Tue Sep 22 11:13:53 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 22 Sep 2015 19:13:53 +0800 Subject: 8136165: AARCH64: Tidy up compiled native calls In-Reply-To: <55FC0C8D.6050508@redhat.com> References: <55F1AA57.70307@redhat.com> <55F1BB94.4080408@redhat.com> <55FC0C8D.6050508@redhat.com> Message-ID: <560137F1.50309@oracle.com> Looks fine to me. Thanks, Vladimir On 9/18/15 9:07 PM, Andrew Haley wrote: > New patch at http://cr.openjdk.java.net/~aph/8136165-2 > > Thanks, > Andrew. > From david.lindholm at oracle.com Tue Sep 22 11:38:52 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Tue, 22 Sep 2015 13:38:52 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends Message-ID: <56013DCC.4060307@oracle.com> Hi, Please review the following patch which removes the need to use err_msg() or err_msg_res() to format strings that are sent to assert(), guarantee(), fatal() and vm_exit_out_of_memory(). These are now variadic macros. So, guarantee(_saved_index == _index, err_msg("saved index: %d index: %d", _saved_index, _index)); is replaced by guarantee(_saved_index == _index, "saved index: %d index: %d", _saved_index, _index); This change also eliminates the need to choose if the buffer for the formatted string should be allocated on the stack (with err_msg()) or in the resource area (with err_msg_res()). There is no longer any need to allocate this extra buffer, since the va_list is sent all the way down to the functions writing the error to screen or file. To accommodate for this, VMError is now AllStatic. All patterns of usages of this class was to create a VMError and immediately call report_and_die() on it. Now, instead of having multiple constructors, there are multiple static VMError::report_and_die(). I have also removed all usages of err_msg() and err_msg_res() in assert(), guarantee(), fatal() and vm_exit_out_of_memory() in this patch. err_msg_res() is completely removed, err_msg() is still used by other functions. Since we have about 1000 asserts with "" as detailed message, I had to disable the gcc-only warning for empty format strings. (As a side note, there is also a JBS bug that describes a name change from assert to vmassert in our code. This changeset does not include such a name change, since we have over 19000 asserts in our code. We still have a "#define assert vmassert" just like before). Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ Testing: Passed JPRT Thanks, David From daniel.daugherty at oracle.com Tue Sep 22 13:04:00 2015 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 22 Sep 2015 07:04:00 -0600 Subject: RFR(S): 8027565 - Enable /d2Zi+ when building with Visual Studio 2013 In-Reply-To: <560119E0.5090902@oracle.com> References: <01e701d0f4ce$a97d8400$fc788c00$@oracle.com> <560119E0.5090902@oracle.com> Message-ID: <560151C0.5080000@oracle.com> On 9/22/15 3:05 AM, Magnus Ihse Bursie wrote: > On 2015-09-22 02:36, Christian Tornqvist wrote: >> Hi everyone, >> >> >> Please review this small change that enables generation of enhanced >> debug >> symbols for optimized code on Windows. The switch is available as /d2Zi+ >> prior to Update 3, when it was officially added as /Zo. Both /Zo and >> /d2Zi+ >> works with VS2013 Update 3 and later. More information about this >> switch can >> be found at https://msdn.microsoft.com/en-us/library/dn785163.aspx > > Christian, > > When was the /d2Zi+ switch introduced? In the bug report: This is an undocumented feature in VS2012 but should be fully supported in VS2013 once it's released. > > /Magnus > >> >> >> I've verified the change by building locally with VS2013 Update 4 and >> with >> JPRT that has the older VS2013 (non-update) version. >> >> >> Webrev: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8027565/webrev.00/ make/windows/makefiles/compile.make No comments. Thumbs up. Dan >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8027565 >> >> >> Thanks, >> >> Christian >> > > From christian.tornqvist at oracle.com Tue Sep 22 14:40:39 2015 From: christian.tornqvist at oracle.com (Christian Tornqvist) Date: Tue, 22 Sep 2015 10:40:39 -0400 Subject: RFR(S): 8027565 - Enable /d2Zi+ when building with Visual Studio 2013 In-Reply-To: <560119E0.5090902@oracle.com> References: <01e701d0f4ce$a97d8400$fc788c00$@oracle.com> <560119E0.5090902@oracle.com> Message-ID: <027d01d0f544$a71951a0$f54bf4e0$@oracle.com> Hi Magnus, > When was the /d2Zi+ switch introduced? As Dan pointed out, it was introduced as an undocumented switch in VS2012 and was planned to be an official switch in VS2013. The official switch was introduced in VS2013 Update 3 and in VS2015 this switch is on by default. Thanks, Christian -----Original Message----- From: Magnus Ihse Bursie [mailto:magnus.ihse.bursie at oracle.com] Sent: Tuesday, September 22, 2015 5:06 AM To: Christian Tornqvist ; hotspot-dev at openjdk.java.net Subject: Re: RFR(S): 8027565 - Enable /d2Zi+ when building with Visual Studio 2013 On 2015-09-22 02:36, Christian Tornqvist wrote: > Hi everyone, > > > Please review this small change that enables generation of enhanced > debug symbols for optimized code on Windows. The switch is available > as /d2Zi+ prior to Update 3, when it was officially added as /Zo. Both > /Zo and /d2Zi+ works with VS2013 Update 3 and later. More information > about this switch can be found at > https://msdn.microsoft.com/en-us/library/dn785163.aspx Christian, When was the /d2Zi+ switch introduced? /Magnus > > > > I've verified the change by building locally with VS2013 Update 4 and > with JPRT that has the older VS2013 (non-update) version. > > > > Webrev: > > http://cr.openjdk.java.net/~ctornqvi/webrev/8027565/webrev.00/ > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8027565 > > > > Thanks, > > Christian > From christian.thalinger at oracle.com Tue Sep 22 16:31:00 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 22 Sep 2015 06:31:00 -1000 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <56013DCC.4060307@oracle.com> References: <56013DCC.4060307@oracle.com> Message-ID: <9F5C5634-B5F0-4D1E-88F6-C9C4DA77C645@oracle.com> > On Sep 22, 2015, at 1:38 AM, David Lindholm wrote: > > Hi, > > Please review the following patch which removes the need to use err_msg() or err_msg_res() to format strings that are sent to assert(), guarantee(), fatal() and vm_exit_out_of_memory(). These are now variadic macros. So, > > guarantee(_saved_index == _index, err_msg("saved index: %d index: %d", _saved_index, _index)); > > is replaced by > > guarantee(_saved_index == _index, "saved index: %d index: %d", _saved_index, _index); Excellent change! > > This change also eliminates the need to choose if the buffer for the formatted string should be allocated on the stack (with err_msg()) or in the resource area (with err_msg_res()). There is no longer any need to allocate this extra buffer, since the va_list is sent all the way down to the functions writing the error to screen or file. I?m not exactly sure but we might have ResourceMarks in the code just to have err_msg_res. Did you see any cases like this? Also, does this mean there are no buffers allocated on the stack to format the message anymore? The reason we introduced err_msg_res was because Solaris Studio produced some bad code on SPARC which blew up the stack. > > To accommodate for this, VMError is now AllStatic. All patterns of usages of this class was to create a VMError and immediately call report_and_die() on it. > > Now, instead of having multiple constructors, there are multiple static VMError::report_and_die(). > > I have also removed all usages of err_msg() and err_msg_res() in assert(), guarantee(), fatal() and vm_exit_out_of_memory() in this patch. err_msg_res() is completely removed, err_msg() is still used by other functions. > > Since we have about 1000 asserts with "" as detailed message, I had to disable the gcc-only warning for empty format strings. > > (As a side note, there is also a JBS bug that describes a name change from assert to vmassert in our code. This changeset does not include such a name change, since we have over 19000 asserts in our code. We still have a "#define assert vmassert" just like before). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 > Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ > > > Testing: Passed JPRT > > > Thanks, > David > From rachel.protacio at oracle.com Tue Sep 22 18:55:22 2015 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Tue, 22 Sep 2015 14:55:22 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55ED9210.6030904@oracle.com> References: <55ED9210.6030904@oracle.com> Message-ID: <5601A41A.1080006@oracle.com> Hi, I believe that utilities/ostream.cpp#logStream::write() needs to have a line at the end update_position(s, len); As it stands, the code updates the _position variable that belongs to the stringStream _current_line (via the _current_line.write() function), but the logStream _position does not get updated. This was causing problems for me as I tried to make use of the outputStream::indent() function. Thanks, Rachel On 9/7/2015 9:33 AM, Marcus Larsson wrote: > Hi, > > Please review the following patch adding the unified logging framework > to hotspot. > > JEP: > https://bugs.openjdk.java.net/browse/JDK-8046148 > > Webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ > > See the JEP description for a general overview of the new feature. > Below are some notes on the implementation. > > The patch adds the new 'share/vm/logging' subdirectory containing the > unified logging framework. The main entry point is log.hpp, which > contains the necessary macros and definitions to use the framework. > > Log tags are defined/listed in logTag.hpp, and are passed as template > arguments to the Log class. Every combination of tags used in a log > call has a corresponding LogTagSet instance, which keeps a track of > all the outputs it should write the log message to (and their levels). > Having tags as template arguments allows mapping directly from a set > of tags to the LogTagSet instance, which means that the overhead for > disabled logging should be low. Currently each log message can be > tagged with up to 5 tags, but this can be increased if ever required > (and with C++11's variadic templates the limit can be removed > completely). > > The LogConfiguration class keeps track of configured outputs (stdout, > stderr, and possible file outputs). Configuration is done either by > command line arguments (-Xlog) or by DCMD. Both methods will in turn > use the LogConfiguration class to perform the parsing & configuration. > This configuration includes iterating over all LogTagSet instances and > updating them accordingly. The LogTagLevelExpression class is used to > represent the selection of tags and levels for a given configuration > request (the "what"-expression). > > The LogDecorators class contains a selection of decorators. Instances > of this class is kept in LogTagSet to track what decorators to use > (this is the union of all decorators used by its outputs). Each log > call will create a LogDecorations instance (note: different classes), > which will contain the actual decoration strings for the log message. > These decorations are used for each output the tagset is set to log > on, and are then discarded. > > The LogPrefix class allows messages of specific sets of tags to be > prefixed. The prefix should supply a printf-style format with > argument. (This allows GC logging to prefix messages of certain > tagsets with GCId.) Prefixes are implemented using template > specializations based on the specified tags, with the > general/unspecialized case giving an empty prefix. > > The LogOutput class defines the interface for all types of log > outputs. LogFileStreamOutput corresponds to FILE* stream based log > outputs. LogFileOutput builds on this and adds the file management and > log rotation support. > > A simple jtreg test is included in this patch. Additional tests will > be added at a later stage. > > Thanks, > Marcus From jon.masamitsu at oracle.com Tue Sep 22 23:04:51 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 22 Sep 2015 16:04:51 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <56008D3B.5010008@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> Message-ID: <5601DE93.2070802@oracle.com> On 9/21/2015 4:05 PM, sangheon.kim wrote: > Hi Kim, > > Thanks for looking at this. > > On 09/21/2015 02:59 PM, Kim Barrett wrote: >> On Sep 10, 2015, at 8:01 PM, sangheon.kim >> wrote: >>> Hi all, >>> >>> Please review this patch for command-line validation for GC flags. >>> This REDO patch is adding ranges and implementing constraint >>> functions for GC flags. >>> >>> Original CR of JDK-8078555 was backout as it made a test failure >>> from 'TestOptionsWithRanges.java'. >>> And also there were some discussion of OOM handling. >>> >>> Most parts are same as JDK-8078555 except below: >>> 1. Changed 'range' for some flags. >>> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags >>> make this test unstable as it tries to allocate huge amount of memory. >>> >>> And below are the suggestion note for JDK-8078555: >>> 1. Exponential notation for 'double' type variable parse: Previously >>> there were some discussion for maximum value for double type flags >>> from code review of JDK-8059557 and JDK-8112746. And Kim and I >>> decided not to add upper limit unless there are problems with >>> DBL_MAX. And as 255 is the maximum length that can be passed via >>> command-line, we introduced exponential notation to avoid this >>> limit. ( arguments.cpp ) >>> 2. These GC flags ranges are not ideal ranges but ranges which don't >>> make problem with current source code. >>> If one flag makes some problem but hard to find good range, I >>> added some ranges. >>> 3. There are some constraint functions to avoid overflow. >>> 4. Test applications are changed: as some of them assumed to be >>> ParallelGC or to check it's output messages. >>> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>> >>> Testing: >>> JPRT, UTE(vm.quick-pcl) and >>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1_globals.hpp >> 142 product(intx, G1ConcRefinementServiceIntervalMillis, >> 300, \ >> ... >> 145 range(0, >> max_jint) \ >> >> Why is this being changed from max_intx to max_jint? > This option is used for 'long' type parameter and it is 32bit for > Windows. > >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1_globals.hpp >> 169 develop(intx, G1RSetRegionEntriesBase, >> 256, \ >> 173 product(intx, G1RSetRegionEntries, >> 0, \ >> 179 develop(intx, G1RSetSparseRegionEntriesBase, >> 4, \ >> 184 product(intx, G1RSetSparseRegionEntries, >> 0, \ >> 240 product(uintx, G1ConcRefinementThreads, >> 0, \ >> >> All of these have their max range value changed to be divided by 4. >> What's this about? What is this magic "4"? > 'G1RSetRegionEntries', 'G1RSetSparseRegionEntries' and > 'G1ConcRefinementThreads' are finally multiplied by pointer size. > So we need to divide by 4 for 32bit to avoid overflow. We don't need > for 64bit but I thought it will be enough for 64bit. > 'G1RSetRegionEntriesBase' and 'G1RSetSparseRegionEntriesBase' changed > to have same range as 'G1RSetRegionEntries' and > 'G1RSetSparseRegionEntries'. Sangheon, I think your suggestion (when we talked today) about dividing by jintSize should work. Kim, There are more logical maximum values that could be reverse-engineered from the implementation but I thought to keep it simple. > >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1_globals.hpp >> 240 product(uintx, G1ConcRefinementThreads, >> 0, \ >> ... >> 243 range(0, >> (max_jint-1)/4) \ >> >> I know we discussed the problem of thread counts. I'd suggested >> perhaps basing the upper bound on the number of processors. (Some >> care might be needed for uniprocessor systems.) I couldn't find any >> followup on that suggestion though. Kim, My apologies but I dissuaded Sangheon from that idea. I don't think the number of processors was enough. A hundred or a thousand times the number of processors should be enough but if I had to pick some multiplier like that, I'd rather pick something limited by the size of the type and back off some. It's still arbitrary but whereas I might some decade be surprised that 1000 * #-of-processors was too few, I'm not going to see 1,000,000,000 be too few. In both these situations if I had to explain myself, I'd rather says something about arithmetic overflow than that I just thought it was big enough. Jon > Firstly I agreed to your opinion of using the number of processors if > we need to give upper bound. > But from the email thread that we discussed, I ended not to handle OOM > from our range/constraint validation framework. > Dmitry told us that testing framework is okay for OOM(only for exit > value of 1) and Gerard also agreed not to handle it from validation. > Finally, I think the only thing that we need for this option is just > excluding this test. > > If we don't exclude tests that are using too many resources, it would > make problem to other tests that running in parallel. > So it would be better to exclude these tests. > > Thanks, > Sangheon > > >> Such an approach would probably >> let us remove this change: >> >> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >> 58 allOptionsAsMap.remove("G1ConcRefinementThreads"); >> >> Note also that this >> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >> 52 allOptionsAsMap.remove("CICompilerCount"); >> is another example of an impossibly huge thread count. >> ------------------------------------------------------------------------------ >> > From christian.thalinger at oracle.com Tue Sep 22 23:13:06 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 22 Sep 2015 13:13:06 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> Message-ID: We decided to add one more change: the way JVMCI options are named and passed. Instead of one option: -Djvmci.options= we now use multiple options in the form of: -Djvmci.option.= Here are the two changesets implementing this: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/0569806cf96f http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/bb2afad00ef1 (the second one is only a cleanup, really). No more source code changes after this, I promise. > On Sep 21, 2015, at 12:24 PM, Christian Thalinger wrote: > > >> On Sep 18, 2015, at 2:19 PM, Christian Thalinger wrote: >> >> >>> On Sep 18, 2015, at 7:00 AM, Christian Thalinger wrote: >>> >>> >>>> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >>>> >>>> To which repository will you finally push these changes? >>>> >>>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>>> they will be integrated there. >>> >>> Good point. >>> >>>> >>>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>>> because of '8135067: Preparatory refactorings for compiler control' >>>> and "8134626: Misc cleanups after generation array removal": >>>> >>>> 3 out of 18 hunks FAILED -- saving rejects to file >>>> src/share/vm/compiler/compileBroker.cpp.rej >>>> 1 out of 5 hunks FAILED -- saving rejects to file >>>> src/share/vm/compiler/compileBroker.hpp.rej >>>> 6 out of 40 hunks FAILED -- saving rejects to file >>>> src/share/vm/runtime/vmStructs.cpp.rej >>>> >>>> Will you update the patches and if yes against which repositories? >>> >>> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >> >> Done. Now I have to create new webrevs? > > Here are new webrevs against hs-comp: > > http://cr.openjdk.java.net/~twisti/8136421/webrev/ > http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ > >> >>> >>>> >>>> If I want to work on the ppc64 implementation, which repository should I use? >>> >>> graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? >>> >>>> >>>> Thank you and best regards, >>>> Volker >>>> >>>> >>>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>>> wrote: >>>>> For the top-level change I always get a strange error when importing it: >>>>> >>>>> patch failed, unable to continue (try -v) >>>>> patch failed, rejects left in working dir >>>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>>> >>>>> It is because of the strange path syntax of the last hunk in the patch file: >>>>> >>>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>> @@ -254,6 +254,14 @@ >>>>> jdk.jfr >>>>> >>>>> >>>>> + jdk.internal.jvmci.hotspot >>>>> + jdk.jfr >>>>> + >>>>> + >>>>> + jdk.internal.jvmci.hotspot.events >>>>> + jdk.jfr >>>>> + >>>>> + >>>>> sun.misc >>>>> java.corba >>>>> java.desktop >>>>> >>>>> >>>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>>> If I remove the redundant './' from the path, everything works fine. >>>>> For some reason only the diffs for modules.xml has this strange path >>>>> syntax in the patch. >>>>> >>>>> Regards, >>>>> Volker >>>>> >>>>> >>>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>>> wrote: >>>>>> Since there are no objections I?m going to push this? >>>>>> >>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>>> >>>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>>> >>>>>>> I would like to add this change: >>>>>>> >>>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>>> Abstract_VM_Version::vm_release(), >>>>>>> Abstract_VM_Version::vm_info_string(), >>>>>>> TieredCompilation ? ", tiered" : "", >>>>>>> +#if INCLUDE_JVMCI >>>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>>> +#endif >>>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>>> gc_mode(), >>>>>>> Abstract_VM_Version::vm_platform_string() >>>>>>> >>>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>>> >>>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>>> >>>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>>> >>>>>>>> Vladimir >>>>>>>> >>>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>>> >>>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>>> >>>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>>> >>>>>>>>>>>> Here are the webrevs: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>>> >>>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>>> >>>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>>> >>>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>>> >>>>>>>>>>> Hi Christian, >>>>>>>>>>> >>>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>>> >>>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>>> >>>>>>>>>>> Some comments: >>>>>>>>>>> >>>>>>>>>>> * modules.xml: >>>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>>> >>>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>>> >>>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>>> >>>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>>> >>>>>>>>>> Thanks for the heads up. >>>>>>>>> >>>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>>> >>>>>>>>>> Ok, thanks. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> /Magnus >>>>>>> >>>>>> >>> >> > From kim.barrett at oracle.com Tue Sep 22 23:34:58 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 22 Sep 2015 19:34:58 -0400 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <56013DCC.4060307@oracle.com> References: <56013DCC.4060307@oracle.com> Message-ID: <62694939-B653-4838-8DCA-06A0CE5E16F9@oracle.com> On Sep 22, 2015, at 7:38 AM, David Lindholm wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 > Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ I've run out of time to look at this today, so here's what I've got so far. Looks good so far, just a few minor things. [I've reviewed the core changes (debug.[ch]pp and vmError.[ch]pp), along with all of the rest of the utilities directory (at the end of the webrev file list), and from the top of the webrev list through the compiler directory, so maybe 1/3 of the way through.] I'll try to finish in the next day or two. ------------------------------------------------------------------------------ src/share/vm/utilities/debug.hpp 43 #define BUFSZ 256 Why rename RES_BUFSZ to BUFSZ? Both, but particularly the latter, seem like poorly chosen macro names in a widely used header. Could we instead have a static constant in FormatBufferBase? ------------------------------------------------------------------------------ src/share/vm/utilities/debug.hpp 108 // Used to format messages Lost period at end of comment. ------------------------------------------------------------------------------ make/linux/makefiles/gcc.make > Since we have about 1000 asserts with "" as detailed message, I had to > disable the gcc-only warning for empty format strings. And they aren't even all that concentrated. At least 141 files in at least 26 directories. Bleh! I guess there's not much choice here. But a bug report seems called for. ------------------------------------------------------------------------------ There are a number of eliminations of err_msg_res from places where the associated ResourceMark is not at all obvious. Since we're dealing with assertions and the like, so that we're on our way to process termination, that probably doesn't matter in practice. Still, I think that deserves an "Excellent!" for improved code readability. ------------------------------------------------------------------------------ src/cpu/ppc/vm/sharedRuntime_ppc.cpp 478 ResourceMark rm; 479 // Note, MaxVectorSize == 8 on PPC64. 480 assert(size <= 8, "%d bytes vectors are not supported", size); 481 return size > 8; ResourceMark no longer needed here. [This is the only lingering ResourceMark I've spotted so far.] Unrelated and not your problem, but that assert followed by the comparison looks really odd! There's another of these odd assert then compare sequences (without a nearby ResourceMark) here: src/cpu/sparc/vm/sharedRuntime_sparc.cpp 319 assert(size <= 8, "%d bytes vectors are not supported", size); 320 return size > 8; ------------------------------------------------------------------------------ src/os/solaris/vm/threadCritical_solaris.cpp 71 fatal("ThreadCritical::~ThreadCritical: mutex_unlock failed " 72 "(%s)", strerror(errno)); Remove line break from the format string. ------------------------------------------------------------------------------ src/os/windows/vm/vmError_windows.cpp 74 VMError::report_and_die(NULL, exception_code, NULL, 75 exceptionInfo->ExceptionRecord, exceptionInfo->ContextRecord); Fix indentation. ------------------------------------------------------------------------------ src/share/vm/code/nmethod.cpp 1712 assert(ic->is_clean(), "nmethod " PTR_FORMAT "not clean %s", from, from->method()->name_and_sig_as_C_string()); ... 2543 fatal("nmethod at " INTPTR_FORMAT " not in zone", this); ... 2551 fatal("findNMethod did not find this nmethod (" INTPTR_FORMAT ")", this); ... 2556 tty->print_cr("\t\tin nmethod at " INTPTR_FORMAT " (pcs)", this); Pre-existing: Don't we need the p2i() dance for the values associated with the [INT]PTR_FORMAT directives? The last isn't touched by your changes, but happens to be nearby. ------------------------------------------------------------------------------ src/share/vm/code/vtableStubs.cpp 225 fatal("bad compiled vtable dispatch: receiver " INTPTR_FORMAT ", " 226 "index %d (vtable length %d)", 227 (address)receiver, index, vt->length()); Pre-existing: Another p2i() dance; using it would eliminate the need for the cast. Also pre-existing nearby: 230 #endif // Product "Product" should be all-caps. ------------------------------------------------------------------------------ From doko at ubuntu.com Wed Sep 23 02:46:15 2015 From: doko at ubuntu.com (Matthias Klose) Date: Wed, 23 Sep 2015 04:46:15 +0200 Subject: RFR (s) 8135298: Fix zero builds for "unknown" architectures on linux. In-Reply-To: <55FABABA.7070207@oracle.com> References: <55F0FF9D.8030309@oracle.com> <55F104AF.5070009@oracle.com> <1441874805.3458.11.camel@redhat.com> <55FA7ED5.7030406@oracle.com> <1442494125.13753.21.camel@redhat.com> <55FABABA.7070207@oracle.com> Message-ID: <56021277.4020705@ubuntu.com> On 17.09.2015 15:06, Magnus Ihse Bursie wrote: > On 2015-09-17 14:48, Severin Gehwolf wrote: >> Hi, >> >> On Thu, 2015-09-17 at 10:50 +0200, Magnus Ihse Bursie wrote: >>> I'm not sure this code does what you think it does. At least when >>> building from the top-level makefile, ZERO_LIBARCH will always be >>> set, >>> so the "unknown" case will not be reached. >> CC'ing Matthias as we're not really building on any platform not >> listed in platform.m4. >> >> Looking at common/autoconf/platform.m4 (line 20 onwards) where VAR_CPU >> seems to be defined, then used as OPENJDK_TARGET_CPU and then as >> OPENJDK_TARGET_CPU_LEGACY_LIB and yet again as ZERO_LIBARCH. It may be >> true for all the CPUs listed in platform.m4. I can only speculate, but >> some distributions still build on more platforms than listed there >> (Debian?). So if some distribution patches that code to support a >> specific arch, what happens then? Has this been considered when you >> wrote the above? > > I can't really tell how the code behaves when additional patches I have not seen > are laid on top of it. :-) However, I *can* see that in hotspot-spec.gmk > ZERO_LIBARCH is *always* defined to have a value, so unless this is patched > around as well, the "unknown" case will never be reached. Otoh, I'm not sure > what that would mean either. The code is not likely to just happen to start > running on an unknown architecture without some nontrivial effort. I just > thought the code would do the same but more clearly if the fallback case just > used OPENJDK_TARGET_CPU_LEGACY_LIB, instead of talking about zero and "unknown". my intent was to provide a default value; if ZERO_LIBARCH can be used for that, then fine. >>> Also, I'd like to hijacking this thread slightly, sorry about that. >>> :) >>> >>> When rewriting the Hotspot makefiles, I've been wondering if it >>> would >>> not make more sense to consider "zero" to be a different >>> architecture, >>> rather than a different "flavor" (like minimal1), as it is treated >>> right >>> now. This patch seems to align with the treatment of zero as an >>> architecture. Just as an open question, do you think that would make >>> sense? >> A Zero (interpreter only) JVM has the goal to be (easily) built on >> many (any?) architectures. There can be Zero on x86_64, on arm32, etc. >> I don't think it makes sense to treat Zero as an architecture itself. >> To me it being a variant/flavor kind of build is correct (depending on >> what the definition of a variant/flavor build is exactly). > Ok, great! Thanks for your input. right, although there architectures where neither hotspot nor zero builds (sparc, sparc64). From marcus.larsson at oracle.com Wed Sep 23 08:20:35 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 23 Sep 2015 10:20:35 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <5601A41A.1080006@oracle.com> References: <55ED9210.6030904@oracle.com> <5601A41A.1080006@oracle.com> Message-ID: <560260D3.3000107@oracle.com> Hi, On 2015-09-22 20:55, Rachel Protacio wrote: > Hi, > > I believe that utilities/ostream.cpp#logStream::write() needs to have > a line at the end > > update_position(s, len); > > As it stands, the code updates the _position variable that belongs to > the stringStream _current_line (via the _current_line.write() > function), but the logStream _position does not get updated. This was > causing problems for me as I tried to make use of the > outputStream::indent() function. > Good catch, I'll add that. Thanks, Marcus > Thanks, > Rachel > > On 9/7/2015 9:33 AM, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch adding the unified logging >> framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. >> Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which >> contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log >> call has a corresponding LogTagSet instance, which keeps a track of >> all the outputs it should write the log message to (and their >> levels). Having tags as template arguments allows mapping directly >> from a set of tags to the LogTagSet instance, which means that the >> overhead for disabled logging should be low. Currently each log >> message can be tagged with up to 5 tags, but this can be increased if >> ever required (and with C++11's variadic templates the limit can be >> removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by >> command line arguments (-Xlog) or by DCMD. Both methods will in turn >> use the LogConfiguration class to perform the parsing & >> configuration. This configuration includes iterating over all >> LogTagSet instances and updating them accordingly. The >> LogTagLevelExpression class is used to represent the selection of >> tags and levels for a given configuration request (the >> "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances >> of this class is kept in LogTagSet to track what decorators to use >> (this is the union of all decorators used by its outputs). Each log >> call will create a LogDecorations instance (note: different classes), >> which will contain the actual decoration strings for the log message. >> These decorations are used for each output the tagset is set to log >> on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with >> argument. (This allows GC logging to prefix messages of certain >> tagsets with GCId.) Prefixes are implemented using template >> specializations based on the specified tags, with the >> general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log >> outputs. LogFileStreamOutput corresponds to FILE* stream based log >> outputs. LogFileOutput builds on this and adds the file management >> and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will >> be added at a later stage. >> >> Thanks, >> Marcus > From david.lindholm at oracle.com Wed Sep 23 08:19:16 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Wed, 23 Sep 2015 10:19:16 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <9F5C5634-B5F0-4D1E-88F6-C9C4DA77C645@oracle.com> References: <56013DCC.4060307@oracle.com> <9F5C5634-B5F0-4D1E-88F6-C9C4DA77C645@oracle.com> Message-ID: <56026084.6050606@oracle.com> Christian, On 2015-09-22 18:31, Christian Thalinger wrote: >> On Sep 22, 2015, at 1:38 AM, David Lindholm wrote: >> >> Hi, >> >> Please review the following patch which removes the need to use err_msg() or err_msg_res() to format strings that are sent to assert(), guarantee(), fatal() and vm_exit_out_of_memory(). These are now variadic macros. So, >> >> guarantee(_saved_index == _index, err_msg("saved index: %d index: %d", _saved_index, _index)); >> >> is replaced by >> >> guarantee(_saved_index == _index, "saved index: %d index: %d", _saved_index, _index); > Excellent change! Thank you! > >> This change also eliminates the need to choose if the buffer for the formatted string should be allocated on the stack (with err_msg()) or in the resource area (with err_msg_res()). There is no longer any need to allocate this extra buffer, since the va_list is sent all the way down to the functions writing the error to screen or file. > I?m not exactly sure but we might have ResourceMarks in the code just to have err_msg_res. Did you see any cases like this? I did not, but Kim Barrett have spotted one so far (I'll fix this). > Also, does this mean there are no buffers allocated on the stack to format the message anymore? Yes, that is correct. > The reason we introduced err_msg_res was because Solaris Studio produced some bad code on SPARC which blew up the stack. Yes, and this should not be needed any more. Thanks, David > >> To accommodate for this, VMError is now AllStatic. All patterns of usages of this class was to create a VMError and immediately call report_and_die() on it. >> >> Now, instead of having multiple constructors, there are multiple static VMError::report_and_die(). >> >> I have also removed all usages of err_msg() and err_msg_res() in assert(), guarantee(), fatal() and vm_exit_out_of_memory() in this patch. err_msg_res() is completely removed, err_msg() is still used by other functions. >> >> Since we have about 1000 asserts with "" as detailed message, I had to disable the gcc-only warning for empty format strings. >> >> (As a side note, there is also a JBS bug that describes a name change from assert to vmassert in our code. This changeset does not include such a name change, since we have over 19000 asserts in our code. We still have a "#define assert vmassert" just like before). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >> >> >> Testing: Passed JPRT >> >> >> Thanks, >> David >> From david.lindholm at oracle.com Wed Sep 23 09:37:52 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Wed, 23 Sep 2015 11:37:52 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <62694939-B653-4838-8DCA-06A0CE5E16F9@oracle.com> References: <56013DCC.4060307@oracle.com> <62694939-B653-4838-8DCA-06A0CE5E16F9@oracle.com> Message-ID: <560272F0.2070005@oracle.com> Kim, Thanks for looking at this! On 2015-09-23 01:34, Kim Barrett wrote: > On Sep 22, 2015, at 7:38 AM, David Lindholm wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ > I've run out of time to look at this today, so here's what I've got so > far. Looks good so far, just a few minor things. > > [I've reviewed the core changes (debug.[ch]pp and vmError.[ch]pp), > along with all of the rest of the utilities directory (at the end of > the webrev file list), and from the top of the webrev list through the > compiler directory, so maybe 1/3 of the way through.] > > I'll try to finish in the next day or two. > > ------------------------------------------------------------------------------ > src/share/vm/utilities/debug.hpp > 43 #define BUFSZ 256 > > Why rename RES_BUFSZ to BUFSZ? Both, but particularly the latter, > seem like poorly chosen macro names in a widely used header. Could we > instead have a static constant in FormatBufferBase? Yes, fixed. > ------------------------------------------------------------------------------ > src/share/vm/utilities/debug.hpp > 108 // Used to format messages > > Lost period at end of comment. Ok, fixed. > ------------------------------------------------------------------------------ > make/linux/makefiles/gcc.make > >> Since we have about 1000 asserts with "" as detailed message, I had to >> disable the gcc-only warning for empty format strings. > And they aren't even all that concentrated. At least 141 files in at > least 26 directories. Bleh! I guess there's not much choice here. > But a bug report seems called for. > > ------------------------------------------------------------------------------ > > There are a number of eliminations of err_msg_res from places where > the associated ResourceMark is not at all obvious. Since we're > dealing with assertions and the like, so that we're on our way to > process termination, that probably doesn't matter in practice. Still, > I think that deserves an "Excellent!" for improved code readability. > > ------------------------------------------------------------------------------ > src/cpu/ppc/vm/sharedRuntime_ppc.cpp > 478 ResourceMark rm; > 479 // Note, MaxVectorSize == 8 on PPC64. > 480 assert(size <= 8, "%d bytes vectors are not supported", size); > 481 return size > 8; > > ResourceMark no longer needed here. [This is the only lingering > ResourceMark I've spotted so far.] Ok, removed. > Unrelated and not your problem, but that assert followed by the > comparison looks really odd! There's another of these odd assert then > compare sequences (without a nearby ResourceMark) here: > > src/cpu/sparc/vm/sharedRuntime_sparc.cpp > 319 assert(size <= 8, "%d bytes vectors are not supported", size); > 320 return size > 8; > > ------------------------------------------------------------------------------ > src/os/solaris/vm/threadCritical_solaris.cpp > 71 fatal("ThreadCritical::~ThreadCritical: mutex_unlock failed " > 72 "(%s)", strerror(errno)); > > Remove line break from the format string. Fixed. > ------------------------------------------------------------------------------ > src/os/windows/vm/vmError_windows.cpp > 74 VMError::report_and_die(NULL, exception_code, NULL, > 75 exceptionInfo->ExceptionRecord, exceptionInfo->ContextRecord); > > Fix indentation. Fixed. > ------------------------------------------------------------------------------ > src/share/vm/code/nmethod.cpp > 1712 assert(ic->is_clean(), "nmethod " PTR_FORMAT "not clean %s", from, from->method()->name_and_sig_as_C_string()); > ... > 2543 fatal("nmethod at " INTPTR_FORMAT " not in zone", this); > ... > 2551 fatal("findNMethod did not find this nmethod (" INTPTR_FORMAT ")", this); > ... > 2556 tty->print_cr("\t\tin nmethod at " INTPTR_FORMAT " (pcs)", this); > > Pre-existing: Don't we need the p2i() dance for the values associated > with the [INT]PTR_FORMAT directives? The last isn't touched by your > changes, but happens to be nearby. You are correct, but this file (and many others) silences GCC warnings for format strings with PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC at the top of the file. This should absolutely be fixed in all these files, but fixing all these is a major task, and nothing I want to do in this change. > ------------------------------------------------------------------------------ > src/share/vm/code/vtableStubs.cpp > 225 fatal("bad compiled vtable dispatch: receiver " INTPTR_FORMAT ", " > 226 "index %d (vtable length %d)", > 227 (address)receiver, index, vt->length()); > > Pre-existing: Another p2i() dance; using it would eliminate the need > for the cast. Same comment as above. There are more format string problems in this file silenced by the pragma - these should be fixed in another change. > Also pre-existing nearby: > > 230 #endif // Product > > "Product" should be all-caps. > Ok, fixed. I think I'll wait for the rest of your comments before sending out a new webrev. Thanks, David From marcus.larsson at oracle.com Wed Sep 23 11:02:15 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 23 Sep 2015 13:02:15 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <6B7DE017-84F8-485E-BA97-FE6DB8601360@kodewerk.com> References: <55ED9210.6030904@oracle.com> <55F05228.7080905@oracle.com> <55F147B1.40505@oracle.com> <55F1A767.8040002@oracle.com> <55F2D9DC.5010401@oracle.com> <55F2DC58.7070004@oracle.com> <55F6C912.2020504@oracle.com> <55F6EF71.80005@oracle.com> <55F7848C.3010509@oracle.com> <899E1B43-19D9-42A3-815C-CC5946384DB8@oracle.com> <49CFB0B7-9DEE-4B62-B0A1-28ED9AA3A5DB@kodewerk.com> <4D5D3A47-AC71-42F6-B3FB-EA1AF886E192@oracle.com> <60282E28-49C5-4549-A83E-CF3C17DFB85E@oracle.com> <0fed0bdb35f81457858dcbcb09a18cc5.squirrel@excalibur.xssl.net> <1A9BDE02-6DEC-461E-836F-1E3E0FECAF5F@oracle.com> <8d1b14ae3127d29dd27d444c25e7c820.squirrel@excalibur.xssl.net> <55FB0EE1.6030708@oracle.com> <6B7DE017-84F8-485E-BA97-FE6DB8601360@kodewerk.com> Message-ID: <560286B7.8050907@oracle.com> Hi, I would prefer to keep the decorations as they are, but I can see the need for some way to distinguish similar decorations (uptime vs time, and tid vs pid). In any case, I've created a follow-up RFE [1] to resolve these issues separately. I don't think they should block us from integrating the framework, we can easily adjust this at a later stage. Thanks, Marcus [1]: https://bugs.openjdk.java.net/browse/JDK-8137009 On 2015-09-18 12:38, Kirk Pepperdine wrote: >> On Sep 17, 2015, at 10:05 PM, Ioi Lam wrote: >> >> Or, could we change the [] to <>? That way it's easier to parse by regexp. >> >> [2015-09-15T10:08:40.407-0100][0.203s] >> regex is: \[([^\[]*)\]\[([^\[]*)\] >> >> <2015-09-15T10:08:40.407-0100><0.203s> >> regex is: <([^>]*)><([^>]*)> > +1 > > It?s been traditional to use ?[? but moving to ? > Regards, > Kirk > From marcus.larsson at oracle.com Wed Sep 23 11:10:58 2015 From: marcus.larsson at oracle.com (Marcus Larsson) Date: Wed, 23 Sep 2015 13:10:58 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <55F7E6B5.2040502@oracle.com> References: <55ED9210.6030904@oracle.com> <55F7E6B5.2040502@oracle.com> Message-ID: <560288C2.5090606@oracle.com> Hi again, Here is the third and hopefully final version of the patch. Incremental webrev: http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01-02/ Full webrev: http://cr.openjdk.java.net/~mlarsson/8046148/webrev.02/ Changes: * Use update_position() in logStream::write so that indent() works properly * Add units (ms, ns) to all time decorations * Additional comments Thanks, Marcus On 2015-09-15 11:36, Marcus Larsson wrote: > Hi, > > Here is the updated version of the patch, including the feedback and > suggestions from Coleen, David and Ioi. > > Incremental webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ > > Full webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ > > Changes include: > * New NMT tag 'mtLogging' used by UL > * Added storage allocation classes for all new classes > * Added Log::puts(const char*) for plain string messages of any > length > * Fixed broken printf-attribute on Log::write functions > * Moved VM startup timestamp to after os::init > * Formatting and code style changes > > Thanks, > Marcus > > > On 2015-09-07 15:33, Marcus Larsson wrote: >> Hi, >> >> Please review the following patch adding the unified logging >> framework to hotspot. >> >> JEP: >> https://bugs.openjdk.java.net/browse/JDK-8046148 >> >> Webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >> >> See the JEP description for a general overview of the new feature. >> Below are some notes on the implementation. >> >> The patch adds the new 'share/vm/logging' subdirectory containing the >> unified logging framework. The main entry point is log.hpp, which >> contains the necessary macros and definitions to use the framework. >> >> Log tags are defined/listed in logTag.hpp, and are passed as template >> arguments to the Log class. Every combination of tags used in a log >> call has a corresponding LogTagSet instance, which keeps a track of >> all the outputs it should write the log message to (and their >> levels). Having tags as template arguments allows mapping directly >> from a set of tags to the LogTagSet instance, which means that the >> overhead for disabled logging should be low. Currently each log >> message can be tagged with up to 5 tags, but this can be increased if >> ever required (and with C++11's variadic templates the limit can be >> removed completely). >> >> The LogConfiguration class keeps track of configured outputs (stdout, >> stderr, and possible file outputs). Configuration is done either by >> command line arguments (-Xlog) or by DCMD. Both methods will in turn >> use the LogConfiguration class to perform the parsing & >> configuration. This configuration includes iterating over all >> LogTagSet instances and updating them accordingly. The >> LogTagLevelExpression class is used to represent the selection of >> tags and levels for a given configuration request (the >> "what"-expression). >> >> The LogDecorators class contains a selection of decorators. Instances >> of this class is kept in LogTagSet to track what decorators to use >> (this is the union of all decorators used by its outputs). Each log >> call will create a LogDecorations instance (note: different classes), >> which will contain the actual decoration strings for the log message. >> These decorations are used for each output the tagset is set to log >> on, and are then discarded. >> >> The LogPrefix class allows messages of specific sets of tags to be >> prefixed. The prefix should supply a printf-style format with >> argument. (This allows GC logging to prefix messages of certain >> tagsets with GCId.) Prefixes are implemented using template >> specializations based on the specified tags, with the >> general/unspecialized case giving an empty prefix. >> >> The LogOutput class defines the interface for all types of log >> outputs. LogFileStreamOutput corresponds to FILE* stream based log >> outputs. LogFileOutput builds on this and adds the file management >> and log rotation support. >> >> A simple jtreg test is included in this patch. Additional tests will >> be added at a later stage. >> >> Thanks, >> Marcus > From vladimir.kozlov at oracle.com Wed Sep 23 11:16:10 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 23 Sep 2015 19:16:10 +0800 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <55FD2E07.6020401@redhat.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55FD2E07.6020401@redhat.com> Message-ID: <560289FA.1060902@oracle.com> It should use next(4) which we have exactly for such cases: map->set_callee_saved(YMMHI_STACK_OFFSET(0), xmm0->as_VMReg()->next(4)); Note, next() is equal next(1). Vladimir On 9/19/15 5:42 PM, Andrew Haley wrote: > On 14/09/15 08:24, Christian Thalinger wrote: > > +#if defined(COMPILER2) || INCLUDE_JVMCI > + if (save_vectors) { > + assert(ymmhi_offset != -1, "save area must exist"); > + map->set_callee_saved(YMMHI_STACK_OFFSET( 0), xmm0->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET( 16), xmm1->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET( 32), xmm2->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET( 48), xmm3->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET( 64), xmm4->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET( 80), xmm5->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET( 96), xmm6->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(112), xmm7->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(128), xmm8->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(144), xmm9->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(160), xmm10->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(176), xmm11->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(192), xmm12->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(208), xmm13->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(224), xmm14->as_VMReg()->next()->next()->next()->next()); > + map->set_callee_saved(YMMHI_STACK_OFFSET(240), xmm15->as_VMReg()->next()->next()->next()->next()); > + } > +#endif > + > > Um, really? There is plenty of very odd code in HotSpot, but this is something else > again. > > Andrew. > From vladimir.kozlov at oracle.com Wed Sep 23 11:43:13 2015 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 23 Sep 2015 19:43:13 +0800 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> Message-ID: <56029051.8020607@oracle.com> Can't say much about top changes but they work so I assume they are correct. I looked on C++ code. We have a lot #if defined(COMPILER2) || INCLUDE_JVMCI May be we should define one value #if defined(HIGH_TIER_COMPILER) ? vmStructs_x86.hpp we may need to add new VM_Version values added recently: CPU_RTM = (1 << 24), // Restricted Transactional Memory instructions CPU_ADX = (1 << 25), CPU_AVX512F = (1 << 26), // AVX 512bit foundation instructions ... Several places (14) in new code use %p which we should change to PRT_FROMAT or similar. Vladimir On 9/22/15 7:52 AM, Christian Thalinger wrote: > Thank you, Igor. > >> On Sep 21, 2015, at 1:48 PM, Igor Veresov wrote: >> >> I?ve seen it all before, but I went through it one more time in its entirety and it looks good. >> >> igor >> >>> On Sep 21, 2015, at 3:24 PM, Christian Thalinger wrote: >>> >>>> >>> >>> Here are new webrevs against hs-comp: >>> >>> http://cr.openjdk.java.net/~twisti/8136421/webrev/ >>> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ >>> > From magnus.ihse.bursie at oracle.com Wed Sep 23 12:11:40 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 23 Sep 2015 14:11:40 +0200 Subject: RFR(S): 8027565 - Enable /d2Zi+ when building with Visual Studio 2013 In-Reply-To: <027d01d0f544$a71951a0$f54bf4e0$@oracle.com> References: <01e701d0f4ce$a97d8400$fc788c00$@oracle.com> <560119E0.5090902@oracle.com> <027d01d0f544$a71951a0$f54bf4e0$@oracle.com> Message-ID: <560296FC.1020002@oracle.com> On 2015-09-22 16:40, Christian Tornqvist wrote: > Hi Magnus, > >> When was the /d2Zi+ switch introduced? > As Dan pointed out, it was introduced as an undocumented switch in VS2012 > and was planned to be an official switch in VS2013. The official switch was > introduced in VS2013 Update 3 and in VS2015 this switch is on by default. Fair enough. There seemed to be no deprecation warning for /d2Zi+ either, so I guess that's fine going forward too. Looks good for me, then. Nice to see debugging data for optimized code, way to go Microsoft (and Christian)! :) /Magnus > > Thanks, > Christian > > -----Original Message----- > From: Magnus Ihse Bursie [mailto:magnus.ihse.bursie at oracle.com] > Sent: Tuesday, September 22, 2015 5:06 AM > To: Christian Tornqvist ; > hotspot-dev at openjdk.java.net > Subject: Re: RFR(S): 8027565 - Enable /d2Zi+ when building with Visual > Studio 2013 > > On 2015-09-22 02:36, Christian Tornqvist wrote: >> Hi everyone, >> >> >> Please review this small change that enables generation of enhanced >> debug symbols for optimized code on Windows. The switch is available >> as /d2Zi+ prior to Update 3, when it was officially added as /Zo. Both >> /Zo and /d2Zi+ works with VS2013 Update 3 and later. More information >> about this switch can be found at >> https://msdn.microsoft.com/en-us/library/dn785163.aspx > Christian, > > When was the /d2Zi+ switch introduced? > > /Magnus > >> >> >> I've verified the change by building locally with VS2013 Update 4 and >> with JPRT that has the older VS2013 (non-update) version. >> >> >> >> Webrev: >> >> http://cr.openjdk.java.net/~ctornqvi/webrev/8027565/webrev.00/ >> >> >> >> Bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8027565 >> >> >> >> Thanks, >> >> Christian >> > From magnus.ihse.bursie at oracle.com Wed Sep 23 13:01:43 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 23 Sep 2015 15:01:43 +0200 Subject: RFR (S): 8136672: editor configuration for IDEs In-Reply-To: <3616187E21868C40AD1B36D41D29F4C136905BEE@FMSMSX106.amr.corp.intel.com> References: <3616187E21868C40AD1B36D41D29F4C136905BEE@FMSMSX106.amr.corp.intel.com> Message-ID: <5602A2B7.1070804@oracle.com> On 2015-09-18 23:59, Wojtowicz, Tomasz wrote: > Review Title: editor configuration for IDEs > Review ID: #8136672 > Diff: http://cr.openjdk.java.net/~mcberg/8136672/webrev.01 > Description: In order to avoid jcheck errors like following ones: > "Tests found: > > src/cpu/x86/vm/macroAssembler_x86.cpp:8686: Trailing whitespace > src/cpu/x86/vm/macroAssembler_x86.hpp:1330: Trailing whitespace > src/cpu/x86/vm/stubGenerator_x86_32.cpp:3001: Trailing whitespace > src/cpu/x86/vm/stubGenerator_x86_64.cpp:3968: Trailing whitespace > src/cpu/x86/vm/stubRoutines_x86.cpp:136: Trailing whitespace > src/share/vm/interpreter/interpreter.cpp:594: Carriage return (^M) > src/share/vm/interpreter/templateInterpreter.cpp:426: Carriage return > (^M) > src/cpu/x86/vm/crc32c.h:31: Trailing whitespace" > > I propose to introduce an .editorconfig file which would encapsulate common layout required settings. EditorConfig (http://editorconfig.org/) provides plugins for most popular editors/IDEs and make sure that every modification adheres to the stated rules: > hotspot/.editorconfig file contents > root = true > > [*] > indent_style = space > indent_size = 2 > end_of_line = lf > charset = utf-8 > trim_trailing_whitespace = true > insert_final_newline = true > Link: https://bugs.openjdk.java.net/browse/JDK-8136672 > Author: Tomasz, Wojtowicz > > -- > Thank you, I've never heard about editorconfig before, but it sounds like a good idea. Does it have any real community impact? I noted that most major IDEs does not support it natively (yet), so the practical effect is probably quite small. However, your editorconfig files seems incomplete. It should probably contain something like [*.java] indent_size = 4 as well, although I must admit I didn't study the format close enough to understand if you could just override the default indent size like that for just java files. /Magnus From coleen.phillimore at oracle.com Wed Sep 23 13:05:24 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 23 Sep 2015 09:05:24 -0400 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <529B4819-0325-41A0-81F0-EA85A0EBE908@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55FC8CB6.2030204@oracle.com> <529B4819-0325-41A0-81F0-EA85A0EBE908@oracle.com> Message-ID: <5602A394.3010207@oracle.com> On 9/21/15 2:35 PM, Christian Thalinger wrote: >> >> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.hpp.udiff.html >> >> >> I thought you were only going to add 3 words to thread? (If you >> reordered the fields, it would be). > > I reordered the fields a bit. > >> >> Having to declare this many fields inline to any class suggests it >> should be refactored into a different class, and a pointer to that >> class in the global thread class. I've made this comment before, I >> know. It seems all the blue in this could be a new class. > > We can make this change later if we want to. > I've filed RFE: https://bugs.openjdk.java.net/browse/JDK-8137018 What's the RFE number for this work? Coleen From coleen.phillimore at oracle.com Wed Sep 23 18:14:04 2015 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 23 Sep 2015 14:14:04 -0400 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <560288C2.5090606@oracle.com> References: <55ED9210.6030904@oracle.com> <55F7E6B5.2040502@oracle.com> <560288C2.5090606@oracle.com> Message-ID: <5602EBEC.7090701@oracle.com> Looks good! Coleen On 9/23/15 7:10 AM, Marcus Larsson wrote: > Hi again, > > Here is the third and hopefully final version of the patch. > > Incremental webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01-02/ > > Full webrev: > http://cr.openjdk.java.net/~mlarsson/8046148/webrev.02/ > > Changes: > * Use update_position() in logStream::write so that indent() works > properly > * Add units (ms, ns) to all time decorations > * Additional comments > > Thanks, > Marcus > > On 2015-09-15 11:36, Marcus Larsson wrote: >> Hi, >> >> Here is the updated version of the patch, including the feedback and >> suggestions from Coleen, David and Ioi. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ >> >> Full webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ >> >> Changes include: >> * New NMT tag 'mtLogging' used by UL >> * Added storage allocation classes for all new classes >> * Added Log::puts(const char*) for plain string messages of >> any length >> * Fixed broken printf-attribute on Log::write functions >> * Moved VM startup timestamp to after os::init >> * Formatting and code style changes >> >> Thanks, >> Marcus >> >> >> On 2015-09-07 15:33, Marcus Larsson wrote: >>> Hi, >>> >>> Please review the following patch adding the unified logging >>> framework to hotspot. >>> >>> JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>> >>> See the JEP description for a general overview of the new feature. >>> Below are some notes on the implementation. >>> >>> The patch adds the new 'share/vm/logging' subdirectory containing >>> the unified logging framework. The main entry point is log.hpp, >>> which contains the necessary macros and definitions to use the >>> framework. >>> >>> Log tags are defined/listed in logTag.hpp, and are passed as >>> template arguments to the Log class. Every combination of tags used >>> in a log call has a corresponding LogTagSet instance, which keeps a >>> track of all the outputs it should write the log message to (and >>> their levels). Having tags as template arguments allows mapping >>> directly from a set of tags to the LogTagSet instance, which means >>> that the overhead for disabled logging should be low. Currently each >>> log message can be tagged with up to 5 tags, but this can be >>> increased if ever required (and with C++11's variadic templates the >>> limit can be removed completely). >>> >>> The LogConfiguration class keeps track of configured outputs >>> (stdout, stderr, and possible file outputs). Configuration is done >>> either by command line arguments (-Xlog) or by DCMD. Both methods >>> will in turn use the LogConfiguration class to perform the parsing & >>> configuration. This configuration includes iterating over all >>> LogTagSet instances and updating them accordingly. The >>> LogTagLevelExpression class is used to represent the selection of >>> tags and levels for a given configuration request (the >>> "what"-expression). >>> >>> The LogDecorators class contains a selection of decorators. >>> Instances of this class is kept in LogTagSet to track what >>> decorators to use (this is the union of all decorators used by its >>> outputs). Each log call will create a LogDecorations instance (note: >>> different classes), which will contain the actual decoration strings >>> for the log message. These decorations are used for each output the >>> tagset is set to log on, and are then discarded. >>> >>> The LogPrefix class allows messages of specific sets of tags to be >>> prefixed. The prefix should supply a printf-style format with >>> argument. (This allows GC logging to prefix messages of certain >>> tagsets with GCId.) Prefixes are implemented using template >>> specializations based on the specified tags, with the >>> general/unspecialized case giving an empty prefix. >>> >>> The LogOutput class defines the interface for all types of log >>> outputs. LogFileStreamOutput corresponds to FILE* stream based log >>> outputs. LogFileOutput builds on this and adds the file management >>> and log rotation support. >>> >>> A simple jtreg test is included in this patch. Additional tests will >>> be added at a later stage. >>> >>> Thanks, >>> Marcus >> > From kim.barrett at oracle.com Wed Sep 23 18:32:47 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 23 Sep 2015 14:32:47 -0400 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <56013DCC.4060307@oracle.com> References: <56013DCC.4060307@oracle.com> Message-ID: <5B4B692B-4EDA-42B3-B648-0A593C85ACF1@oracle.com> On Sep 22, 2015, at 7:38 AM, David Lindholm wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 > Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ Another batch, this time covering gc through memory. I need to take a break from this, but I should be able to finish tomorrow. Mostly formatting issues. BTW, although I'm finding some of these, I'm also noticing that you tidied up a lot more. Thanks for that. ------------------------------------------------------------------------------ src/share/vm/gc/cms/compactibleFreeListSpace.cpp 2877 assert(n * word_sz == fc->size(), 2878 "Chunk size " SIZE_FORMAT " is not exactly splittable by " 2879 SIZE_FORMAT " sized chunks of size " SIZE_FORMAT, 2880 fc->size(), n, word_sz); Pre-existing: Indentation of message arguments. ------------------------------------------------------------------------------ src/share/vm/gc/g1/bufferingOopClosure.cpp 157 assert((got) == (expected), \ 158 "Expected: %d, got: %d, when running testCount(%d, %d, %d)", \ 159 (got), (expected), num_narrow, num_full, do_oop_order) 194 assert((got) == (expected), \ 195 "Expected: %d, got: %d. testIsBufferEmptyOrFull(%d, %d, %s, %s)", \ 196 (got), (expected), num_narrow, num_full, \ 197 BOOL_TO_STR(expect_empty), BOOL_TO_STR(expect_full)) 234 assert(boc.is_buffer_empty(), 235 "Should be empty after call to done(). testEmptyAfterDone(%d, %d)", 236 num_narrow, num_full); Pre-existing: Indentation of message arguments. ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1AllocRegion.hpp 37 #define G1_ALLOC_REGION_MSG(message) \ 38 "[%s] %s c: %u b: %s r: " PTR_FORMAT " u: " SIZE_FORMAT, \ 39 _name, message, _count, BOOL_TO_STR(_bot_updates), \ 40 p2i(_alloc_region), _used_bytes_before Misaligned line continuations. I think this doesn't belong in this header; it's not something that a client can use. Suggest moving it to g1AllocRegion.inline.hpp. Suggest changing this to G1_ALLOC_REGION_ASSERT, with predicate argument and full assert expression as expansion. Macros with unusual syntactic expansions can be confusing, so best avoided unless they provide some benefit that can't be obtained in some other way. ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1AllocRegion.hpp 49 class G1AllocRegion VALUE_OBJ_CLASS_SPEC { 50 friend class ar_ext_msg; ar_ext_msg no longer exists. ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1Allocator.cpp 432 assert((end_alignment_in_bytes >> LogHeapWordSize) < HeapRegion::min_region_size_in_words(), 433 "alignment " SIZE_FORMAT " too large", end_alignment_in_bytes); Pre-existing: Indentation of message arguments. ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1BiasedArray.cpp 44 guarantee(biased_index >= bias() && biased_index < (bias() + length()), 45 "Biased index out of bounds, index: " SIZE_FORMAT " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); 50 guarantee(biased_index >= bias() && biased_index <= (bias() + length()), 51 "Biased index out of inclusive bounds, index: " SIZE_FORMAT " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); Pre-existing: Indentation of message arguments. Maybe also split really long lines? ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1BlockOffsetTable.cpp 364 assert((_array->offset_array(orig_index) == 0 && 365 blk_start == boundary) || 366 (_array->offset_array(orig_index) > 0 && 367 _array->offset_array(orig_index) <= N_words), [Pre-existing, and not modified by this changeset, but nearby.] Indentation of 366 and 367 are wrong. ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1BlockOffsetTable.hpp 151 assert(offset <= N_words, 152 "%s - " 153 "offset: " SIZE_FORMAT ", N_words: %u", 154 msg, offset, (uint)N_words); Maybe eliminate the line break in the format string? ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1BlockOffsetTable.inline.hpp 95 assert(result >= _reserved.start() && result < _reserved.end(), 96 "bad address from index result " PTR_FORMAT 97 " _reserved.start() " PTR_FORMAT " _reserved.end() " 98 PTR_FORMAT, 99 p2i(result), p2i(_reserved.start()), p2i(_reserved.end())); Maybe eliminate the second line break in the format string? ------------------------------------------------------------------------------ src/share/vm/gc/g1/g1CollectedHeap.cpp 2540 assert(_old_marking_cycles_started == _old_marking_cycles_completed || 2541 _old_marking_cycles_started == _old_marking_cycles_completed + 1, 2542 "Wrong marking cycle count (started: %d, completed: %d)", 2543 _old_marking_cycles_started, _old_marking_cycles_completed); Pre-existing: Indentation of arguments. ------------------------------------------------------------------------------ src/share/vm/gc/g1/heapRegionSet.cpp Suggest eliminating hrs_ext_msg and hrs_err_msg in a manner similar to elimination of ar_ext_msg, taking into account earlier comments in that area. Please do that under a new RFE; this change set is quite larg enough already. ------------------------------------------------------------------------------ src/share/vm/gc/g1/heapRegionType.hpp 30 #define hrt_assert_is_valid(tag) \ 31 assert(is_valid((tag)), "invalid HR type: %u", (uint) (tag)) Pre-existing: I don't think this should be a macro. I suggest it should be a private member function using NOT_DEBUG_RETURN and an ASSERT-only definition in the .cpp. https://bugs.openjdk.java.net/browse/JDK-8137046 ------------------------------------------------------------------------------ src/share/vm/gc/parallel/mutableNUMASpace.cpp 87 assert(words_to_fill >= CollectedHeap::min_fill_size(), 88 "Remaining size (" SIZE_FORMAT ") is too small to fill (based on " SIZE_FORMAT " and " SIZE_FORMAT ")", 89 words_to_fill, words_left_to_fill, CollectedHeap::filler_array_max_size()); Pre-existing: Indentation of message arguments. ------------------------------------------------------------------------------ src/share/vm/gc/shared/gcCause.hpp 97 assert(cause != GCCause::_old_generation_too_full_to_scavenge && 98 cause != GCCause::_old_generation_expanded_on_last_scavenge, 99 "This GCCause may be correct but is not expected yet: %s", 100 to_string(cause)); Pre-existing: Indentation of arguments. ------------------------------------------------------------------------------ src/share/vm/gc/shared/space.cpp 606 assert(oop(last)->is_oop(),PTR_FORMAT " should be an object start", p2i(last)); Add space between comma and PTR_FORMAT. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Wed Sep 23 18:36:12 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 23 Sep 2015 14:36:12 -0400 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <560272F0.2070005@oracle.com> References: <56013DCC.4060307@oracle.com> <62694939-B653-4838-8DCA-06A0CE5E16F9@oracle.com> <560272F0.2070005@oracle.com> Message-ID: On Sep 23, 2015, at 5:37 AM, David Lindholm wrote: > >> src/share/vm/code/nmethod.cpp >> 1712 assert(ic->is_clean(), "nmethod " PTR_FORMAT "not clean %s", from, from->method()->name_and_sig_as_C_string()); >> ... >> 2543 fatal("nmethod at " INTPTR_FORMAT " not in zone", this); >> ... >> 2551 fatal("findNMethod did not find this nmethod (" INTPTR_FORMAT ")", this); >> ... >> 2556 tty->print_cr("\t\tin nmethod at " INTPTR_FORMAT " (pcs)", this); >> >> Pre-existing: Don't we need the p2i() dance for the values associated >> with the [INT]PTR_FORMAT directives? The last isn't touched by your >> changes, but happens to be nearby. > > You are correct, but this file (and many others) silences GCC warnings for format strings with PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC at the top of the file. This should absolutely be fixed in all these files, but fixing all these is a major task, and nothing I want to do in this change. Thanks for the explanation. And I see there are already some relevant bug reports, though I didn't see any covering the code directory. > I think I'll wait for the rest of your comments before sending out a new webrev. > Great. I'll try to finish up soon. From christian.thalinger at oracle.com Wed Sep 23 19:02:21 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 23 Sep 2015 09:02:21 -1000 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <56026084.6050606@oracle.com> References: <56013DCC.4060307@oracle.com> <9F5C5634-B5F0-4D1E-88F6-C9C4DA77C645@oracle.com> <56026084.6050606@oracle.com> Message-ID: <8A69D929-F288-4C16-9619-4F086479E1A9@oracle.com> > On Sep 22, 2015, at 10:19 PM, David Lindholm wrote: > > Christian, > > On 2015-09-22 18:31, Christian Thalinger wrote: >>> On Sep 22, 2015, at 1:38 AM, David Lindholm wrote: >>> >>> Hi, >>> >>> Please review the following patch which removes the need to use err_msg() or err_msg_res() to format strings that are sent to assert(), guarantee(), fatal() and vm_exit_out_of_memory(). These are now variadic macros. So, >>> >>> guarantee(_saved_index == _index, err_msg("saved index: %d index: %d", _saved_index, _index)); >>> >>> is replaced by >>> >>> guarantee(_saved_index == _index, "saved index: %d index: %d", _saved_index, _index); >> Excellent change! > > Thank you! > >> >>> This change also eliminates the need to choose if the buffer for the formatted string should be allocated on the stack (with err_msg()) or in the resource area (with err_msg_res()). There is no longer any need to allocate this extra buffer, since the va_list is sent all the way down to the functions writing the error to screen or file. >> I?m not exactly sure but we might have ResourceMarks in the code just to have err_msg_res. Did you see any cases like this? > > I did not, but Kim Barrett have spotted one so far (I'll fix this). > >> Also, does this mean there are no buffers allocated on the stack to format the message anymore? > > Yes, that is correct. > >> The reason we introduced err_msg_res was because Solaris Studio produced some bad code on SPARC which blew up the stack. > > Yes, and this should not be needed any more. I might be repeating myself but? excellent! > > > Thanks, > David > >> >>> To accommodate for this, VMError is now AllStatic. All patterns of usages of this class was to create a VMError and immediately call report_and_die() on it. >>> >>> Now, instead of having multiple constructors, there are multiple static VMError::report_and_die(). >>> >>> I have also removed all usages of err_msg() and err_msg_res() in assert(), guarantee(), fatal() and vm_exit_out_of_memory() in this patch. err_msg_res() is completely removed, err_msg() is still used by other functions. >>> >>> Since we have about 1000 asserts with "" as detailed message, I had to disable the gcc-only warning for empty format strings. >>> >>> (As a side note, there is also a JBS bug that describes a name change from assert to vmassert in our code. This changeset does not include such a name change, since we have over 19000 asserts in our code. We still have a "#define assert vmassert" just like before). >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >>> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >>> >>> >>> Testing: Passed JPRT >>> >>> >>> Thanks, >>> David From christian.thalinger at oracle.com Wed Sep 23 19:05:29 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 23 Sep 2015 09:05:29 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <5602A394.3010207@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55FC8CB6.2030204@oracle.com> <529B4819-0325-41A0-81F0-EA85A0EBE908@oracle.com> <5602A394.3010207@oracle.com> Message-ID: <4CFF2763-19C2-4181-A2AB-3B37724C8FDE@oracle.com> > On Sep 23, 2015, at 3:05 AM, Coleen Phillimore wrote: > > > > On 9/21/15 2:35 PM, Christian Thalinger wrote: >>> >>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/src/share/vm/runtime/thread.hpp.udiff.html >>> >>> I thought you were only going to add 3 words to thread? (If you reordered the fields, it would be). >> >> I reordered the fields a bit. >> >>> >>> Having to declare this many fields inline to any class suggests it should be refactored into a different class, and a pointer to that class in the global thread class. I've made this comment before, I know. It seems all the blue in this could be a new class. >> >> We can make this change later if we want to. >> > > I've filed RFE: https://bugs.openjdk.java.net/browse/JDK-8137018 Thanks. > > What's the RFE number for this work? I?ve linked it to JEP 243. > > Coleen > From edward.nevill at gmail.com Wed Sep 23 19:30:31 2015 From: edward.nevill at gmail.com (Edward Nevill) Date: Wed, 23 Sep 2015 20:30:31 +0100 Subject: 8136165: AARCH64: Tidy up compiled native calls In-Reply-To: <57FE21CF-5FBE-4BBF-8065-30DA33130DC7@oracle.com> References: <55F1AA57.70307@redhat.com> <55F1BB94.4080408@redhat.com> <55FC0C8D.6050508@redhat.com> <57FE21CF-5FBE-4BBF-8065-30DA33130DC7@oracle.com> Message-ID: On Tue, Sep 22, 2015 at 12:12 PM, Roland Westrelin < roland.westrelin at oracle.com> wrote: > > New patch at http://cr.openjdk.java.net/~aph/8136165-2 > > That looks ok to me but it would be better if someone working on the > AARCH64 port took a look as well. > > Roland. Hi, I have eyeballed the patch and done a smoke test. Looks good to me. Much cleaner. Ed. From kim.barrett at oracle.com Wed Sep 23 20:00:42 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 23 Sep 2015 16:00:42 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <5601DE93.2070802@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <5601DE93.2070802@oracle.com> Message-ID: <82D592A7-5926-47B0-ACEE-85EE2E5488C0@oracle.com> On Sep 22, 2015, at 7:04 PM, Jon Masamitsu wrote: > >>> src/share/vm/gc/g1/g1_globals.hpp >>> 240 product(uintx, G1ConcRefinementThreads, 0, \ >>> ... >>> 243 range(0, (max_jint-1)/4) \ >>> >>> I know we discussed the problem of thread counts. I'd suggested >>> perhaps basing the upper bound on the number of processors. (Some >>> care might be needed for uniprocessor systems.) I couldn't find any >>> followup on that suggestion though. > > Kim, > > My apologies but I dissuaded Sangheon from that idea. I don't think the > number of processors was enough. A hundred or a thousand times the > number of processors should be enough but if I had to pick some > multiplier like that, I'd rather pick something limited by the size > of the type and back off some. It's still arbitrary but whereas I might > some decade be surprised that 1000 * #-of-processors was too few, > I'm not going to see 1,000,000,000 be too few. I suspect we're using different meanings for "processor". I meant "number of processing units", e.g. the number of lines in Linux /proc/cpuinfo that begin with the word "processor". A more precise term might be "hardware concurrency". A maximum multiplier of 1-2 seems to me sufficient for these generally CPU-intensive tasks. From kim.barrett at oracle.com Wed Sep 23 20:35:11 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 23 Sep 2015 16:35:11 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <56008D3B.5010008@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> Message-ID: On Sep 21, 2015, at 7:05 PM, sangheon.kim wrote: > On 09/21/2015 02:59 PM, Kim Barrett wrote: >> On Sep 10, 2015, at 8:01 PM, sangheon.kim wrote: >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>> >>> Testing: >>> JPRT, UTE(vm.quick-pcl) and test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >> ------------------------------------------------------------------------------ >> src/share/vm/gc/g1/g1_globals.hpp >> 142 product(intx, G1ConcRefinementServiceIntervalMillis, 300, \ >> ... >> 145 range(0, max_jint) \ >> >> Why is this being changed from max_intx to max_jint? > This option is used for 'long' type parameter and it is 32bit for Windows. I think you are saying that on Windows the value of this option is passed to some API that is limited to a 32bit value? If so, then OK. >> ------------------------------------------------------------------------------ >> src/share/vm/gc/g1/g1_globals.hpp >> 169 develop(intx, G1RSetRegionEntriesBase, 256, \ >> 173 product(intx, G1RSetRegionEntries, 0, \ >> 179 develop(intx, G1RSetSparseRegionEntriesBase, 4, \ >> 184 product(intx, G1RSetSparseRegionEntries, 0, \ >> 240 product(uintx, G1ConcRefinementThreads, 0, \ >> >> All of these have their max range value changed to be divided by 4. >> What's this about? What is this magic "4"? > 'G1RSetRegionEntries', 'G1RSetSparseRegionEntries' and 'G1ConcRefinementThreads' are finally multiplied by pointer size. > So we need to divide by 4 for 32bit to avoid overflow. We don't need for 64bit but I thought it will be enough for 64bit. > 'G1RSetRegionEntriesBase' and 'G1RSetSparseRegionEntriesBase' changed to have same range as 'G1RSetRegionEntries' and 'G1RSetSparseRegionEntries?. Dividing by 4 doesn't account for being multiplied by pointer size on a 64bit platform. In light of that, I don't understand this part of your response: "We don't need for 64bit but I thought it will be enough for 64bit." >> ------------------------------------------------------------------------------ >> src/share/vm/gc/g1/g1_globals.hpp >> 240 product(uintx, G1ConcRefinementThreads, 0, \ >> ... >> 243 range(0, (max_jint-1)/4) \ >> >> I know we discussed the problem of thread counts. I'd suggested >> perhaps basing the upper bound on the number of processors. (Some >> care might be needed for uniprocessor systems.) I couldn't find any >> followup on that suggestion though. > Firstly I agreed to your opinion of using the number of processors if we need to give upper bound. > But from the email thread that we discussed, I ended not to handle OOM from our range/constraint validation framework. > Dmitry told us that testing framework is okay for OOM(only for exit value of 1) and Gerard also agreed not to handle it from validation. > Finally, I think the only thing that we need for this option is just excluding this test. > > If we don't exclude tests that are using too many resources, it would make problem to other tests that running in parallel. > So it would be better to exclude these tests. See my response to Jon on this part. I'm OK with going ahead with this changeset despite the above open discussions. The rest of the changes look good to me, and these couple of lingering items are no worse than, and perhaps an improvement on, the status quo. We can follow up on the above items separately. From sangheon.kim at oracle.com Wed Sep 23 21:00:04 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 23 Sep 2015 14:00:04 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> Message-ID: <560312D4.7050906@oracle.com> On 09/23/2015 01:35 PM, Kim Barrett wrote: > On Sep 21, 2015, at 7:05 PM, sangheon.kim wrote: >> On 09/21/2015 02:59 PM, Kim Barrett wrote: >>> On Sep 10, 2015, at 8:01 PM, sangheon.kim wrote: >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>>> >>>> Testing: >>>> JPRT, UTE(vm.quick-pcl) and test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >>> ------------------------------------------------------------------------------ >>> src/share/vm/gc/g1/g1_globals.hpp >>> 142 product(intx, G1ConcRefinementServiceIntervalMillis, 300, \ >>> ... >>> 145 range(0, max_jint) \ >>> >>> Why is this being changed from max_intx to max_jint? >> This option is used for 'long' type parameter and it is 32bit for Windows. > I think you are saying that on Windows the value of this option is > passed to some API that is limited to a 32bit value? If so, then OK. Yes, you are right. > >>> ------------------------------------------------------------------------------ >>> src/share/vm/gc/g1/g1_globals.hpp >>> 169 develop(intx, G1RSetRegionEntriesBase, 256, \ >>> 173 product(intx, G1RSetRegionEntries, 0, \ >>> 179 develop(intx, G1RSetSparseRegionEntriesBase, 4, \ >>> 184 product(intx, G1RSetSparseRegionEntries, 0, \ >>> 240 product(uintx, G1ConcRefinementThreads, 0, \ >>> >>> All of these have their max range value changed to be divided by 4. >>> What's this about? What is this magic "4"? >> 'G1RSetRegionEntries', 'G1RSetSparseRegionEntries' and 'G1ConcRefinementThreads' are finally multiplied by pointer size. >> So we need to divide by 4 for 32bit to avoid overflow. We don't need for 64bit but I thought it will be enough for 64bit. >> 'G1RSetRegionEntriesBase' and 'G1RSetSparseRegionEntriesBase' changed to have same range as 'G1RSetRegionEntries' and 'G1RSetSparseRegionEntries?. > Dividing by 4 doesn't account for being multiplied by pointer size on > a 64bit platform. You are right. This is limitation for 32bit. Both 'jint' and 'intx' are 32bit for 32bit platform and these options are used for 'size_t' type parameter after multiplying by pointer size. So we need to divide by 4 on 32bit platform. When it comes to 64bit platform, we don't need to divide by 4 as 'size_t' is big enough to cover 'max_jint * 8'. But I think that value divided by 4 would be enough for 64bit platform. Jon also pointed '4' seems strange, so I suggested to use 'jintSize'. What do you think? > In light of that, I don't understand this part of > your response: "We don't need for 64bit but I thought it will be > enough for 64bit." I think I explained above. > >>> ------------------------------------------------------------------------------ >>> src/share/vm/gc/g1/g1_globals.hpp >>> 240 product(uintx, G1ConcRefinementThreads, 0, \ >>> ... >>> 243 range(0, (max_jint-1)/4) \ >>> >>> I know we discussed the problem of thread counts. I'd suggested >>> perhaps basing the upper bound on the number of processors. (Some >>> care might be needed for uniprocessor systems.) I couldn't find any >>> followup on that suggestion though. >> Firstly I agreed to your opinion of using the number of processors if we need to give upper bound. >> But from the email thread that we discussed, I ended not to handle OOM from our range/constraint validation framework. >> Dmitry told us that testing framework is okay for OOM(only for exit value of 1) and Gerard also agreed not to handle it from validation. >> Finally, I think the only thing that we need for this option is just excluding this test. >> >> If we don't exclude tests that are using too many resources, it would make problem to other tests that running in parallel. >> So it would be better to exclude these tests. > See my response to Jon on this part. > > I'm OK with going ahead with this changeset despite the above open > discussions. The rest of the changes look good to me, and these > couple of lingering items are no worse than, and perhaps an > improvement on, the status quo. We can follow up on the above items > separately. Okay, following up seems good idea. Thanks, Sangheon > > From christian.thalinger at oracle.com Wed Sep 23 21:24:53 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 23 Sep 2015 11:24:53 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> Message-ID: <5346E5A5-582F-4BB4-BBC1-B2EF83ABA12E@oracle.com> We just realized that: jdk.internal.jvmci.hotspot.HotSpotResolvedJavaMethod.hasBalancedMonitors() is unused so we removed that code plus the test: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/9a268a7f1a12 > On Sep 22, 2015, at 1:13 PM, Christian Thalinger wrote: > > We decided to add one more change: the way JVMCI options are named and passed. > > Instead of one option: > > -Djvmci.options= > > we now use multiple options in the form of: > > -Djvmci.option.= > > Here are the two changesets implementing this: > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/0569806cf96f > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/bb2afad00ef1 > > (the second one is only a cleanup, really). > > No more source code changes after this, I promise. > >> On Sep 21, 2015, at 12:24 PM, Christian Thalinger wrote: >> >> >>> On Sep 18, 2015, at 2:19 PM, Christian Thalinger wrote: >>> >>> >>>> On Sep 18, 2015, at 7:00 AM, Christian Thalinger wrote: >>>> >>>> >>>>> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >>>>> >>>>> To which repository will you finally push these changes? >>>>> >>>>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>>>> they will be integrated there. >>>> >>>> Good point. >>>> >>>>> >>>>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>>>> because of '8135067: Preparatory refactorings for compiler control' >>>>> and "8134626: Misc cleanups after generation array removal": >>>>> >>>>> 3 out of 18 hunks FAILED -- saving rejects to file >>>>> src/share/vm/compiler/compileBroker.cpp.rej >>>>> 1 out of 5 hunks FAILED -- saving rejects to file >>>>> src/share/vm/compiler/compileBroker.hpp.rej >>>>> 6 out of 40 hunks FAILED -- saving rejects to file >>>>> src/share/vm/runtime/vmStructs.cpp.rej >>>>> >>>>> Will you update the patches and if yes against which repositories? >>>> >>>> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >>> >>> Done. Now I have to create new webrevs? >> >> Here are new webrevs against hs-comp: >> >> http://cr.openjdk.java.net/~twisti/8136421/webrev/ >> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ >> >>> >>>> >>>>> >>>>> If I want to work on the ppc64 implementation, which repository should I use? >>>> >>>> graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? >>>> >>>>> >>>>> Thank you and best regards, >>>>> Volker >>>>> >>>>> >>>>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>>>> wrote: >>>>>> For the top-level change I always get a strange error when importing it: >>>>>> >>>>>> patch failed, unable to continue (try -v) >>>>>> patch failed, rejects left in working dir >>>>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>>>> >>>>>> It is because of the strange path syntax of the last hunk in the patch file: >>>>>> >>>>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>> @@ -254,6 +254,14 @@ >>>>>> jdk.jfr >>>>>> >>>>>> >>>>>> + jdk.internal.jvmci.hotspot >>>>>> + jdk.jfr >>>>>> + >>>>>> + >>>>>> + jdk.internal.jvmci.hotspot.events >>>>>> + jdk.jfr >>>>>> + >>>>>> + >>>>>> sun.misc >>>>>> java.corba >>>>>> java.desktop >>>>>> >>>>>> >>>>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>>>> If I remove the redundant './' from the path, everything works fine. >>>>>> For some reason only the diffs for modules.xml has this strange path >>>>>> syntax in the patch. >>>>>> >>>>>> Regards, >>>>>> Volker >>>>>> >>>>>> >>>>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>>>> wrote: >>>>>>> Since there are no objections I?m going to push this? >>>>>>> >>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>>>> >>>>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>>>> >>>>>>>> I would like to add this change: >>>>>>>> >>>>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>>>> Abstract_VM_Version::vm_release(), >>>>>>>> Abstract_VM_Version::vm_info_string(), >>>>>>>> TieredCompilation ? ", tiered" : "", >>>>>>>> +#if INCLUDE_JVMCI >>>>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>>>> +#endif >>>>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>>>> gc_mode(), >>>>>>>> Abstract_VM_Version::vm_platform_string() >>>>>>>> >>>>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>>>> >>>>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>>>> >>>>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>>>> >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>>>> >>>>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>>>> >>>>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>>>> >>>>>>>>>>>>> Here are the webrevs: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>>>> >>>>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>>>> >>>>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>>>> >>>>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>>>> >>>>>>>>>>>> Hi Christian, >>>>>>>>>>>> >>>>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>>>> >>>>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>>>> >>>>>>>>>>>> Some comments: >>>>>>>>>>>> >>>>>>>>>>>> * modules.xml: >>>>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>>>> >>>>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>>>> >>>>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>>>> >>>>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>>>> >>>>>>>>>>> Thanks for the heads up. >>>>>>>>>> >>>>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>>>> >>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>>>> >>>>>>>>>>> Ok, thanks. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> /Magnus >>>>>>>> >>>>>>> >>>> >>> >> > From christian.thalinger at oracle.com Wed Sep 23 21:38:45 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 23 Sep 2015 11:38:45 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <560289FA.1060902@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55FD2E07.6020401@redhat.com> <560289FA.1060902@oracle.com> Message-ID: > On Sep 23, 2015, at 1:16 AM, Vladimir Kozlov wrote: > > It should use next(4) which we have exactly for such cases: > > map->set_callee_saved(YMMHI_STACK_OFFSET(0), xmm0->as_VMReg()->next(4)); > > Note, next() is equal next(1). Done: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6703dfe1e8f9 > > Vladimir > > On 9/19/15 5:42 PM, Andrew Haley wrote: >> On 14/09/15 08:24, Christian Thalinger wrote: >> >> +#if defined(COMPILER2) || INCLUDE_JVMCI >> + if (save_vectors) { >> + assert(ymmhi_offset != -1, "save area must exist"); >> + map->set_callee_saved(YMMHI_STACK_OFFSET( 0), xmm0->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET( 16), xmm1->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET( 32), xmm2->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET( 48), xmm3->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET( 64), xmm4->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET( 80), xmm5->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET( 96), xmm6->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(112), xmm7->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(128), xmm8->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(144), xmm9->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(160), xmm10->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(176), xmm11->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(192), xmm12->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(208), xmm13->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(224), xmm14->as_VMReg()->next()->next()->next()->next()); >> + map->set_callee_saved(YMMHI_STACK_OFFSET(240), xmm15->as_VMReg()->next()->next()->next()->next()); >> + } >> +#endif >> + >> >> Um, really? There is plenty of very odd code in HotSpot, but this is something else >> again. >> >> Andrew. >> From christian.thalinger at oracle.com Wed Sep 23 22:22:23 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Wed, 23 Sep 2015 12:22:23 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <56029051.8020607@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> <56029051.8020607@oracle.com> Message-ID: > On Sep 23, 2015, at 1:43 AM, Vladimir Kozlov wrote: > > Can't say much about top changes but they work so I assume they are correct. > > I looked on C++ code. > > We have a lot #if defined(COMPILER2) || INCLUDE_JVMCI > May be we should define one value #if defined(HIGH_TIER_COMPILER) ? I?m not sure about this. We can keep it in mind and change it later if it turns out it would be useful. > > vmStructs_x86.hpp we may need to add new VM_Version values added recently: > > CPU_RTM = (1 << 24), // Restricted Transactional Memory instructions > CPU_ADX = (1 << 25), > CPU_AVX512F = (1 << 26), // AVX 512bit foundation instructions > ? Might be a good idea. I did add all of them to jdk.internal.jvmci.amd64.AMD64.CPUFeature but I didn?t add code to process them in jdk.internal.jvmci.hotspot.amd64.AMD64HotSpotJVMCIBackendFactory.computeFeatures(HotSpotVMConfig). Do you want me to do that too? http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/f88785857043 > > Several places (14) in new code use %p which we should change to PRT_FROMAT or similar. I only found 13: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/83f08ad44950 > > Vladimir > > On 9/22/15 7:52 AM, Christian Thalinger wrote: >> Thank you, Igor. >> >>> On Sep 21, 2015, at 1:48 PM, Igor Veresov wrote: >>> >>> I?ve seen it all before, but I went through it one more time in its entirety and it looks good. >>> >>> igor >>> >>>> On Sep 21, 2015, at 3:24 PM, Christian Thalinger wrote: >>>> >>>>> >>>> >>>> Here are new webrevs against hs-comp: >>>> >>>> http://cr.openjdk.java.net/~twisti/8136421/webrev/ >>>> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ >>>> >> From kim.barrett at oracle.com Wed Sep 23 23:23:00 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 23 Sep 2015 19:23:00 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <560312D4.7050906@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <560312D4.7050906@oracle.com> Message-ID: On Sep 23, 2015, at 5:00 PM, sangheon.kim wrote: > >>>> src/share/vm/gc/g1/g1_globals.hpp >>>> 169 develop(intx, G1RSetRegionEntriesBase, 256, \ >>>> 173 product(intx, G1RSetRegionEntries, 0, \ >>>> 179 develop(intx, G1RSetSparseRegionEntriesBase, 4, \ >>>> 184 product(intx, G1RSetSparseRegionEntries, 0, \ >>>> 240 product(uintx, G1ConcRefinementThreads, 0, \ >>>> >>>> All of these have their max range value changed to be divided by 4. >>>> What's this about? What is this magic "4"? >>> 'G1RSetRegionEntries', 'G1RSetSparseRegionEntries' and 'G1ConcRefinementThreads' are finally multiplied by pointer size. >>> So we need to divide by 4 for 32bit to avoid overflow. We don't need for 64bit but I thought it will be enough for 64bit. >>> 'G1RSetRegionEntriesBase' and 'G1RSetSparseRegionEntriesBase' changed to have same range as 'G1RSetRegionEntries' and 'G1RSetSparseRegionEntries?. >> Dividing by 4 doesn't account for being multiplied by pointer size on >> a 64bit platform. > You are right. This is limitation for 32bit. > Both 'jint' and 'intx' are 32bit for 32bit platform and these options are used for 'size_t' type parameter after multiplying by pointer size. > So we need to divide by 4 on 32bit platform. > When it comes to 64bit platform, we don't need to divide by 4 as 'size_t' is big enough to cover 'max_jint * 8'. > But I think that value divided by 4 would be enough for 64bit platform. > > Jon also pointed '4' seems strange, so I suggested to use 'jintSize'. > What do you think? > >> In light of that, I don't understand this part of >> your response: "We don't need for 64bit but I thought it will be >> enough for 64bit." > I think I explained above. I see now. jintSize would be better than 4. But I think even better would be pointer size, though I haven?t found an existing constant for that. Maybe sizeof(address)? The only disadvantage to that is the max value is then smaller on 64bit platforms than on 32bit platforms. I doubt the difference is interesting in practice though. From sangheon.kim at oracle.com Wed Sep 23 23:57:31 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Wed, 23 Sep 2015 16:57:31 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <560312D4.7050906@oracle.com> Message-ID: <56033C6B.3060002@oracle.com> On 09/23/2015 04:23 PM, Kim Barrett wrote: > On Sep 23, 2015, at 5:00 PM, sangheon.kim wrote: >>>>> src/share/vm/gc/g1/g1_globals.hpp >>>>> 169 develop(intx, G1RSetRegionEntriesBase, 256, \ >>>>> 173 product(intx, G1RSetRegionEntries, 0, \ >>>>> 179 develop(intx, G1RSetSparseRegionEntriesBase, 4, \ >>>>> 184 product(intx, G1RSetSparseRegionEntries, 0, \ >>>>> 240 product(uintx, G1ConcRefinementThreads, 0, \ >>>>> >>>>> All of these have their max range value changed to be divided by 4. >>>>> What's this about? What is this magic "4"? >>>> 'G1RSetRegionEntries', 'G1RSetSparseRegionEntries' and 'G1ConcRefinementThreads' are finally multiplied by pointer size. >>>> So we need to divide by 4 for 32bit to avoid overflow. We don't need for 64bit but I thought it will be enough for 64bit. >>>> 'G1RSetRegionEntriesBase' and 'G1RSetSparseRegionEntriesBase' changed to have same range as 'G1RSetRegionEntries' and 'G1RSetSparseRegionEntries?. >>> Dividing by 4 doesn't account for being multiplied by pointer size on >>> a 64bit platform. >> You are right. This is limitation for 32bit. >> Both 'jint' and 'intx' are 32bit for 32bit platform and these options are used for 'size_t' type parameter after multiplying by pointer size. >> So we need to divide by 4 on 32bit platform. >> When it comes to 64bit platform, we don't need to divide by 4 as 'size_t' is big enough to cover 'max_jint * 8'. >> But I think that value divided by 4 would be enough for 64bit platform. >> >> Jon also pointed '4' seems strange, so I suggested to use 'jintSize'. >> What do you think? >> >>> In light of that, I don't understand this part of >>> your response: "We don't need for 64bit but I thought it will be >>> enough for 64bit." >> I think I explained above. > I see now. jintSize would be better than 4. But I think even better would be pointer size, > though I haven?t found an existing constant for that. Maybe sizeof(address)? We have 'const int wordSize = sizeof(char*)'. > The only > disadvantage to that is the max value is then smaller on 64bit platforms than on 32bit > platforms. I doubt the difference is interesting in practice though. I agree that there's no big difference in practice. I just wanted to avoid to have smaller upper limit for 64bit platforms. But if you prefer to use 'wordSize' instead of '4', I'm fine. It would give better readability. Thanks, Sangheon > > From staffan.larsen at oracle.com Thu Sep 24 08:15:59 2015 From: staffan.larsen at oracle.com (Staffan Larsen) Date: Thu, 24 Sep 2015 10:15:59 +0200 Subject: RFR (L): 8046148: JEP 158 Unified JVM Logging In-Reply-To: <5602EBEC.7090701@oracle.com> References: <55ED9210.6030904@oracle.com> <55F7E6B5.2040502@oracle.com> <560288C2.5090606@oracle.com> <5602EBEC.7090701@oracle.com> Message-ID: <0FB6EE79-7A68-4DFE-B207-1DDFF7CF7195@oracle.com> > On 23 sep 2015, at 20:14, Coleen Phillimore wrote: > > > Looks good! > Coleen +1 > > On 9/23/15 7:10 AM, Marcus Larsson wrote: >> Hi again, >> >> Here is the third and hopefully final version of the patch. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01-02/ >> >> Full webrev: >> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.02/ >> >> Changes: >> * Use update_position() in logStream::write so that indent() works properly >> * Add units (ms, ns) to all time decorations >> * Additional comments >> >> Thanks, >> Marcus >> >> On 2015-09-15 11:36, Marcus Larsson wrote: >>> Hi, >>> >>> Here is the updated version of the patch, including the feedback and suggestions from Coleen, David and Ioi. >>> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00-01/ >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.01/ >>> >>> Changes include: >>> * New NMT tag 'mtLogging' used by UL >>> * Added storage allocation classes for all new classes >>> * Added Log::puts(const char*) for plain string messages of any length >>> * Fixed broken printf-attribute on Log::write functions >>> * Moved VM startup timestamp to after os::init >>> * Formatting and code style changes >>> >>> Thanks, >>> Marcus >>> >>> >>> On 2015-09-07 15:33, Marcus Larsson wrote: >>>> Hi, >>>> >>>> Please review the following patch adding the unified logging framework to hotspot. >>>> >>>> JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8046148 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~mlarsson/8046148/webrev.00/ >>>> >>>> See the JEP description for a general overview of the new feature. Below are some notes on the implementation. >>>> >>>> The patch adds the new 'share/vm/logging' subdirectory containing the unified logging framework. The main entry point is log.hpp, which contains the necessary macros and definitions to use the framework. >>>> >>>> Log tags are defined/listed in logTag.hpp, and are passed as template arguments to the Log class. Every combination of tags used in a log call has a corresponding LogTagSet instance, which keeps a track of all the outputs it should write the log message to (and their levels). Having tags as template arguments allows mapping directly from a set of tags to the LogTagSet instance, which means that the overhead for disabled logging should be low. Currently each log message can be tagged with up to 5 tags, but this can be increased if ever required (and with C++11's variadic templates the limit can be removed completely). >>>> >>>> The LogConfiguration class keeps track of configured outputs (stdout, stderr, and possible file outputs). Configuration is done either by command line arguments (-Xlog) or by DCMD. Both methods will in turn use the LogConfiguration class to perform the parsing & configuration. This configuration includes iterating over all LogTagSet instances and updating them accordingly. The LogTagLevelExpression class is used to represent the selection of tags and levels for a given configuration request (the "what"-expression). >>>> >>>> The LogDecorators class contains a selection of decorators. Instances of this class is kept in LogTagSet to track what decorators to use (this is the union of all decorators used by its outputs). Each log call will create a LogDecorations instance (note: different classes), which will contain the actual decoration strings for the log message. These decorations are used for each output the tagset is set to log on, and are then discarded. >>>> >>>> The LogPrefix class allows messages of specific sets of tags to be prefixed. The prefix should supply a printf-style format with argument. (This allows GC logging to prefix messages of certain tagsets with GCId.) Prefixes are implemented using template specializations based on the specified tags, with the general/unspecialized case giving an empty prefix. >>>> >>>> The LogOutput class defines the interface for all types of log outputs. LogFileStreamOutput corresponds to FILE* stream based log outputs. LogFileOutput builds on this and adds the file management and log rotation support. >>>> >>>> A simple jtreg test is included in this patch. Additional tests will be added at a later stage. >>>> >>>> Thanks, >>>> Marcus >>> >> > From david.lindholm at oracle.com Thu Sep 24 08:21:10 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Thu, 24 Sep 2015 10:21:10 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <5B4B692B-4EDA-42B3-B648-0A593C85ACF1@oracle.com> References: <56013DCC.4060307@oracle.com> <5B4B692B-4EDA-42B3-B648-0A593C85ACF1@oracle.com> Message-ID: <5603B276.9050900@oracle.com> Kim, Thanks again for looking at this. I have fixed your comments below, and even though you have some files left I have uploaded a new webrev with the changes I have made so far: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.01/ http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00-01/ (differential) I will also file a new RFE to fix src/share/vm/gc/g1/heapRegionSet.cpp per your suggestion. Thanks, David On 2015-09-23 20:32, Kim Barrett wrote: > On Sep 22, 2015, at 7:38 AM, David Lindholm wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ > Another batch, this time covering gc through memory. I need to take a > break from this, but I should be able to finish tomorrow. > > Mostly formatting issues. BTW, although I'm finding some of these, I'm > also noticing that you tidied up a lot more. Thanks for that. > > ------------------------------------------------------------------------------ > src/share/vm/gc/cms/compactibleFreeListSpace.cpp > 2877 assert(n * word_sz == fc->size(), > 2878 "Chunk size " SIZE_FORMAT " is not exactly splittable by " > 2879 SIZE_FORMAT " sized chunks of size " SIZE_FORMAT, > 2880 fc->size(), n, word_sz); > > Pre-existing: Indentation of message arguments. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/bufferingOopClosure.cpp > 157 assert((got) == (expected), \ > 158 "Expected: %d, got: %d, when running testCount(%d, %d, %d)", \ > 159 (got), (expected), num_narrow, num_full, do_oop_order) > > 194 assert((got) == (expected), \ > 195 "Expected: %d, got: %d. testIsBufferEmptyOrFull(%d, %d, %s, %s)", \ > 196 (got), (expected), num_narrow, num_full, \ > 197 BOOL_TO_STR(expect_empty), BOOL_TO_STR(expect_full)) > > 234 assert(boc.is_buffer_empty(), > 235 "Should be empty after call to done(). testEmptyAfterDone(%d, %d)", > 236 num_narrow, num_full); > > Pre-existing: Indentation of message arguments. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1AllocRegion.hpp > 37 #define G1_ALLOC_REGION_MSG(message) \ > 38 "[%s] %s c: %u b: %s r: " PTR_FORMAT " u: " SIZE_FORMAT, \ > 39 _name, message, _count, BOOL_TO_STR(_bot_updates), \ > 40 p2i(_alloc_region), _used_bytes_before > > Misaligned line continuations. > > I think this doesn't belong in this header; it's not something that a > client can use. Suggest moving it to g1AllocRegion.inline.hpp. > > Suggest changing this to G1_ALLOC_REGION_ASSERT, with predicate > argument and full assert expression as expansion. Macros with unusual > syntactic expansions can be confusing, so best avoided unless they > provide some benefit that can't be obtained in some other way. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1AllocRegion.hpp > 49 class G1AllocRegion VALUE_OBJ_CLASS_SPEC { > 50 friend class ar_ext_msg; > > ar_ext_msg no longer exists. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1Allocator.cpp > 432 assert((end_alignment_in_bytes >> LogHeapWordSize) < HeapRegion::min_region_size_in_words(), > 433 "alignment " SIZE_FORMAT " too large", end_alignment_in_bytes); > > Pre-existing: Indentation of message arguments. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1BiasedArray.cpp > 44 guarantee(biased_index >= bias() && biased_index < (bias() + length()), > 45 "Biased index out of bounds, index: " SIZE_FORMAT " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); > > 50 guarantee(biased_index >= bias() && biased_index <= (bias() + length()), > 51 "Biased index out of inclusive bounds, index: " SIZE_FORMAT " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); > > Pre-existing: Indentation of message arguments. > > Maybe also split really long lines? > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1BlockOffsetTable.cpp > 364 assert((_array->offset_array(orig_index) == 0 && > 365 blk_start == boundary) || > 366 (_array->offset_array(orig_index) > 0 && > 367 _array->offset_array(orig_index) <= N_words), > > [Pre-existing, and not modified by this changeset, but nearby.] > > Indentation of 366 and 367 are wrong. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1BlockOffsetTable.hpp > 151 assert(offset <= N_words, > 152 "%s - " > 153 "offset: " SIZE_FORMAT ", N_words: %u", > 154 msg, offset, (uint)N_words); > > Maybe eliminate the line break in the format string? > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1BlockOffsetTable.inline.hpp > 95 assert(result >= _reserved.start() && result < _reserved.end(), > 96 "bad address from index result " PTR_FORMAT > 97 " _reserved.start() " PTR_FORMAT " _reserved.end() " > 98 PTR_FORMAT, > 99 p2i(result), p2i(_reserved.start()), p2i(_reserved.end())); > > Maybe eliminate the second line break in the format string? > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1CollectedHeap.cpp > 2540 assert(_old_marking_cycles_started == _old_marking_cycles_completed || > 2541 _old_marking_cycles_started == _old_marking_cycles_completed + 1, > 2542 "Wrong marking cycle count (started: %d, completed: %d)", > 2543 _old_marking_cycles_started, _old_marking_cycles_completed); > > Pre-existing: Indentation of arguments. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/heapRegionSet.cpp > > Suggest eliminating hrs_ext_msg and hrs_err_msg in a manner similar to > elimination of ar_ext_msg, taking into account earlier comments in > that area. > > Please do that under a new RFE; this change set is quite larg enough > already. > > ------------------------------------------------------------------------------ > src/share/vm/gc/g1/heapRegionType.hpp > 30 #define hrt_assert_is_valid(tag) \ > 31 assert(is_valid((tag)), "invalid HR type: %u", (uint) (tag)) > > Pre-existing: I don't think this should be a macro. I suggest it > should be a private member function using NOT_DEBUG_RETURN and an > ASSERT-only definition in the .cpp. > > https://bugs.openjdk.java.net/browse/JDK-8137046 > > ------------------------------------------------------------------------------ > src/share/vm/gc/parallel/mutableNUMASpace.cpp > 87 assert(words_to_fill >= CollectedHeap::min_fill_size(), > 88 "Remaining size (" SIZE_FORMAT ") is too small to fill (based on " SIZE_FORMAT " and " SIZE_FORMAT ")", > 89 words_to_fill, words_left_to_fill, CollectedHeap::filler_array_max_size()); > > Pre-existing: Indentation of message arguments. > > ------------------------------------------------------------------------------ > src/share/vm/gc/shared/gcCause.hpp > 97 assert(cause != GCCause::_old_generation_too_full_to_scavenge && > 98 cause != GCCause::_old_generation_expanded_on_last_scavenge, > 99 "This GCCause may be correct but is not expected yet: %s", > 100 to_string(cause)); > > Pre-existing: Indentation of arguments. > > ------------------------------------------------------------------------------ > src/share/vm/gc/shared/space.cpp > 606 assert(oop(last)->is_oop(),PTR_FORMAT " should be an object start", p2i(last)); > > Add space between comma and PTR_FORMAT. > > ------------------------------------------------------------------------------ > From david.lindholm at oracle.com Thu Sep 24 08:48:05 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Thu, 24 Sep 2015 10:48:05 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: References: <56013DCC.4060307@oracle.com> <62694939-B653-4838-8DCA-06A0CE5E16F9@oracle.com> <560272F0.2070005@oracle.com> Message-ID: <5603B8C5.70004@oracle.com> Kim, On 2015-09-23 20:36, Kim Barrett wrote: > On Sep 23, 2015, at 5:37 AM, David Lindholm wrote: >>> src/share/vm/code/nmethod.cpp >>> 1712 assert(ic->is_clean(), "nmethod " PTR_FORMAT "not clean %s", from, from->method()->name_and_sig_as_C_string()); >>> ... >>> 2543 fatal("nmethod at " INTPTR_FORMAT " not in zone", this); >>> ... >>> 2551 fatal("findNMethod did not find this nmethod (" INTPTR_FORMAT ")", this); >>> ... >>> 2556 tty->print_cr("\t\tin nmethod at " INTPTR_FORMAT " (pcs)", this); >>> >>> Pre-existing: Don't we need the p2i() dance for the values associated >>> with the [INT]PTR_FORMAT directives? The last isn't touched by your >>> changes, but happens to be nearby. >> You are correct, but this file (and many others) silences GCC warnings for format strings with PRAGMA_FORMAT_MUTE_WARNINGS_FOR_GCC at the top of the file. This should absolutely be fixed in all these files, but fixing all these is a major task, and nothing I want to do in this change. > Thanks for the explanation. And I see there are already some relevant > bug reports, though I didn't see any covering the code directory. I have assigned these bug reports to me now, and I intend to fix all of them regardless of directory. Thanks, David >> I think I'll wait for the rest of your comments before sending out a new webrev. >> > Great. I'll try to finish up soon. > > From per.liden at oracle.com Thu Sep 24 12:56:08 2015 From: per.liden at oracle.com (Per Liden) Date: Thu, 24 Sep 2015 14:56:08 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <5603B276.9050900@oracle.com> References: <56013DCC.4060307@oracle.com> <5B4B692B-4EDA-42B3-B648-0A593C85ACF1@oracle.com> <5603B276.9050900@oracle.com> Message-ID: <5603F2E8.7070607@oracle.com> Hi David, On 2015-09-24 10:21, David Lindholm wrote: > Kim, > > Thanks again for looking at this. I have fixed your comments below, and > even though you have some files left I have uploaded a new webrev with > the changes I have made so far: > > http://cr.openjdk.java.net/~david/JDK-8080775/webrev.01/ > http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00-01/ (differential) > This is a really nice cleanup! Just a few comments: g1AllocRegion.inline.hpp: ------------------------- Could we rename G1_ALLOC_REGION_ASSERT() to something like assert_alloc_region(), to match the style used by assert_heap_locked() and friends in g1CollectedHeap.hpp. heapRegionSet.hpp/heapRegionSet.cpp: ------------------------------------ It feels like we should get rid of hrs_err_msg and hrs_ext_msg here also, and replace it with an assert_heap_region_set() or something, or is there a reason why we're keeping those? vmError.cpp: ------------ VMError::_current_step could be a local variable in report() instead of a static. VMError::_current_step_info could be reset to "" in the END macro, instead of outside of report(). VMError::_verbose could be an argument to report() instead of a static. cheers, /Per > I will also file a new RFE to fix src/share/vm/gc/g1/heapRegionSet.cpp > per your suggestion. > > Thanks, > David > > > On 2015-09-23 20:32, Kim Barrett wrote: >> On Sep 22, 2015, at 7:38 AM, David Lindholm >> wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >>> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >> Another batch, this time covering gc through memory. I need to take a >> break from this, but I should be able to finish tomorrow. >> >> Mostly formatting issues. BTW, although I'm finding some of these, I'm >> also noticing that you tidied up a lot more. Thanks for that. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/cms/compactibleFreeListSpace.cpp >> 2877 assert(n * word_sz == fc->size(), >> 2878 "Chunk size " SIZE_FORMAT " is not exactly splittable by " >> 2879 SIZE_FORMAT " sized chunks of size " SIZE_FORMAT, >> 2880 fc->size(), n, word_sz); >> >> Pre-existing: Indentation of message arguments. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/bufferingOopClosure.cpp >> 157 assert((got) == >> (expected), \ >> 158 "Expected: %d, got: %d, when running testCount(%d, >> %d, %d)", \ >> 159 (got), (expected), num_narrow, num_full, do_oop_order) >> >> 194 assert((got) == >> (expected), \ >> 195 "Expected: %d, got: %d. testIsBufferEmptyOrFull(%d, >> %d, %s, %s)", \ >> 196 (got), (expected), num_narrow, >> num_full, \ >> 197 BOOL_TO_STR(expect_empty), BOOL_TO_STR(expect_full)) >> >> 234 assert(boc.is_buffer_empty(), >> 235 "Should be empty after call to done(). >> testEmptyAfterDone(%d, %d)", >> 236 num_narrow, num_full); >> >> Pre-existing: Indentation of message arguments. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1AllocRegion.hpp >> 37 #define G1_ALLOC_REGION_MSG(message) \ >> 38 "[%s] %s c: %u b: %s r: " PTR_FORMAT " u: " SIZE_FORMAT, \ >> 39 _name, message, _count, BOOL_TO_STR(_bot_updates), \ >> 40 p2i(_alloc_region), _used_bytes_before >> >> Misaligned line continuations. >> >> I think this doesn't belong in this header; it's not something that a >> client can use. Suggest moving it to g1AllocRegion.inline.hpp. >> >> Suggest changing this to G1_ALLOC_REGION_ASSERT, with predicate >> argument and full assert expression as expansion. Macros with unusual >> syntactic expansions can be confusing, so best avoided unless they >> provide some benefit that can't be obtained in some other way. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1AllocRegion.hpp >> 49 class G1AllocRegion VALUE_OBJ_CLASS_SPEC { >> 50 friend class ar_ext_msg; >> >> ar_ext_msg no longer exists. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1Allocator.cpp >> 432 assert((end_alignment_in_bytes >> LogHeapWordSize) < >> HeapRegion::min_region_size_in_words(), >> 433 "alignment " SIZE_FORMAT " too large", >> end_alignment_in_bytes); >> >> Pre-existing: Indentation of message arguments. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1BiasedArray.cpp >> 44 guarantee(biased_index >= bias() && biased_index < (bias() + >> length()), >> 45 "Biased index out of bounds, index: " SIZE_FORMAT " bias: " >> SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); >> >> 50 guarantee(biased_index >= bias() && biased_index <= (bias() + >> length()), >> 51 "Biased index out of inclusive bounds, index: " SIZE_FORMAT >> " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), >> length()); >> >> Pre-existing: Indentation of message arguments. >> >> Maybe also split really long lines? >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1BlockOffsetTable.cpp >> 364 assert((_array->offset_array(orig_index) == 0 && >> 365 blk_start == boundary) || >> 366 (_array->offset_array(orig_index) > 0 && >> 367 _array->offset_array(orig_index) <= N_words), >> >> [Pre-existing, and not modified by this changeset, but nearby.] >> >> Indentation of 366 and 367 are wrong. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1BlockOffsetTable.hpp >> 151 assert(offset <= N_words, >> 152 "%s - " >> 153 "offset: " SIZE_FORMAT ", N_words: %u", >> 154 msg, offset, (uint)N_words); >> >> Maybe eliminate the line break in the format string? >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1BlockOffsetTable.inline.hpp >> 95 assert(result >= _reserved.start() && result < _reserved.end(), >> 96 "bad address from index result " PTR_FORMAT >> 97 " _reserved.start() " PTR_FORMAT " _reserved.end() " >> 98 PTR_FORMAT, >> 99 p2i(result), p2i(_reserved.start()), >> p2i(_reserved.end())); >> >> Maybe eliminate the second line break in the format string? >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/g1CollectedHeap.cpp >> 2540 assert(_old_marking_cycles_started == >> _old_marking_cycles_completed || >> 2541 _old_marking_cycles_started == _old_marking_cycles_completed >> + 1, >> 2542 "Wrong marking cycle count (started: %d, completed: %d)", >> 2543 _old_marking_cycles_started, _old_marking_cycles_completed); >> >> Pre-existing: Indentation of arguments. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/heapRegionSet.cpp >> >> Suggest eliminating hrs_ext_msg and hrs_err_msg in a manner similar to >> elimination of ar_ext_msg, taking into account earlier comments in >> that area. >> >> Please do that under a new RFE; this change set is quite larg enough >> already. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/g1/heapRegionType.hpp >> 30 #define hrt_assert_is_valid(tag) \ >> 31 assert(is_valid((tag)), "invalid HR type: %u", (uint) (tag)) >> >> Pre-existing: I don't think this should be a macro. I suggest it >> should be a private member function using NOT_DEBUG_RETURN and an >> ASSERT-only definition in the .cpp. >> >> https://bugs.openjdk.java.net/browse/JDK-8137046 >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/parallel/mutableNUMASpace.cpp >> 87 assert(words_to_fill >= CollectedHeap::min_fill_size(), >> 88 "Remaining size (" SIZE_FORMAT ") is too small to >> fill (based on " SIZE_FORMAT " and " SIZE_FORMAT ")", >> 89 words_to_fill, words_left_to_fill, >> CollectedHeap::filler_array_max_size()); >> >> Pre-existing: Indentation of message arguments. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/shared/gcCause.hpp >> 97 assert(cause != >> GCCause::_old_generation_too_full_to_scavenge && >> 98 cause != GCCause::_old_generation_expanded_on_last_scavenge, >> 99 "This GCCause may be correct but is not expected yet: %s", >> 100 to_string(cause)); >> >> Pre-existing: Indentation of arguments. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/gc/shared/space.cpp >> 606 assert(oop(last)->is_oop(),PTR_FORMAT " should be an object >> start", p2i(last)); >> >> Add space between comma and PTR_FORMAT. >> >> ------------------------------------------------------------------------------ >> >> > From david.lindholm at oracle.com Thu Sep 24 14:25:38 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Thu, 24 Sep 2015 16:25:38 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <5603F2E8.7070607@oracle.com> References: <56013DCC.4060307@oracle.com> <5B4B692B-4EDA-42B3-B648-0A593C85ACF1@oracle.com> <5603B276.9050900@oracle.com> <5603F2E8.7070607@oracle.com> Message-ID: <560407E2.1080708@oracle.com> Hi Per, Thanks for the review! Comments inline. On 2015-09-24 14:56, Per Liden wrote: > This is a really nice cleanup! Just a few comments: Thanks! > > g1AllocRegion.inline.hpp: > ------------------------- > Could we rename G1_ALLOC_REGION_ASSERT() to something like > assert_alloc_region(), to match the style used by assert_heap_locked() > and friends in g1CollectedHeap.hpp. Fixed. > heapRegionSet.hpp/heapRegionSet.cpp: > ------------------------------------ > It feels like we should get rid of hrs_err_msg and hrs_ext_msg here > also, and replace it with an assert_heap_region_set() or something, or > is there a reason why we're keeping those? Yes, per Kims suggestion I will add another RFE to fix this in another changeset. > vmError.cpp: > ------------ > VMError::_current_step could be a local variable in report() instead > of a static. Yes, but the local variable must still be static, otherwise crashreporting during crashreporting won't work. > VMError::_current_step_info could be reset to "" in the END macro, > instead of outside of report(). Yes, I reset both current_step_info and current_step in END now. > VMError::_verbose could be an argument to report() instead of a static. Fixed. Thanks, David > > cheers, > /Per > > >> I will also file a new RFE to fix src/share/vm/gc/g1/heapRegionSet.cpp >> per your suggestion. >> >> Thanks, >> David >> >> >> On 2015-09-23 20:32, Kim Barrett wrote: >>> On Sep 22, 2015, at 7:38 AM, David Lindholm >>> wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >>>> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >>> Another batch, this time covering gc through memory. I need to take a >>> break from this, but I should be able to finish tomorrow. >>> >>> Mostly formatting issues. BTW, although I'm finding some of these, I'm >>> also noticing that you tidied up a lot more. Thanks for that. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/cms/compactibleFreeListSpace.cpp >>> 2877 assert(n * word_sz == fc->size(), >>> 2878 "Chunk size " SIZE_FORMAT " is not exactly splittable by " >>> 2879 SIZE_FORMAT " sized chunks of size " SIZE_FORMAT, >>> 2880 fc->size(), n, word_sz); >>> >>> Pre-existing: Indentation of message arguments. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/bufferingOopClosure.cpp >>> 157 assert((got) == >>> (expected), \ >>> 158 "Expected: %d, got: %d, when running testCount(%d, >>> %d, %d)", \ >>> 159 (got), (expected), num_narrow, num_full, do_oop_order) >>> >>> 194 assert((got) == >>> (expected), \ >>> 195 "Expected: %d, got: %d. testIsBufferEmptyOrFull(%d, >>> %d, %s, %s)", \ >>> 196 (got), (expected), num_narrow, >>> num_full, \ >>> 197 BOOL_TO_STR(expect_empty), BOOL_TO_STR(expect_full)) >>> >>> 234 assert(boc.is_buffer_empty(), >>> 235 "Should be empty after call to done(). >>> testEmptyAfterDone(%d, %d)", >>> 236 num_narrow, num_full); >>> >>> Pre-existing: Indentation of message arguments. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1AllocRegion.hpp >>> 37 #define G1_ALLOC_REGION_MSG(message) \ >>> 38 "[%s] %s c: %u b: %s r: " PTR_FORMAT " u: " SIZE_FORMAT, \ >>> 39 _name, message, _count, BOOL_TO_STR(_bot_updates), \ >>> 40 p2i(_alloc_region), _used_bytes_before >>> >>> Misaligned line continuations. >>> >>> I think this doesn't belong in this header; it's not something that a >>> client can use. Suggest moving it to g1AllocRegion.inline.hpp. >>> >>> Suggest changing this to G1_ALLOC_REGION_ASSERT, with predicate >>> argument and full assert expression as expansion. Macros with unusual >>> syntactic expansions can be confusing, so best avoided unless they >>> provide some benefit that can't be obtained in some other way. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1AllocRegion.hpp >>> 49 class G1AllocRegion VALUE_OBJ_CLASS_SPEC { >>> 50 friend class ar_ext_msg; >>> >>> ar_ext_msg no longer exists. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1Allocator.cpp >>> 432 assert((end_alignment_in_bytes >> LogHeapWordSize) < >>> HeapRegion::min_region_size_in_words(), >>> 433 "alignment " SIZE_FORMAT " too large", >>> end_alignment_in_bytes); >>> >>> Pre-existing: Indentation of message arguments. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1BiasedArray.cpp >>> 44 guarantee(biased_index >= bias() && biased_index < (bias() + >>> length()), >>> 45 "Biased index out of bounds, index: " SIZE_FORMAT " bias: " >>> SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); >>> >>> 50 guarantee(biased_index >= bias() && biased_index <= (bias() + >>> length()), >>> 51 "Biased index out of inclusive bounds, index: " SIZE_FORMAT >>> " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), >>> length()); >>> >>> Pre-existing: Indentation of message arguments. >>> >>> Maybe also split really long lines? >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1BlockOffsetTable.cpp >>> 364 assert((_array->offset_array(orig_index) == 0 && >>> 365 blk_start == boundary) || >>> 366 (_array->offset_array(orig_index) > 0 && >>> 367 _array->offset_array(orig_index) <= N_words), >>> >>> [Pre-existing, and not modified by this changeset, but nearby.] >>> >>> Indentation of 366 and 367 are wrong. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1BlockOffsetTable.hpp >>> 151 assert(offset <= N_words, >>> 152 "%s - " >>> 153 "offset: " SIZE_FORMAT ", N_words: %u", >>> 154 msg, offset, (uint)N_words); >>> >>> Maybe eliminate the line break in the format string? >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1BlockOffsetTable.inline.hpp >>> 95 assert(result >= _reserved.start() && result < _reserved.end(), >>> 96 "bad address from index result " PTR_FORMAT >>> 97 " _reserved.start() " PTR_FORMAT " _reserved.end() " >>> 98 PTR_FORMAT, >>> 99 p2i(result), p2i(_reserved.start()), >>> p2i(_reserved.end())); >>> >>> Maybe eliminate the second line break in the format string? >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/g1CollectedHeap.cpp >>> 2540 assert(_old_marking_cycles_started == >>> _old_marking_cycles_completed || >>> 2541 _old_marking_cycles_started == _old_marking_cycles_completed >>> + 1, >>> 2542 "Wrong marking cycle count (started: %d, completed: %d)", >>> 2543 _old_marking_cycles_started, _old_marking_cycles_completed); >>> >>> Pre-existing: Indentation of arguments. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/heapRegionSet.cpp >>> >>> Suggest eliminating hrs_ext_msg and hrs_err_msg in a manner similar to >>> elimination of ar_ext_msg, taking into account earlier comments in >>> that area. >>> >>> Please do that under a new RFE; this change set is quite larg enough >>> already. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/g1/heapRegionType.hpp >>> 30 #define hrt_assert_is_valid(tag) \ >>> 31 assert(is_valid((tag)), "invalid HR type: %u", (uint) (tag)) >>> >>> Pre-existing: I don't think this should be a macro. I suggest it >>> should be a private member function using NOT_DEBUG_RETURN and an >>> ASSERT-only definition in the .cpp. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8137046 >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/parallel/mutableNUMASpace.cpp >>> 87 assert(words_to_fill >= CollectedHeap::min_fill_size(), >>> 88 "Remaining size (" SIZE_FORMAT ") is too small to >>> fill (based on " SIZE_FORMAT " and " SIZE_FORMAT ")", >>> 89 words_to_fill, words_left_to_fill, >>> CollectedHeap::filler_array_max_size()); >>> >>> Pre-existing: Indentation of message arguments. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/shared/gcCause.hpp >>> 97 assert(cause != >>> GCCause::_old_generation_too_full_to_scavenge && >>> 98 cause != >>> GCCause::_old_generation_expanded_on_last_scavenge, >>> 99 "This GCCause may be correct but is not expected yet: %s", >>> 100 to_string(cause)); >>> >>> Pre-existing: Indentation of arguments. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/gc/shared/space.cpp >>> 606 assert(oop(last)->is_oop(),PTR_FORMAT " should be an object >>> start", p2i(last)); >>> >>> Add space between comma and PTR_FORMAT. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> >> From kim.barrett at oracle.com Thu Sep 24 17:39:59 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 24 Sep 2015 13:39:59 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <56033C6B.3060002@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <560312D4.7050906@oracle.com> <56033C6B.3060002@oracle.com> Message-ID: <30CDCCA5-2DD9-42C9-9A0F-DAFC53D56730@oracle.com> On Sep 23, 2015, at 7:57 PM, sangheon.kim wrote: > >> I see now. jintSize would be better than 4. But I think even better would be pointer size, >> though I haven?t found an existing constant for that. Maybe sizeof(address)? > We have 'const int wordSize = sizeof(char*)'. > >> The only >> disadvantage to that is the max value is then smaller on 64bit platforms than on 32bit >> platforms. I doubt the difference is interesting in practice though. > I agree that there's no big difference in practice. > I just wanted to avoid to have smaller upper limit for 64bit platforms. > But if you prefer to use 'wordSize' instead of '4', I'm fine. It would give better readability. Yes, please. From christian.thalinger at oracle.com Thu Sep 24 18:57:12 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Thu, 24 Sep 2015 08:57:12 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> Message-ID: <97150580-7329-438F-9B64-F47E3BFE9F08@oracle.com> > On Sep 21, 2015, at 12:24 PM, Christian Thalinger wrote: > > >> On Sep 18, 2015, at 2:19 PM, Christian Thalinger wrote: >> >> >>> On Sep 18, 2015, at 7:00 AM, Christian Thalinger wrote: >>> >>> >>>> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >>>> >>>> To which repository will you finally push these changes? >>>> >>>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>>> they will be integrated there. >>> >>> Good point. >>> >>>> >>>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>>> because of '8135067: Preparatory refactorings for compiler control' >>>> and "8134626: Misc cleanups after generation array removal": >>>> >>>> 3 out of 18 hunks FAILED -- saving rejects to file >>>> src/share/vm/compiler/compileBroker.cpp.rej >>>> 1 out of 5 hunks FAILED -- saving rejects to file >>>> src/share/vm/compiler/compileBroker.hpp.rej >>>> 6 out of 40 hunks FAILED -- saving rejects to file >>>> src/share/vm/runtime/vmStructs.cpp.rej >>>> >>>> Will you update the patches and if yes against which repositories? >>> >>> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >> >> Done. Now I have to create new webrevs? > > Here are new webrevs against hs-comp: > > http://cr.openjdk.java.net/~twisti/8136421/webrev/ > http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ I have refreshed these webrevs. They contain all the changes we discussed here plus a bunch of new tests that SQE finished. > >> >>> >>>> >>>> If I want to work on the ppc64 implementation, which repository should I use? >>> >>> graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? >>> >>>> >>>> Thank you and best regards, >>>> Volker >>>> >>>> >>>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>>> wrote: >>>>> For the top-level change I always get a strange error when importing it: >>>>> >>>>> patch failed, unable to continue (try -v) >>>>> patch failed, rejects left in working dir >>>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>>> >>>>> It is because of the strange path syntax of the last hunk in the patch file: >>>>> >>>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>> @@ -254,6 +254,14 @@ >>>>> jdk.jfr >>>>> >>>>> >>>>> + jdk.internal.jvmci.hotspot >>>>> + jdk.jfr >>>>> + >>>>> + >>>>> + jdk.internal.jvmci.hotspot.events >>>>> + jdk.jfr >>>>> + >>>>> + >>>>> sun.misc >>>>> java.corba >>>>> java.desktop >>>>> >>>>> >>>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>>> If I remove the redundant './' from the path, everything works fine. >>>>> For some reason only the diffs for modules.xml has this strange path >>>>> syntax in the patch. >>>>> >>>>> Regards, >>>>> Volker >>>>> >>>>> >>>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>>> wrote: >>>>>> Since there are no objections I?m going to push this? >>>>>> >>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>>> >>>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>>> >>>>>>> I would like to add this change: >>>>>>> >>>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>>> Abstract_VM_Version::vm_release(), >>>>>>> Abstract_VM_Version::vm_info_string(), >>>>>>> TieredCompilation ? ", tiered" : "", >>>>>>> +#if INCLUDE_JVMCI >>>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>>> +#endif >>>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>>> gc_mode(), >>>>>>> Abstract_VM_Version::vm_platform_string() >>>>>>> >>>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>>> >>>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>>> >>>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>>> >>>>>>>> Vladimir >>>>>>>> >>>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>>> >>>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>>> >>>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>>> >>>>>>>>>>>> Here are the webrevs: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>>> >>>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>>> >>>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>>> >>>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>>> >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>>> >>>>>>>>>>> Hi Christian, >>>>>>>>>>> >>>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>>> >>>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>>> >>>>>>>>>>> Some comments: >>>>>>>>>>> >>>>>>>>>>> * modules.xml: >>>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>>> >>>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>>> >>>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>>> >>>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>>> >>>>>>>>>> Thanks for the heads up. >>>>>>>>> >>>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>>> >>>>>>>>>> Ok, thanks. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> /Magnus >>>>>>> >>>>>> >>> >> > From jon.masamitsu at oracle.com Thu Sep 24 22:16:04 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Thu, 24 Sep 2015 15:16:04 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <82D592A7-5926-47B0-ACEE-85EE2E5488C0@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <5601DE93.2070802@oracle.com> <82D592A7-5926-47B0-ACEE-85EE2E5488C0@oracle.com> Message-ID: <56047624.40402@oracle.com> On 09/23/2015 01:00 PM, Kim Barrett wrote: > On Sep 22, 2015, at 7:04 PM, Jon Masamitsu wrote: >>>> src/share/vm/gc/g1/g1_globals.hpp >>>> 240 product(uintx, G1ConcRefinementThreads, 0, \ >>>> ... >>>> 243 range(0, (max_jint-1)/4) \ >>>> >>>> I know we discussed the problem of thread counts. I'd suggested >>>> perhaps basing the upper bound on the number of processors. (Some >>>> care might be needed for uniprocessor systems.) I couldn't find any >>>> followup on that suggestion though. >> Kim, >> >> My apologies but I dissuaded Sangheon from that idea. I don't think the >> number of processors was enough. A hundred or a thousand times the >> number of processors should be enough but if I had to pick some >> multiplier like that, I'd rather pick something limited by the size >> of the type and back off some. It's still arbitrary but whereas I might >> some decade be surprised that 1000 * #-of-processors was too few, >> I'm not going to see 1,000,000,000 be too few. > I suspect we're using different meanings for "processor". I meant > "number of processing units", e.g. the number of lines in Linux > /proc/cpuinfo that begin with the word "processor". A more precise > term might be "hardware concurrency". A maximum multiplier of 1-2 > seems to me sufficient for these generally CPU-intensive tasks.243 range(0, (max_jint-1)/4) > I try to use the term "hardware threads" which I think matches your definition and I agree that I've never seen a case where a small integer * #of hardware threads was not a reasonable limit. But I think the test programs that check the ranges are generated by looking at lines in the source such as 243 243 range(0, (max_jint-1)/4) If the line had some call to a function to get the number of hardware threads I could see things going wrong - Maybe generation of tests is not on the same machine as run the tests and range is set at the time the test is generated - Maybe the determination of the number of hardware threads is not reliable - Maybe code gets moved around so that the number of hardware threads are not available when argument parsing is done Something simple like a really big number trimmed back some still appeals to me. Jon From per.liden at oracle.com Fri Sep 25 06:15:10 2015 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 Sep 2015 08:15:10 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <560407E2.1080708@oracle.com> References: <56013DCC.4060307@oracle.com> <5B4B692B-4EDA-42B3-B648-0A593C85ACF1@oracle.com> <5603B276.9050900@oracle.com> <5603F2E8.7070607@oracle.com> <560407E2.1080708@oracle.com> Message-ID: <5604E66E.9070303@oracle.com> Hi, On 2015-09-24 16:25, David Lindholm wrote: > Hi Per, > > Thanks for the review! Comments inline. > > > On 2015-09-24 14:56, Per Liden wrote: >> This is a really nice cleanup! Just a few comments: > > Thanks! > >> >> g1AllocRegion.inline.hpp: >> ------------------------- >> Could we rename G1_ALLOC_REGION_ASSERT() to something like >> assert_alloc_region(), to match the style used by assert_heap_locked() >> and friends in g1CollectedHeap.hpp. > > Fixed. > >> heapRegionSet.hpp/heapRegionSet.cpp: >> ------------------------------------ >> It feels like we should get rid of hrs_err_msg and hrs_ext_msg here >> also, and replace it with an assert_heap_region_set() or something, or >> is there a reason why we're keeping those? > > Yes, per Kims suggestion I will add another RFE to fix this in another > changeset. > >> vmError.cpp: >> ------------ >> VMError::_current_step could be a local variable in report() instead >> of a static. > > Yes, but the local variable must still be static, otherwise > crashreporting during crashreporting won't work. Ah, ok. In that case I think it's better to leave next to current_step_info, since that are tightly connected to each other. cheers, /Per > >> VMError::_current_step_info could be reset to "" in the END macro, >> instead of outside of report(). > > Yes, I reset both current_step_info and current_step in END now. > >> VMError::_verbose could be an argument to report() instead of a static. > > Fixed. > > > Thanks, > David > >> >> cheers, >> /Per >> >> >>> I will also file a new RFE to fix src/share/vm/gc/g1/heapRegionSet.cpp >>> per your suggestion. >>> >>> Thanks, >>> David >>> >>> >>> On 2015-09-23 20:32, Kim Barrett wrote: >>>> On Sep 22, 2015, at 7:38 AM, David Lindholm >>>> wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >>>>> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >>>> Another batch, this time covering gc through memory. I need to take a >>>> break from this, but I should be able to finish tomorrow. >>>> >>>> Mostly formatting issues. BTW, although I'm finding some of these, I'm >>>> also noticing that you tidied up a lot more. Thanks for that. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/cms/compactibleFreeListSpace.cpp >>>> 2877 assert(n * word_sz == fc->size(), >>>> 2878 "Chunk size " SIZE_FORMAT " is not exactly splittable by " >>>> 2879 SIZE_FORMAT " sized chunks of size " SIZE_FORMAT, >>>> 2880 fc->size(), n, word_sz); >>>> >>>> Pre-existing: Indentation of message arguments. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/bufferingOopClosure.cpp >>>> 157 assert((got) == >>>> (expected), \ >>>> 158 "Expected: %d, got: %d, when running testCount(%d, >>>> %d, %d)", \ >>>> 159 (got), (expected), num_narrow, num_full, do_oop_order) >>>> >>>> 194 assert((got) == >>>> (expected), \ >>>> 195 "Expected: %d, got: %d. testIsBufferEmptyOrFull(%d, >>>> %d, %s, %s)", \ >>>> 196 (got), (expected), num_narrow, >>>> num_full, \ >>>> 197 BOOL_TO_STR(expect_empty), BOOL_TO_STR(expect_full)) >>>> >>>> 234 assert(boc.is_buffer_empty(), >>>> 235 "Should be empty after call to done(). >>>> testEmptyAfterDone(%d, %d)", >>>> 236 num_narrow, num_full); >>>> >>>> Pre-existing: Indentation of message arguments. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1AllocRegion.hpp >>>> 37 #define G1_ALLOC_REGION_MSG(message) \ >>>> 38 "[%s] %s c: %u b: %s r: " PTR_FORMAT " u: " SIZE_FORMAT, \ >>>> 39 _name, message, _count, BOOL_TO_STR(_bot_updates), \ >>>> 40 p2i(_alloc_region), _used_bytes_before >>>> >>>> Misaligned line continuations. >>>> >>>> I think this doesn't belong in this header; it's not something that a >>>> client can use. Suggest moving it to g1AllocRegion.inline.hpp. >>>> >>>> Suggest changing this to G1_ALLOC_REGION_ASSERT, with predicate >>>> argument and full assert expression as expansion. Macros with unusual >>>> syntactic expansions can be confusing, so best avoided unless they >>>> provide some benefit that can't be obtained in some other way. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1AllocRegion.hpp >>>> 49 class G1AllocRegion VALUE_OBJ_CLASS_SPEC { >>>> 50 friend class ar_ext_msg; >>>> >>>> ar_ext_msg no longer exists. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1Allocator.cpp >>>> 432 assert((end_alignment_in_bytes >> LogHeapWordSize) < >>>> HeapRegion::min_region_size_in_words(), >>>> 433 "alignment " SIZE_FORMAT " too large", >>>> end_alignment_in_bytes); >>>> >>>> Pre-existing: Indentation of message arguments. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1BiasedArray.cpp >>>> 44 guarantee(biased_index >= bias() && biased_index < (bias() + >>>> length()), >>>> 45 "Biased index out of bounds, index: " SIZE_FORMAT " bias: " >>>> SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); >>>> >>>> 50 guarantee(biased_index >= bias() && biased_index <= (bias() + >>>> length()), >>>> 51 "Biased index out of inclusive bounds, index: " SIZE_FORMAT >>>> " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), >>>> length()); >>>> >>>> Pre-existing: Indentation of message arguments. >>>> >>>> Maybe also split really long lines? >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1BlockOffsetTable.cpp >>>> 364 assert((_array->offset_array(orig_index) == 0 && >>>> 365 blk_start == boundary) || >>>> 366 (_array->offset_array(orig_index) > 0 && >>>> 367 _array->offset_array(orig_index) <= N_words), >>>> >>>> [Pre-existing, and not modified by this changeset, but nearby.] >>>> >>>> Indentation of 366 and 367 are wrong. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1BlockOffsetTable.hpp >>>> 151 assert(offset <= N_words, >>>> 152 "%s - " >>>> 153 "offset: " SIZE_FORMAT ", N_words: %u", >>>> 154 msg, offset, (uint)N_words); >>>> >>>> Maybe eliminate the line break in the format string? >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1BlockOffsetTable.inline.hpp >>>> 95 assert(result >= _reserved.start() && result < _reserved.end(), >>>> 96 "bad address from index result " PTR_FORMAT >>>> 97 " _reserved.start() " PTR_FORMAT " _reserved.end() " >>>> 98 PTR_FORMAT, >>>> 99 p2i(result), p2i(_reserved.start()), >>>> p2i(_reserved.end())); >>>> >>>> Maybe eliminate the second line break in the format string? >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/g1CollectedHeap.cpp >>>> 2540 assert(_old_marking_cycles_started == >>>> _old_marking_cycles_completed || >>>> 2541 _old_marking_cycles_started == _old_marking_cycles_completed >>>> + 1, >>>> 2542 "Wrong marking cycle count (started: %d, completed: %d)", >>>> 2543 _old_marking_cycles_started, _old_marking_cycles_completed); >>>> >>>> Pre-existing: Indentation of arguments. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/heapRegionSet.cpp >>>> >>>> Suggest eliminating hrs_ext_msg and hrs_err_msg in a manner similar to >>>> elimination of ar_ext_msg, taking into account earlier comments in >>>> that area. >>>> >>>> Please do that under a new RFE; this change set is quite larg enough >>>> already. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/g1/heapRegionType.hpp >>>> 30 #define hrt_assert_is_valid(tag) \ >>>> 31 assert(is_valid((tag)), "invalid HR type: %u", (uint) (tag)) >>>> >>>> Pre-existing: I don't think this should be a macro. I suggest it >>>> should be a private member function using NOT_DEBUG_RETURN and an >>>> ASSERT-only definition in the .cpp. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8137046 >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/parallel/mutableNUMASpace.cpp >>>> 87 assert(words_to_fill >= CollectedHeap::min_fill_size(), >>>> 88 "Remaining size (" SIZE_FORMAT ") is too small to >>>> fill (based on " SIZE_FORMAT " and " SIZE_FORMAT ")", >>>> 89 words_to_fill, words_left_to_fill, >>>> CollectedHeap::filler_array_max_size()); >>>> >>>> Pre-existing: Indentation of message arguments. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/shared/gcCause.hpp >>>> 97 assert(cause != >>>> GCCause::_old_generation_too_full_to_scavenge && >>>> 98 cause != >>>> GCCause::_old_generation_expanded_on_last_scavenge, >>>> 99 "This GCCause may be correct but is not expected yet: %s", >>>> 100 to_string(cause)); >>>> >>>> Pre-existing: Indentation of arguments. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> src/share/vm/gc/shared/space.cpp >>>> 606 assert(oop(last)->is_oop(),PTR_FORMAT " should be an object >>>> start", p2i(last)); >>>> >>>> Add space between comma and PTR_FORMAT. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> >>> > From zoltan.majo at oracle.com Fri Sep 25 06:36:32 2015 From: zoltan.majo at oracle.com (=?UTF-8?B?Wm9sdMOhbiBNYWrDsw==?=) Date: Fri, 25 Sep 2015 08:36:32 +0200 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F219CE.3010006@oracle.com> References: <55F219CE.3010006@oracle.com> Message-ID: <5604EB70.8060206@oracle.com> Hi Sangheon, I spotted a small issue in TestG1HeapRegionSize.java: You've left a println statement in the code (although you've commented it out). 58 //System.out.println(stdout); Otherwise it looks good to me. Thank you and best regards, Zoltan On 09/11/2015 02:01 AM, sangheon.kim wrote: > Hi all, > > Please review this patch for command-line validation for GC flags. > This REDO patch is adding ranges and implementing constraint functions > for GC flags. > > Original CR of JDK-8078555 was backout as it made a test failure from > 'TestOptionsWithRanges.java'. > And also there were some discussion of OOM handling. > > Most parts are same as JDK-8078555 except below: > 1. Changed 'range' for some flags. > 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags > make this test unstable as it tries to allocate huge amount of memory. > > And below are the suggestion note for JDK-8078555: > 1. Exponential notation for 'double' type variable parse: Previously > there were some discussion for maximum value for double type flags > from code review of JDK-8059557 and JDK-8112746. And Kim and I decided > not to add upper limit unless there are problems with DBL_MAX. And as > 255 is the maximum length that can be passed via command-line, we > introduced exponential notation to avoid this limit. ( arguments.cpp ) > 2. These GC flags ranges are not ideal ranges but ranges which don't > make problem with current source code. > If one flag makes some problem but hard to find good range, I > added some ranges. > 3. There are some constraint functions to avoid overflow. > 4. Test applications are changed: as some of them assumed to be > ParallelGC or to check it's output messages. > 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8134995 > > Webrev: > http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ > http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 > > Testing: > JPRT, UTE(vm.quick-pcl) and > test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. > > Thanks, > Sangheon > From magnus.ihse.bursie at oracle.com Fri Sep 25 10:20:00 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 25 Sep 2015 12:20:00 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <97150580-7329-438F-9B64-F47E3BFE9F08@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> <97150580-7329-438F-9B64-F47E3BFE9F08@oracle.com> Message-ID: <56051FD0.4010303@oracle.com> On 2015-09-24 20:57, Christian Thalinger wrote: >> On Sep 21, 2015, at 12:24 PM, Christian Thalinger wrote: >> >> >>> On Sep 18, 2015, at 2:19 PM, Christian Thalinger wrote: >>> >>> >>>> On Sep 18, 2015, at 7:00 AM, Christian Thalinger wrote: >>>> >>>> >>>>> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >>>>> >>>>> To which repository will you finally push these changes? >>>>> >>>>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>>>> they will be integrated there. >>>> Good point. >>>> >>>>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>>>> because of '8135067: Preparatory refactorings for compiler control' >>>>> and "8134626: Misc cleanups after generation array removal": >>>>> >>>>> 3 out of 18 hunks FAILED -- saving rejects to file >>>>> src/share/vm/compiler/compileBroker.cpp.rej >>>>> 1 out of 5 hunks FAILED -- saving rejects to file >>>>> src/share/vm/compiler/compileBroker.hpp.rej >>>>> 6 out of 40 hunks FAILED -- saving rejects to file >>>>> src/share/vm/runtime/vmStructs.cpp.rej >>>>> >>>>> Will you update the patches and if yes against which repositories? >>>> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >>> Done. Now I have to create new webrevs? >> Here are new webrevs against hs-comp: >> >> http://cr.openjdk.java.net/~twisti/8136421/webrev/ >> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ > I have refreshed these webrevs. They contain all the changes we discussed here plus a bunch of new tests that SQE finished. Hi Christian, I didn't see Volkers proposed fix about undefined-bool-conversion: > If this comes only from adlc, you could fix for adlc only by adding > > WARNINGS_ARE_ERRORS += -Wno-undefined-bool-conversion > > to make/linux/makefiles/adlc.make instead of make/linux/makefiles/gcc.make While this is really a fix for an unrelated issue (support for a newer version of clang), I do believe Volker's suggestion to only add it to adlc is much better, if you really need to take this in as part of the JVMCI integration. Apart from this, the makefile changes (still) look good. /Magnus > >>>>> If I want to work on the ppc64 implementation, which repository should I use? >>>> graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? >>>> >>>>> Thank you and best regards, >>>>> Volker >>>>> >>>>> >>>>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>>>> wrote: >>>>>> For the top-level change I always get a strange error when importing it: >>>>>> >>>>>> patch failed, unable to continue (try -v) >>>>>> patch failed, rejects left in working dir >>>>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>>>> >>>>>> It is because of the strange path syntax of the last hunk in the patch file: >>>>>> >>>>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>> @@ -254,6 +254,14 @@ >>>>>> jdk.jfr >>>>>> >>>>>> >>>>>> + jdk.internal.jvmci.hotspot >>>>>> + jdk.jfr >>>>>> + >>>>>> + >>>>>> + jdk.internal.jvmci.hotspot.events >>>>>> + jdk.jfr >>>>>> + >>>>>> + >>>>>> sun.misc >>>>>> java.corba >>>>>> java.desktop >>>>>> >>>>>> >>>>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>>>> If I remove the redundant './' from the path, everything works fine. >>>>>> For some reason only the diffs for modules.xml has this strange path >>>>>> syntax in the patch. >>>>>> >>>>>> Regards, >>>>>> Volker >>>>>> >>>>>> >>>>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>>>> wrote: >>>>>>> Since there are no objections I?m going to push this? >>>>>>> >>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>>>> >>>>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>>>> >>>>>>>> I would like to add this change: >>>>>>>> >>>>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>>>> Abstract_VM_Version::vm_release(), >>>>>>>> Abstract_VM_Version::vm_info_string(), >>>>>>>> TieredCompilation ? ", tiered" : "", >>>>>>>> +#if INCLUDE_JVMCI >>>>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>>>> +#endif >>>>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>>>> gc_mode(), >>>>>>>> Abstract_VM_Version::vm_platform_string() >>>>>>>> >>>>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>>>> >>>>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>>>> >>>>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>>>> >>>>>>>>> Vladimir >>>>>>>>> >>>>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>>>> >>>>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>>>> >>>>>>>>>>>>> Here are the webrevs: >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>>>> >>>>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>>>> >>>>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>>>> >>>>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>>>> >>>>>>>>>>>> Hi Christian, >>>>>>>>>>>> >>>>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>>>> >>>>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>>>> >>>>>>>>>>>> Some comments: >>>>>>>>>>>> >>>>>>>>>>>> * modules.xml: >>>>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>>>> >>>>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>>>> >>>>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>>>> >>>>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>>>> Thanks for the heads up. >>>>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>>>> >>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>>>> >>>>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>>>> Ok, thanks. >>>>>>>>>>> >>>>>>>>>>>> /Magnus From david.lindholm at oracle.com Fri Sep 25 12:52:56 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Fri, 25 Sep 2015 14:52:56 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <5604E66E.9070303@oracle.com> References: <56013DCC.4060307@oracle.com> <5B4B692B-4EDA-42B3-B648-0A593C85ACF1@oracle.com> <5603B276.9050900@oracle.com> <5603F2E8.7070607@oracle.com> <560407E2.1080708@oracle.com> <5604E66E.9070303@oracle.com> Message-ID: <560543A8.3060303@oracle.com> Per and Kim, The latest webrev is here: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.02/ It contains fixes for Pers comments and for the comments from Kim that I have gotten so far. Thanks, David On 2015-09-25 08:15, Per Liden wrote: > Hi, > > On 2015-09-24 16:25, David Lindholm wrote: >> Hi Per, >> >> Thanks for the review! Comments inline. >> >> >> On 2015-09-24 14:56, Per Liden wrote: >>> This is a really nice cleanup! Just a few comments: >> >> Thanks! >> >>> >>> g1AllocRegion.inline.hpp: >>> ------------------------- >>> Could we rename G1_ALLOC_REGION_ASSERT() to something like >>> assert_alloc_region(), to match the style used by assert_heap_locked() >>> and friends in g1CollectedHeap.hpp. >> >> Fixed. >> >>> heapRegionSet.hpp/heapRegionSet.cpp: >>> ------------------------------------ >>> It feels like we should get rid of hrs_err_msg and hrs_ext_msg here >>> also, and replace it with an assert_heap_region_set() or something, or >>> is there a reason why we're keeping those? >> >> Yes, per Kims suggestion I will add another RFE to fix this in another >> changeset. >> >>> vmError.cpp: >>> ------------ >>> VMError::_current_step could be a local variable in report() instead >>> of a static. >> >> Yes, but the local variable must still be static, otherwise >> crashreporting during crashreporting won't work. > > Ah, ok. In that case I think it's better to leave next to > current_step_info, since that are tightly connected to each other. > > cheers, > /Per > >> >>> VMError::_current_step_info could be reset to "" in the END macro, >>> instead of outside of report(). >> >> Yes, I reset both current_step_info and current_step in END now. >> >>> VMError::_verbose could be an argument to report() instead of a static. >> >> Fixed. >> >> >> Thanks, >> David >> >>> >>> cheers, >>> /Per >>> >>> >>>> I will also file a new RFE to fix src/share/vm/gc/g1/heapRegionSet.cpp >>>> per your suggestion. >>>> >>>> Thanks, >>>> David >>>> >>>> >>>> On 2015-09-23 20:32, Kim Barrett wrote: >>>>> On Sep 22, 2015, at 7:38 AM, David Lindholm >>>>> wrote: >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8080775 >>>>>> Webrev: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >>>>> Another batch, this time covering gc through memory. I need to >>>>> take a >>>>> break from this, but I should be able to finish tomorrow. >>>>> >>>>> Mostly formatting issues. BTW, although I'm finding some of >>>>> these, I'm >>>>> also noticing that you tidied up a lot more. Thanks for that. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/cms/compactibleFreeListSpace.cpp >>>>> 2877 assert(n * word_sz == fc->size(), >>>>> 2878 "Chunk size " SIZE_FORMAT " is not exactly splittable by " >>>>> 2879 SIZE_FORMAT " sized chunks of size " SIZE_FORMAT, >>>>> 2880 fc->size(), n, word_sz); >>>>> >>>>> Pre-existing: Indentation of message arguments. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/bufferingOopClosure.cpp >>>>> 157 assert((got) == >>>>> (expected), \ >>>>> 158 "Expected: %d, got: %d, when running testCount(%d, >>>>> %d, %d)", \ >>>>> 159 (got), (expected), num_narrow, num_full, >>>>> do_oop_order) >>>>> >>>>> 194 assert((got) == >>>>> (expected), \ >>>>> 195 "Expected: %d, got: %d. testIsBufferEmptyOrFull(%d, >>>>> %d, %s, %s)", \ >>>>> 196 (got), (expected), num_narrow, >>>>> num_full, \ >>>>> 197 BOOL_TO_STR(expect_empty), >>>>> BOOL_TO_STR(expect_full)) >>>>> >>>>> 234 assert(boc.is_buffer_empty(), >>>>> 235 "Should be empty after call to done(). >>>>> testEmptyAfterDone(%d, %d)", >>>>> 236 num_narrow, num_full); >>>>> >>>>> Pre-existing: Indentation of message arguments. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1AllocRegion.hpp >>>>> 37 #define G1_ALLOC_REGION_MSG(message) \ >>>>> 38 "[%s] %s c: %u b: %s r: " PTR_FORMAT " u: " SIZE_FORMAT, \ >>>>> 39 _name, message, _count, BOOL_TO_STR(_bot_updates), \ >>>>> 40 p2i(_alloc_region), _used_bytes_before >>>>> >>>>> Misaligned line continuations. >>>>> >>>>> I think this doesn't belong in this header; it's not something that a >>>>> client can use. Suggest moving it to g1AllocRegion.inline.hpp. >>>>> >>>>> Suggest changing this to G1_ALLOC_REGION_ASSERT, with predicate >>>>> argument and full assert expression as expansion. Macros with >>>>> unusual >>>>> syntactic expansions can be confusing, so best avoided unless they >>>>> provide some benefit that can't be obtained in some other way. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1AllocRegion.hpp >>>>> 49 class G1AllocRegion VALUE_OBJ_CLASS_SPEC { >>>>> 50 friend class ar_ext_msg; >>>>> >>>>> ar_ext_msg no longer exists. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1Allocator.cpp >>>>> 432 assert((end_alignment_in_bytes >> LogHeapWordSize) < >>>>> HeapRegion::min_region_size_in_words(), >>>>> 433 "alignment " SIZE_FORMAT " too large", >>>>> end_alignment_in_bytes); >>>>> >>>>> Pre-existing: Indentation of message arguments. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1BiasedArray.cpp >>>>> 44 guarantee(biased_index >= bias() && biased_index < (bias() + >>>>> length()), >>>>> 45 "Biased index out of bounds, index: " SIZE_FORMAT " bias: " >>>>> SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), length()); >>>>> >>>>> 50 guarantee(biased_index >= bias() && biased_index <= (bias() + >>>>> length()), >>>>> 51 "Biased index out of inclusive bounds, index: " SIZE_FORMAT >>>>> " bias: " SIZE_FORMAT " length: " SIZE_FORMAT, biased_index, bias(), >>>>> length()); >>>>> >>>>> Pre-existing: Indentation of message arguments. >>>>> >>>>> Maybe also split really long lines? >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1BlockOffsetTable.cpp >>>>> 364 assert((_array->offset_array(orig_index) == 0 && >>>>> 365 blk_start == boundary) || >>>>> 366 (_array->offset_array(orig_index) > 0 && >>>>> 367 _array->offset_array(orig_index) <= N_words), >>>>> >>>>> [Pre-existing, and not modified by this changeset, but nearby.] >>>>> >>>>> Indentation of 366 and 367 are wrong. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1BlockOffsetTable.hpp >>>>> 151 assert(offset <= N_words, >>>>> 152 "%s - " >>>>> 153 "offset: " SIZE_FORMAT ", N_words: %u", >>>>> 154 msg, offset, (uint)N_words); >>>>> >>>>> Maybe eliminate the line break in the format string? >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1BlockOffsetTable.inline.hpp >>>>> 95 assert(result >= _reserved.start() && result < >>>>> _reserved.end(), >>>>> 96 "bad address from index result " PTR_FORMAT >>>>> 97 " _reserved.start() " PTR_FORMAT " _reserved.end() " >>>>> 98 PTR_FORMAT, >>>>> 99 p2i(result), p2i(_reserved.start()), >>>>> p2i(_reserved.end())); >>>>> >>>>> Maybe eliminate the second line break in the format string? >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/g1CollectedHeap.cpp >>>>> 2540 assert(_old_marking_cycles_started == >>>>> _old_marking_cycles_completed || >>>>> 2541 _old_marking_cycles_started == _old_marking_cycles_completed >>>>> + 1, >>>>> 2542 "Wrong marking cycle count (started: %d, completed: %d)", >>>>> 2543 _old_marking_cycles_started, _old_marking_cycles_completed); >>>>> >>>>> Pre-existing: Indentation of arguments. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/heapRegionSet.cpp >>>>> >>>>> Suggest eliminating hrs_ext_msg and hrs_err_msg in a manner >>>>> similar to >>>>> elimination of ar_ext_msg, taking into account earlier comments in >>>>> that area. >>>>> >>>>> Please do that under a new RFE; this change set is quite larg enough >>>>> already. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/g1/heapRegionType.hpp >>>>> 30 #define hrt_assert_is_valid(tag) \ >>>>> 31 assert(is_valid((tag)), "invalid HR type: %u", (uint) (tag)) >>>>> >>>>> Pre-existing: I don't think this should be a macro. I suggest it >>>>> should be a private member function using NOT_DEBUG_RETURN and an >>>>> ASSERT-only definition in the .cpp. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8137046 >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/parallel/mutableNUMASpace.cpp >>>>> 87 assert(words_to_fill >= >>>>> CollectedHeap::min_fill_size(), >>>>> 88 "Remaining size (" SIZE_FORMAT ") is too small to >>>>> fill (based on " SIZE_FORMAT " and " SIZE_FORMAT ")", >>>>> 89 words_to_fill, words_left_to_fill, >>>>> CollectedHeap::filler_array_max_size()); >>>>> >>>>> Pre-existing: Indentation of message arguments. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/shared/gcCause.hpp >>>>> 97 assert(cause != >>>>> GCCause::_old_generation_too_full_to_scavenge && >>>>> 98 cause != >>>>> GCCause::_old_generation_expanded_on_last_scavenge, >>>>> 99 "This GCCause may be correct but is not expected yet: >>>>> %s", >>>>> 100 to_string(cause)); >>>>> >>>>> Pre-existing: Indentation of arguments. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> src/share/vm/gc/shared/space.cpp >>>>> 606 assert(oop(last)->is_oop(),PTR_FORMAT " should be an object >>>>> start", p2i(last)); >>>>> >>>>> Add space between comma and PTR_FORMAT. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> >>>> >> From sangheon.kim at oracle.com Fri Sep 25 16:01:06 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Fri, 25 Sep 2015 09:01:06 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <5604EB70.8060206@oracle.com> References: <55F219CE.3010006@oracle.com> <5604EB70.8060206@oracle.com> Message-ID: <56056FC2.1070701@oracle.com> Hi Zoltan, Thank you for reviewing this! On 09/24/2015 11:36 PM, Zolt?n Maj? wrote: > Hi Sangheon, > > > I spotted a small issue in TestG1HeapRegionSize.java: You've left a > println statement in the code (although you've commented it out). > > 58 //System.out.println(stdout); Nice! I'll include this at next webrev. Thanks, Sangheon > > > Otherwise it looks good to me. > > Thank you and best regards, > > > Zoltan > > > On 09/11/2015 02:01 AM, sangheon.kim wrote: >> Hi all, >> >> Please review this patch for command-line validation for GC flags. >> This REDO patch is adding ranges and implementing constraint >> functions for GC flags. >> >> Original CR of JDK-8078555 was backout as it made a test failure from >> 'TestOptionsWithRanges.java'. >> And also there were some discussion of OOM handling. >> >> Most parts are same as JDK-8078555 except below: >> 1. Changed 'range' for some flags. >> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These flags >> make this test unstable as it tries to allocate huge amount of memory. >> >> And below are the suggestion note for JDK-8078555: >> 1. Exponential notation for 'double' type variable parse: Previously >> there were some discussion for maximum value for double type flags >> from code review of JDK-8059557 and JDK-8112746. And Kim and I >> decided not to add upper limit unless there are problems with >> DBL_MAX. And as 255 is the maximum length that can be passed via >> command-line, we introduced exponential notation to avoid this limit. >> ( arguments.cpp ) >> 2. These GC flags ranges are not ideal ranges but ranges which don't >> make problem with current source code. >> If one flag makes some problem but hard to find good range, I >> added some ranges. >> 3. There are some constraint functions to avoid overflow. >> 4. Test applications are changed: as some of them assumed to be >> ParallelGC or to check it's output messages. >> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8134995 >> >> Webrev: >> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >> >> Testing: >> JPRT, UTE(vm.quick-pcl) and >> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >> >> Thanks, >> Sangheon >> > From sangheon.kim at oracle.com Fri Sep 25 17:42:16 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Fri, 25 Sep 2015 10:42:16 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <30CDCCA5-2DD9-42C9-9A0F-DAFC53D56730@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <560312D4.7050906@oracle.com> <56033C6B.3060002@oracle.com> <30CDCCA5-2DD9-42C9-9A0F-DAFC53D56730@oracle.com> Message-ID: <56058778.3080302@oracle.com> Hi Kim, Jon and Zoltan, Here's next version of webrev. This includes: - changed from '4' to 'wordSize' at g1_globals.hpp. (Kim and Jon? ) - removed test code which is already commented out at TestG1HeapRegionSize.java. (Zoltan) webrev: cr.openjdk.java.net/~sangheki/8134995/webrev.01 cr.openjdk.java.net/~sangheki/8134995/webrev.01_to_00 Thanks, Sangheon On 09/24/2015 10:39 AM, Kim Barrett wrote: > On Sep 23, 2015, at 7:57 PM, sangheon.kim wrote: >>> I see now. jintSize would be better than 4. But I think even better would be pointer size, >>> though I haven?t found an existing constant for that. Maybe sizeof(address)? >> We have 'const int wordSize = sizeof(char*)'. >> >>> The only >>> disadvantage to that is the max value is then smaller on 64bit platforms than on 32bit >>> platforms. I doubt the difference is interesting in practice though. >> I agree that there's no big difference in practice. >> I just wanted to avoid to have smaller upper limit for 64bit platforms. >> But if you prefer to use 'wordSize' instead of '4', I'm fine. It would give better readability. > Yes, please. > From kim.barrett at oracle.com Fri Sep 25 18:26:10 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 25 Sep 2015 14:26:10 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <56058778.3080302@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <560312D4.7050906@oracle.com> <56033C6B.3060002@oracle.com> <30CDCCA5-2DD9-42C9-9A0F-DAFC53D56730@oracle.com> <56058778.3080302@oracle.com> Message-ID: On Sep 25, 2015, at 1:42 PM, sangheon.kim wrote: > > Hi Kim, Jon and Zoltan, > > Here's next version of webrev. > This includes: > - changed from '4' to 'wordSize' at g1_globals.hpp. (Kim and Jon? ) > - removed test code which is already commented out at TestG1HeapRegionSize.java. (Zoltan) > > webrev: > cr.openjdk.java.net/~sangheki/8134995/webrev.01 > cr.openjdk.java.net/~sangheki/8134995/webrev.01_to_00 Looks good. From kim.barrett at oracle.com Fri Sep 25 18:47:46 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 25 Sep 2015 14:47:46 -0400 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <56047624.40402@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <5601DE93.2070802@oracle.com> <82D592A7-5926-47B0-ACEE-85EE2E5488C0@oracle.com> <56047624.40402@oracle.com> Message-ID: <80AE71B0-6E35-43EB-964C-CE57AD541587@oracle.com> On Sep 24, 2015, at 6:16 PM, Jon Masamitsu wrote: > > But I think the test programs that check the ranges are > generated by looking at lines in the source such as 243 > > 243 range(0, (max_jint-1)/4) The particularly interesting test right now is TestOptionsWithRanges, which looks at the recorded range bounds in the VM running the test, and launches a new VM with option values based on those discovered range values. > If the line had some call to a function to get the number of hardware > threads I could see things going wrong > - Maybe generation of tests is not on the same machine as run the tests and > range is set at the time the test is generated Yes, this situation would break TestOptionsWithRanges as presently written. We don?t presently do anything like that though, that I know of. > - Maybe the determination of the number of hardware threads is not reliable We have lots of code that uses that number, so such a problem would be a significant bug. > - Maybe code gets moved around so that the number of hardware threads > are not available when argument parsing is done We use the number of hardware threads in the ergonomic setting of a number of options. > Something simple like a really big number trimmed back some still > appeals to me. My issue with that is that we then need to disable the associated option test, and we may need to make the code that uses the option be defensive against wacky values that we?d rather have filtered out by argument parsing. I realize that?s a complex problem though, and not always amenable to a good solution. And in this case the code using the option likely still needs to be defensive, since even a ?reasonable? value might exhaust available resources. If the using code needs to be defensive anyway, then I don?t see the point in having some more or less randomly chosen upper bound for argument checking. And we don?t need a lower bound check since the type of the option is unsigned. From christian.thalinger at oracle.com Fri Sep 25 20:00:40 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 25 Sep 2015 10:00:40 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <56051FD0.4010303@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> <97150580-7329-438F-9B64-F47E3BFE9F08@oracle.com> <56051FD0.4010303@oracle.com> Message-ID: <1F931EBE-F2C0-413D-BFE4-787362577384@oracle.com> > On Sep 25, 2015, at 12:20 AM, Magnus Ihse Bursie wrote: > > On 2015-09-24 20:57, Christian Thalinger wrote: >>> On Sep 21, 2015, at 12:24 PM, Christian Thalinger wrote: >>> >>> >>>> On Sep 18, 2015, at 2:19 PM, Christian Thalinger wrote: >>>> >>>> >>>>> On Sep 18, 2015, at 7:00 AM, Christian Thalinger wrote: >>>>> >>>>> >>>>>> On Sep 17, 2015, at 11:00 PM, Volker Simonis wrote: >>>>>> >>>>>> To which repository will you finally push these changes? >>>>>> >>>>>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>>>>> they will be integrated there. >>>>> Good point. >>>>> >>>>>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>>>>> because of '8135067: Preparatory refactorings for compiler control' >>>>>> and "8134626: Misc cleanups after generation array removal": >>>>>> >>>>>> 3 out of 18 hunks FAILED -- saving rejects to file >>>>>> src/share/vm/compiler/compileBroker.cpp.rej >>>>>> 1 out of 5 hunks FAILED -- saving rejects to file >>>>>> src/share/vm/compiler/compileBroker.hpp.rej >>>>>> 6 out of 40 hunks FAILED -- saving rejects to file >>>>>> src/share/vm/runtime/vmStructs.cpp.rej >>>>>> >>>>>> Will you update the patches and if yes against which repositories? >>>>> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >>>> Done. Now I have to create new webrevs? >>> Here are new webrevs against hs-comp: >>> >>> http://cr.openjdk.java.net/~twisti/8136421/webrev/ >>> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ >> I have refreshed these webrevs. They contain all the changes we discussed here plus a bunch of new tests that SQE finished. > > Hi Christian, > > I didn't see Volkers proposed fix about undefined-bool-conversion: > >> If this comes only from adlc, you could fix for adlc only by adding >> >> WARNINGS_ARE_ERRORS += -Wno-undefined-bool-conversion >> >> to make/linux/makefiles/adlc.make instead of make/linux/makefiles/gcc.make > > While this is really a fix for an unrelated issue (support for a newer version of clang), I do believe Volker's suggestion to only add it to adlc is much better, if you really need to take this in as part of the JVMCI integration. Sorry I dropped this. After Gilles? hint I remembered the code in question was removed with: [JDK-8041620] Solaris Studio 12.4 C++ 5.13 change in behavior for placing friend declarations within surrounding scope - Java Bug System and the remaining ones were fixed with: [JDK-8077364] "if( !this )" construct prevents build on Xcode 6.3 - Java Bug System I?ve removed the change: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6de55f563ee1 > > Apart from this, the makefile changes (still) look good. Btw. we found a bug in creating the OptionDescriptors files; we get duplicate entries: $ cat build/macosx-x86_64-normal-server-release/jdk/modules/java.base/META-INF/services/jdk.internal.jvmci.options.OptionDescriptors jdk.internal.jvmci.compiler.Compiler_OptionDescriptors jdk.internal.jvmci.hotspot.HotSpotConstantReflectionProvider_OptionDescriptors jdk.internal.jvmci.hotspot.HotSpotResolvedJavaFieldImpl_OptionDescriptors jdk.internal.jvmci.hotspot.HotSpotResolvedJavaMethodImpl_OptionDescriptors jdk.internal.jvmci.compiler.Compiler_OptionDescriptors jdk.internal.jvmci.hotspot.HotSpotConstantReflectionProvider_OptionDescriptors jdk.internal.jvmci.hotspot.HotSpotResolvedJavaFieldImpl_OptionDescriptors jdk.internal.jvmci.hotspot.HotSpotResolvedJavaMethodImpl_OptionDescriptors ? Would this be the right fix? diff -r db1a815d2f6c make/gensrc/Gensrc-java.base.gmk --- a/make/gensrc/Gensrc-java.base.gmk Thu Sep 24 15:35:49 2015 -1000 +++ b/make/gensrc/Gensrc-java.base.gmk Fri Sep 25 18:18:35 2015 +0200 @@ -94,6 +94,7 @@ $(GENSRC_DIR)/_gensrc_proc_done $(MKDIR) -p $(@D) ($(CD) $(GENSRC_DIR)/META-INF/jvmci.options && \ + $(RM) -f $@; \ for i in $$(ls); do \ echo $${i}_OptionDescriptors >> $@; \ done) And I see the same behavior for HotSpotJVMCIBackendFactory: $ cat build/macosx-x86_64-normal-server-release/jdk/modules/java.base/META-INF/services/jdk.internal.jvmci.hotspot.HotSpotJVMCIBackendFactory jdk.internal.jvmci.hotspot.amd64.AMD64HotSpotJVMCIBackendFactory jdk.internal.jvmci.hotspot.sparc.SPARCHotSpotJVMCIBackendFactory jdk.internal.jvmci.hotspot.amd64.AMD64HotSpotJVMCIBackendFactory jdk.internal.jvmci.hotspot.sparc.SPARCHotSpotJVMCIBackendFactory ? So I think a similar fix needs to be applied there. > > /Magnus > >> >>>>>> If I want to work on the ppc64 implementation, which repository should I use? >>>>> graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? >>>>> >>>>>> Thank you and best regards, >>>>>> Volker >>>>>> >>>>>> >>>>>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>>>>> wrote: >>>>>>> For the top-level change I always get a strange error when importing it: >>>>>>> >>>>>>> patch failed, unable to continue (try -v) >>>>>>> patch failed, rejects left in working dir >>>>>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>>>>> >>>>>>> It is because of the strange path syntax of the last hunk in the patch file: >>>>>>> >>>>>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>>> @@ -254,6 +254,14 @@ >>>>>>> jdk.jfr >>>>>>> >>>>>>> >>>>>>> + jdk.internal.jvmci.hotspot >>>>>>> + jdk.jfr >>>>>>> + >>>>>>> + >>>>>>> + jdk.internal.jvmci.hotspot.events >>>>>>> + jdk.jfr >>>>>>> + >>>>>>> + >>>>>>> sun.misc >>>>>>> java.corba >>>>>>> java.desktop >>>>>>> >>>>>>> >>>>>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>>>>> If I remove the redundant './' from the path, everything works fine. >>>>>>> For some reason only the diffs for modules.xml has this strange path >>>>>>> syntax in the patch. >>>>>>> >>>>>>> Regards, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>>>>> wrote: >>>>>>>> Since there are no objections I?m going to push this? >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>>>>> >>>>>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>>>>> >>>>>>>>> I would like to add this change: >>>>>>>>> >>>>>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>>>>> Abstract_VM_Version::vm_release(), >>>>>>>>> Abstract_VM_Version::vm_info_string(), >>>>>>>>> TieredCompilation ? ", tiered" : "", >>>>>>>>> +#if INCLUDE_JVMCI >>>>>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>>>>> +#endif >>>>>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>>>>> gc_mode(), >>>>>>>>> Abstract_VM_Version::vm_platform_string() >>>>>>>>> >>>>>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>>>>> >>>>>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>>>>> >>>>>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>>>>> >>>>>>>>>> Vladimir >>>>>>>>>> >>>>>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Here are the webrevs: >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>>>>> >>>>>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>>>>> >>>>>>>>>>>>> Hi Christian, >>>>>>>>>>>>> >>>>>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>>>>> >>>>>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>>>>> >>>>>>>>>>>>> Some comments: >>>>>>>>>>>>> >>>>>>>>>>>>> * modules.xml: >>>>>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>>>>> >>>>>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>>>>> >>>>>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>>>>> >>>>>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>>>>> Thanks for the heads up. >>>>>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>>>>> >>>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>>>>> >>>>>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>>>>> Ok, thanks. >>>>>>>>>>>> >>>>>>>>>>>>> /Magnus From sangheon.kim at oracle.com Fri Sep 25 22:52:31 2015 From: sangheon.kim at oracle.com (sangheon.kim) Date: Fri, 25 Sep 2015 15:52:31 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <55F6D878.1070903@oracle.com> References: <55F219CE.3010006@oracle.com> <55F319B8.1000803@oracle.com> <55F32A81.8000307@oracle.com> <55F3E484.70107@oracle.com> <55F6D878.1070903@oracle.com> Message-ID: <5605D02F.6070307@oracle.com> Hi Gerard, I found that I missed your comment from webrev.01. Here's a new one which includes yours for TestOptionsWithRanges.java. http://cr.openjdk.java.net/~sangheki/8134995/webrev.02/ http://cr.openjdk.java.net/~sangheki/8134995/webrev.02_to_01 Thanks, Sangheon On 09/14/2015 07:23 AM, gerard ziemski wrote: > Thank you. I have no more comments - reviewed. > > > cheers > > > On 09/12/2015 03:38 AM, sangheon.kim wrote: >> Hi Gerard, >> >> On 09/11/2015 12:24 PM, sangheon.kim wrote: >>> Hi Gerard, >>> >>> Thank you for looking at this. >>> >>> On 09/11/2015 11:13 AM, gerard ziemski wrote: >>>> hi Sangheon, >>>> >>>> #1 >>>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>> >>>> Please change the comment to: >>>> >>>> + /* >>>> + * Exclude below options as their maximum value would >>>> consume too much memory >>>> + * and would affect other tests that run in parallel. >>>> + */ >>> Okay, I will fix as you suggested. >>> >>>> >>>> >>>> #2 What tests did you run? Did you run >>>> test/runtime/CommandLine/OptionsValidation on all platforms >>>> (including embedded)? >>> No. >>> I ran tests under test/runtime/CommandLine/OptionsValidation >>> (especially TestOptionsWithRanges.java) for all platforms >>> except embedded. >>> Let me back after testing on embedded. >> I ran for embedded (linux-arm64, linux-armvh, linux-armvfpsflt, >> linux-armvfphflt, linux-armsflt) and all of them PASSED >> for test/runtime/CommandLine/OptionsValidation. >> >> Thanks, >> Sangheon >> >> >>> >>> Thanks, >>> Sangheon >>> >>> >>>> >>>> >>>> cheers >>>> >>>> >>>> On 09/10/2015 07:01 PM, sangheon.kim wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch for command-line validation for GC flags. >>>>> This REDO patch is adding ranges and implementing constraint >>>>> functions for GC flags. >>>>> >>>>> Original CR of JDK-8078555 was backout as it made a test failure >>>>> from 'TestOptionsWithRanges.java'. >>>>> And also there were some discussion of OOM handling. >>>>> >>>>> Most parts are same as JDK-8078555 except below: >>>>> 1. Changed 'range' for some flags. >>>>> 2. Excluded 3 flags for TestOptionsWithRanges.java test. These >>>>> flags make this test unstable as it tries to allocate >>>>> huge amount of memory. >>>>> >>>>> And below are the suggestion note for JDK-8078555: >>>>> 1. Exponential notation for 'double' type variable parse: >>>>> Previously there were some discussion for maximum value for >>>>> double type flags from code review of JDK-8059557 and JDK-8112746. >>>>> And Kim and I decided not to add upper limit unless >>>>> there are problems with DBL_MAX. And as 255 is the maximum length >>>>> that can be passed via command-line, we introduced >>>>> exponential notation to avoid this limit. ( arguments.cpp ) >>>>> 2. These GC flags ranges are not ideal ranges but ranges which >>>>> don't make problem with current source code. >>>>> If one flag makes some problem but hard to find good range, I >>>>> added some ranges. >>>>> 3. There are some constraint functions to avoid overflow. >>>>> 4. Test applications are changed: as some of them assumed to be >>>>> ParallelGC or to check it's output messages. >>>>> 5. Includes cleanup of JDK-8133565: GC -2nd followup to JDK-8059557. >>>>> >>>>> CR: >>>>> https://bugs.openjdk.java.net/browse/JDK-8134995 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00/ >>>>> http://cr.openjdk.java.net/~sangheki/8134995/webrev.00_to_8078555 >>>>> >>>>> Testing: >>>>> JPRT, UTE(vm.quick-pcl) and >>>>> test/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java. >>>>> >>>>> >>>>> Thanks, >>>>> Sangheon >>>>> >>>>> >>> >> >> From jon.masamitsu at oracle.com Fri Sep 25 22:57:22 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Fri, 25 Sep 2015 15:57:22 -0700 Subject: RFR 8134995(M): [REDO] GC: implement ranges (optionally constraints) for those flags that have them missing In-Reply-To: <80AE71B0-6E35-43EB-964C-CE57AD541587@oracle.com> References: <55F219CE.3010006@oracle.com> <9B6DBBBB-F2ED-4E8A-8ABF-7882856398EA@oracle.com> <56008D3B.5010008@oracle.com> <5601DE93.2070802@oracle.com> <82D592A7-5926-47B0-ACEE-85EE2E5488C0@oracle.com> <56047624.40402@oracle.com> <80AE71B0-6E35-43EB-964C-CE57AD541587@oracle.com> Message-ID: <5605D152.2030402@oracle.com> On 9/25/2015 11:47 AM, Kim Barrett wrote: > On Sep 24, 2015, at 6:16 PM, Jon Masamitsu wrote: >> But I think the test programs that check the ranges are >> generated by looking at lines in the source such as 243 >> >> 243 range(0, (max_jint-1)/4) > The particularly interesting test right now is TestOptionsWithRanges, which looks at > the recorded range bounds in the VM running the test, and launches a new VM with > option values based on those discovered range values. > >> If the line had some call to a function to get the number of hardware >> threads I could see things going wrong >> - Maybe generation of tests is not on the same machine as run the tests and >> range is set at the time the test is generated > Yes, this situation would break TestOptionsWithRanges as presently written. We don?t > presently do anything like that though, that I know of. OK. I won't be concerned about that. > >> - Maybe the determination of the number of hardware threads is not reliable > We have lots of code that uses that number, so such a problem would be a significant bug. Yes, lots of policy depends on the number but I don't think any of it would break if the number is different run to run. And the small integer multiplier will hide any of that variation. > >> - Maybe code gets moved around so that the number of hardware threads >> are not available when argument parsing is done > We use the number of hardware threads in the ergonomic setting of a number of options. Ergonomics sets options non-optimally but don't fail in a way that is easily noticed. > >> Something simple like a really big number trimmed back some still >> appeals to me. > My issue with that is that we then need to disable the associated option test, and we may need > to make the code that uses the option be defensive against wacky values that we?d rather have > filtered out by argument parsing. I realize that?s a complex problem though, and not always > amenable to a good solution. And in this case the code using the option likely still needs to be > defensive, since even a ?reasonable? value might exhaust available resources. My problem is that what seems "wacky" to everyone may not in the end be "wacky". > > If the using code needs to be defensive anyway, then I don?t see the point in having some more > or less randomly chosen upper bound for argument checking. And we don?t need a lower bound > check since the type of the option is unsigned. I agree with this as long as anything that is specified on the command line for ParallelGCThreads will fit into a uint. After some reality checks here I think removing the range check in favor of a constraint function where we could add more defensive checks would be an improvement. Jon > From sebastian.sickelmann at gmx.de Sun Sep 27 16:58:04 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Sun, 27 Sep 2015 18:58:04 +0200 Subject: RFR: JDK-5108778 Too many instances of java.lang.Boolean created in Java application Message-ID: <5608201C.4020109@gmx.de> Hello, my name is Sebastian Sickelmann and i signed the OCA. Actually I am searching through the JBS for low hanging fruits. Right now i am looking through the openjdk-sources and try to evaluate if i can make something about JDK-5108778. As I am not an author, I am actually not able to host webrevs on cr.openjdk.java.net. Is there someone who would support at hosting the hotspot-part of JDK-5108778 on cr.openjdk.java.net for reviewing? I placed the hotspot part in my dropbox at: https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev/index.html or as zip: https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev.zip I executed all jtreg tests in hotspot/test and get no more errors than before the change. -- Sebastian From manasthakur17 at gmail.com Mon Sep 28 08:59:59 2015 From: manasthakur17 at gmail.com (Manas Thakur) Date: Mon, 28 Sep 2015 14:29:59 +0530 Subject: Speeding up the build process of OpenJDK Message-ID: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> Hello all, I use OpenJDK 8 built on Mac OS X Yosemite. The compilation process after doing even minor changes is very slow (takes about 15-20 minutes; with ccache enabled). Is it because the gcc version I have to use on Yosemite is older (gcc 4.2 supplied with Xcode 4.6.3)? If yes, any ideas on speeding up the incremental builds? Regards, Manas From magnus.ihse.bursie at oracle.com Mon Sep 28 09:16:59 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 28 Sep 2015 11:16:59 +0200 Subject: Speeding up the build process of OpenJDK In-Reply-To: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> References: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> Message-ID: <5609058B.5080902@oracle.com> On 2015-09-28 10:59, Manas Thakur wrote: > Hello all, > > I use OpenJDK 8 built on Mac OS X Yosemite. The compilation process after doing even minor changes is very slow (takes about 15-20 minutes; with ccache enabled). Is it because the gcc version I have to use on Yosemite is older (gcc 4.2 supplied with Xcode 4.6.3)? If yes, any ideas on speeding up the incremental builds? Is this for building only hotspot? E.g., "make hotspot"? /Magnus From manasthakur17 at gmail.com Mon Sep 28 09:22:43 2015 From: manasthakur17 at gmail.com (Manas Thakur) Date: Mon, 28 Sep 2015 14:52:43 +0530 Subject: Speeding up the build process of OpenJDK In-Reply-To: <5609058B.5080902@oracle.com> References: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> <5609058B.5080902@oracle.com> Message-ID: <9A51A451-BF2D-4B9D-982E-5BF6D1707709@gmail.com> No, it includes the images (make images). But the changes I do are usually only in hotspot. Regards, Manas > On 28-Sep-2015, at 2:46 pm, Magnus Ihse Bursie wrote: > > On 2015-09-28 10:59, Manas Thakur wrote: >> Hello all, >> >> I use OpenJDK 8 built on Mac OS X Yosemite. The compilation process after doing even minor changes is very slow (takes about 15-20 minutes; with ccache enabled). Is it because the gcc version I have to use on Yosemite is older (gcc 4.2 supplied with Xcode 4.6.3)? If yes, any ideas on speeding up the incremental builds? > > Is this for building only hotspot? E.g., "make hotspot"? > > /Magnus From magnus.ihse.bursie at oracle.com Mon Sep 28 09:22:35 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 28 Sep 2015 11:22:35 +0200 Subject: Speeding up the build process of OpenJDK In-Reply-To: <9A51A451-BF2D-4B9D-982E-5BF6D1707709@gmail.com> References: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> <5609058B.5080902@oracle.com> <9A51A451-BF2D-4B9D-982E-5BF6D1707709@gmail.com> Message-ID: <560906DB.3080706@oracle.com> On 2015-09-28 11:22, Manas Thakur wrote: > No, it includes the images (make images). But the changes I do are usually only in hotspot. Building complete images is slower than a default build ("exploded image") and most often not needed. Try doing it in separate steps: make hotspot make jdk make images and check how long each step takes. /Magnus > > Regards, > Manas > >> On 28-Sep-2015, at 2:46 pm, Magnus Ihse Bursie wrote: >> >> On 2015-09-28 10:59, Manas Thakur wrote: >>> Hello all, >>> >>> I use OpenJDK 8 built on Mac OS X Yosemite. The compilation process after doing even minor changes is very slow (takes about 15-20 minutes; with ccache enabled). Is it because the gcc version I have to use on Yosemite is older (gcc 4.2 supplied with Xcode 4.6.3)? If yes, any ideas on speeding up the incremental builds? >> Is this for building only hotspot? E.g., "make hotspot"? >> >> /Magnus From magnus.ihse.bursie at oracle.com Mon Sep 28 09:25:11 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 28 Sep 2015 11:25:11 +0200 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <1F931EBE-F2C0-413D-BFE4-787362577384@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> <97150580-7329-438F-9B64-F47E3BFE9F08@oracle.com> <56051FD0.4010303@oracle.com> <1F931EBE-F2C0-413D-BFE4-787362577384@oracle.com> Message-ID: <56090777.2090807@oracle.com> On 2015-09-25 22:00, Christian Thalinger wrote: > >> On Sep 25, 2015, at 12:20 AM, Magnus Ihse Bursie >> > > wrote: >> >> On 2015-09-24 20:57, Christian Thalinger wrote: >>>> On Sep 21, 2015, at 12:24 PM, Christian Thalinger >>>> >>> > wrote: >>>> >>>> >>>>> On Sep 18, 2015, at 2:19 PM, Christian Thalinger >>>>> >>>> > wrote: >>>>> >>>>> >>>>>> On Sep 18, 2015, at 7:00 AM, Christian Thalinger >>>>>> >>>>> > wrote: >>>>>> >>>>>> >>>>>>> On Sep 17, 2015, at 11:00 PM, Volker Simonis >>>>>>> > wrote: >>>>>>> >>>>>>> To which repository will you finally push these changes? >>>>>>> >>>>>>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>>>>>> they will be integrated there. >>>>>> Good point. >>>>>> >>>>>>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>>>>>> because of '8135067: Preparatory refactorings for compiler control' >>>>>>> and "8134626: Misc cleanups after generation array removal": >>>>>>> >>>>>>> 3 out of 18 hunks FAILED -- saving rejects to file >>>>>>> src/share/vm/compiler/compileBroker.cpp.rej >>>>>>> 1 out of 5 hunks FAILED -- saving rejects to file >>>>>>> src/share/vm/compiler/compileBroker.hpp.rej >>>>>>> 6 out of 40 hunks FAILED -- saving rejects to file >>>>>>> src/share/vm/runtime/vmStructs.cpp.rej >>>>>>> >>>>>>> Will you update the patches and if yes against which repositories? >>>>>> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >>>>> Done. Now I have to create new webrevs? >>>> Here are new webrevs against hs-comp: >>>> >>>> http://cr.openjdk.java.net/~twisti/8136421/webrev/ >>>> >>>> >>> > >>>> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ >>>> >>>> >>> > >>> I have refreshed these webrevs. They contain all the changes we >>> discussed here plus a bunch of new tests that SQE finished. >> >> Hi Christian, >> >> I didn't see Volkers proposed fix about undefined-bool-conversion: >> >>> If this comes only from adlc, you could fix for adlc only by adding >>> >>> WARNINGS_ARE_ERRORS += -Wno-undefined-bool-conversion >>> >>> to make/linux/makefiles/adlc.make instead of >>> make/linux/makefiles/gcc.make >> >> While this is really a fix for an unrelated issue (support for a >> newer version of clang), I do believe Volker's suggestion to only add >> it to adlc is much better, if you really need to take this in as part >> of the JVMCI integration. > > Sorry I dropped this. After Gilles? hint I remembered the code in > question was removed with: > > [JDK-8041620] Solaris Studio 12.4 C++ 5.13 change in behavior for > placing friend declarations within surrounding scope - Java Bug System > > > and the remaining ones were fixed with: > > [JDK-8077364] "if( !this )" construct prevents build on Xcode 6.3 - > Java Bug System > > I?ve removed the change: > > http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6de55f563ee1 Good! > >> >> Apart from this, the makefile changes (still) look good. > > Btw. we found a bug in creating the OptionDescriptors files; we get > duplicate entries: > > $ cat > build/macosx-x86_64-normal-server-release/jdk/modules/java.base/META-INF/services/jdk.internal.jvmci.options.OptionDescriptors > > jdk.internal.jvmci.compiler.Compiler_OptionDescriptors > jdk.internal.jvmci.hotspot.HotSpotConstantReflectionProvider_OptionDescriptors > jdk.internal.jvmci.hotspot.HotSpotResolvedJavaFieldImpl_OptionDescriptors > jdk.internal.jvmci.hotspot.HotSpotResolvedJavaMethodImpl_OptionDescriptors > jdk.internal.jvmci.compiler.Compiler_OptionDescriptors > jdk.internal.jvmci.hotspot.HotSpotConstantReflectionProvider_OptionDescriptors > jdk.internal.jvmci.hotspot.HotSpotResolvedJavaFieldImpl_OptionDescriptors > jdk.internal.jvmci.hotspot.HotSpotResolvedJavaMethodImpl_OptionDescriptors > ? > > Would this be the right fix? > > diff -r db1a815d2f6c make/gensrc/Gensrc-java.base.gmk > --- a/make/gensrc/Gensrc-java.base.gmkThu Sep 24 15:35:49 2015 -1000 > +++ b/make/gensrc/Gensrc-java.base.gmkFri Sep 25 18:18:35 2015 +0200 > @@ -94,6 +94,7 @@ > $(GENSRC_DIR)/_gensrc_proc_done > $(MKDIR) -p $(@D) > ($(CD) $(GENSRC_DIR)/META-INF/jvmci.options && \ > + $(RM) -f $@; \ > for i in $$(ls); do \ > echo $${i}_OptionDescriptors >> $@; \ > done) > That seems like a reasonable fix, yes. /Magnus > And I see the same behavior for HotSpotJVMCIBackendFactory: > > $ cat > build/macosx-x86_64-normal-server-release/jdk/modules/java.base/META-INF/services/jdk.internal.jvmci.hotspot.HotSpotJVMCIBackendFactory > > jdk.internal.jvmci.hotspot.amd64.AMD64HotSpotJVMCIBackendFactory > jdk.internal.jvmci.hotspot.sparc.SPARCHotSpotJVMCIBackendFactory > jdk.internal.jvmci.hotspot.amd64.AMD64HotSpotJVMCIBackendFactory > jdk.internal.jvmci.hotspot.sparc.SPARCHotSpotJVMCIBackendFactory > ? > > So I think a similar fix needs to be applied there. > >> >> /Magnus >> >>> >>>>>>> If I want to work on the ppc64 implementation, which repository >>>>>>> should I use? >>>>>> graal-jvmci-9. Are you working on this for fun or do you want to >>>>>> have that integrated with this JEP? >>>>>> >>>>>>> Thank you and best regards, >>>>>>> Volker >>>>>>> >>>>>>> >>>>>>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>>>>>> > wrote: >>>>>>>> For the top-level change I always get a strange error when >>>>>>>> importing it: >>>>>>>> >>>>>>>> patch failed, unable to continue (try -v) >>>>>>>> patch failed, rejects left in working dir >>>>>>>> errors during apply, please fix and refresh >>>>>>>> 8062493_jvmci_top_0911.v1.patch >>>>>>>> >>>>>>>> It is because of the strange path syntax of the last hunk in >>>>>>>> the patch file: >>>>>>>> >>>>>>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>>>> @@ -254,6 +254,14 @@ >>>>>>>> jdk.jfr >>>>>>>> >>>>>>>> >>>>>>>> + jdk.internal.jvmci.hotspot >>>>>>>> + jdk.jfr >>>>>>>> + >>>>>>>> + >>>>>>>> + jdk.internal.jvmci.hotspot.events >>>>>>>> + jdk.jfr >>>>>>>> + >>>>>>>> + >>>>>>>> sun.misc >>>>>>>> java.corba >>>>>>>> java.desktop >>>>>>>> >>>>>>>> >>>>>>>> Notice the strange syntax "old/./modules.xml" and >>>>>>>> "new/./modules.xml". >>>>>>>> If I remove the redundant './' from the path, everything works >>>>>>>> fine. >>>>>>>> For some reason only the diffs for modules.xml has this strange >>>>>>>> path >>>>>>>> syntax in the patch. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Volker >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>>>>>> >>>>>>> > wrote: >>>>>>>>> Since there are no objections I?m going to push this? >>>>>>>>> >>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>>>>>> >>>>>>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I would like to add this change: >>>>>>>>>> >>>>>>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 >>>>>>>>>> 14:28:33 2015 -1000 >>>>>>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 >>>>>>>>>> 15:04:02 2015 -1000 >>>>>>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>>>>>> Abstract_VM_Version::vm_release(), >>>>>>>>>> Abstract_VM_Version::vm_info_string(), >>>>>>>>>> TieredCompilation ? ", tiered" : "", >>>>>>>>>> +#if INCLUDE_JVMCI >>>>>>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>>>>>> +#endif >>>>>>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>>>>>> gc_mode(), >>>>>>>>>> Abstract_VM_Version::vm_platform_string() >>>>>>>>>> >>>>>>>>>> It would be useful to see in the crash logs if this >>>>>>>>>> experimental feature was turned on. >>>>>>>>>> >>>>>>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>>>>>> >>>>>>>>>>> Vladimir >>>>>>>>>>> >>>>>>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Here are the webrevs: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The change has already undergone a few iterations of >>>>>>>>>>>>>>> internal review by different people of different teams. >>>>>>>>>>>>>>> Most comments and suggestions were handled accordingly. >>>>>>>>>>>>>>> Although there is one open item we agreed we will >>>>>>>>>>>>>>> address after the integration of JEP 243 and that work >>>>>>>>>>>>>>> is captured in RFE: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> SQE is still working on the tests and all test tasks can >>>>>>>>>>>>>>> be seen as sub-tasks of the JEP. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Christian, >>>>>>>>>>>>>> >>>>>>>>>>>>>> (Adding build-dev since this review includes some major >>>>>>>>>>>>>> build changes.) >>>>>>>>>>>>>> >>>>>>>>>>>>>> The makefile changes looks good in general to me. I have >>>>>>>>>>>>>> not reviewed the source code changes. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Some comments: >>>>>>>>>>>>>> >>>>>>>>>>>>>> * modules.xml: >>>>>>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for >>>>>>>>>>>>>> modifying this file. >>>>>>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>>>>>> >>>>>>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>>>>>> Seems unfortunate to have to disable a new warning >>>>>>>>>>>>>> (undefined-bool-conversion) for newly incorporated code. >>>>>>>>>>>>>> Is it not possible to fix the code emitting this warning >>>>>>>>>>>>>> instead? >>>>>>>>>>>>> We had this question before. It?s not obvious because you >>>>>>>>>>>>> can?t see it in the regular diff views but this is under: >>>>>>>>>>>>> >>>>>>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>>>>>> >>>>>>>>>>>>>> * make/common/MakeBase.gmk and >>>>>>>>>>>>>> hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now >>>>>>>>>>>>>> a new function that performs the same function as >>>>>>>>>>>>>> CreatePath, but it's named PathList. (It's a bit >>>>>>>>>>>>>> unfortunate that this code snippet has bounced around as >>>>>>>>>>>>>> patches without a definite name.) I assume you are going >>>>>>>>>>>>>> to push this through a hotspot forest. If the PathList >>>>>>>>>>>>>> patch reaches the hotspot repo before this, please remove >>>>>>>>>>>>>> the CreatePath from MakeBase, and change the calls to >>>>>>>>>>>>>> CreatePath to PathList instead. (I could only find such >>>>>>>>>>>>>> calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If >>>>>>>>>>>>>> this patch goes in before that, we'll need to give a >>>>>>>>>>>>>> heads-up to the integrator about this conflict. >>>>>>>>>>>>> Thanks for the heads up. >>>>>>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve >>>>>>>>>>>> integrated two changesets: >>>>>>>>>>>> >>>>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab >>>>>>>>>>>> >>>>>>>>>>> > >>>>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 >>>>>>>>>>>> >>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>>>> Another potential coming merge conflict relates to >>>>>>>>>>>>>> ListPathsSafely in Gensrc-java.base.gmk. There is >>>>>>>>>>>>>> currenlty a review out from Erik which modifies the API >>>>>>>>>>>>>> for ListPathsSafely. If/when it goes in, the call to >>>>>>>>>>>>>> ListPathsSafely in Gensrc-java.base.gmk will need to be >>>>>>>>>>>>>> modified (Erik can give advice on how). Depending on >>>>>>>>>>>>>> timing, this too might hit the integrator rather than >>>>>>>>>>>>>> your push. >>>>>>>>>>>>> Ok, thanks. >>>>>>>>>>>>> >>>>>>>>>>>>>> /Magnus > From manasthakur17 at gmail.com Mon Sep 28 10:20:48 2015 From: manasthakur17 at gmail.com (Manas Thakur) Date: Mon, 28 Sep 2015 15:50:48 +0530 Subject: Speeding up the build process of OpenJDK In-Reply-To: <560906DB.3080706@oracle.com> References: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> <5609058B.5080902@oracle.com> <9A51A451-BF2D-4B9D-982E-5BF6D1707709@gmail.com> <560906DB.3080706@oracle.com> Message-ID: I could observe that hotspot itself takes more than 90% of the time. The compilation first takes approximately two minutes before the generation of the precompiled header. Then the compilation of various files takes about 15-20 minutes. Regards, Manas > On 28-Sep-2015, at 2:52 pm, Magnus Ihse Bursie wrote: > > On 2015-09-28 11:22, Manas Thakur wrote: >> No, it includes the images (make images). But the changes I do are usually only in hotspot. > Building complete images is slower than a default build ("exploded image") and most often not needed. > > Try doing it in separate steps: > make hotspot > make jdk > make images > > and check how long each step takes. > > /Magnus > >> >> Regards, >> Manas >> >>> On 28-Sep-2015, at 2:46 pm, Magnus Ihse Bursie wrote: >>> >>> On 2015-09-28 10:59, Manas Thakur wrote: >>>> Hello all, >>>> >>>> I use OpenJDK 8 built on Mac OS X Yosemite. The compilation process after doing even minor changes is very slow (takes about 15-20 minutes; with ccache enabled). Is it because the gcc version I have to use on Yosemite is older (gcc 4.2 supplied with Xcode 4.6.3)? If yes, any ideas on speeding up the incremental builds? >>> Is this for building only hotspot? E.g., "make hotspot"? >>> >>> /Magnus > From bengt.rutisson at oracle.com Mon Sep 28 13:21:05 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Mon, 28 Sep 2015 15:21:05 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <55F17982.3080606@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> Message-ID: <56093EC1.9090109@oracle.com> Hi Jon and Per, I've been trying to get the version proposed in webrev.02 to work but I ran into a show stopper for this approach. The G1 concurrent mark thread is at times executing (and logging) at the same time as a young or mixed GC is executing. There is no good way of handling this with a common place to store the GC ID. Instead I've decided to go back to storing the current GC ID in the thread. That way the G1 concurrent marking and the young/mixed GCs will have their own GC IDs. Here's an updated webrev: http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ This is the same as webrev.01 that both of you already looked at. I've made two minor adjustments: - I removed the currentNamedthread() function in gcId.cpp that Per pointed out in his review of webrev.01. Instead I use Thread::current() directly. - I made GCIdMark a StackObj. Otherwise the patch is the same as in webrev.01. Thanks, Bengt On 2015-09-10 14:37, Bengt Rutisson wrote: > > Hi Jon, > > Thanks for looking at this! > > On 2015-09-10 01:00, Jon Masamitsu wrote: >> Bengt, >> >> When a CMS concurrent collection hands off to a STW foreground >> collection, >> (done in acquire_control_and_collect()), I expected a new GCId would be >> used. Did I miss it? That STW collection does not go through >> do_collection(). > > The call to acquire_control_and_collect() originates from > GenCollectedHeap::collect_generation() and there is a new GCIdMark in > there that will create a new GCId for the STW collection. That's the > "extra" GCIdMark that Per asked about in his review. > > *But* I wanted to verify that it worked properly since you asked about > it and I realized that there is another bug. > > The GCId that is set up for the concurrent cycle, the one set up in > ConcurrentMarkSweepThread::run(), will is still active over the > complete STW foreground collection. That's fine in my model since the > new GCIdMark in GenCollectedHeap::collect_generation() will cache that > GCId. But the ConcurrentMarkSweep thread is not completely idle even > though control has been left to the foreground collection in > acquire_control_and_collect(). So, there is some logging done (by fore > example CMSPhaseAccounting) that is done at the same time as the > foreground collection. This logging will now use the foreground GCId > instead of the CMS GCId. > > I haven't had time to dig in to the details of that just yet. But this > is an unintended change of the logging, so I would like to fix it. > Hopefully I'll have an updated webrev tomorrow. > > Thanks, > Bengt > > >> >> Jon >> >> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>> >>> Hi again, >>> >>> I found an issue with how G1 young collections kick off concurrent >>> marks. There is a short window where we might have two active GC IDs >>> at the same time. I've fixed this and updated the webrevs. In case >>> anyone had already started looking at the webrevs you need to do a >>> refresh now. The fix is in >>> G1CollectedHeap::do_collection_pause_at_safepoint(). >>> >>> Thanks, >>> Bengt >>> >>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>> >>>> Hi Per, >>>> >>>> Thanks for looking at this! >>>> >>>> On 2015-09-08 15:23, Per Liden wrote: >>>>> Hi Bengt, >>>>> >>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>> >>>>>> Hi everyone, >>>>>> >>>>>> This is mostly a GC related patch, but it adds a field to the Thread >>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>> >>>>>> Could I have a couple of reviews for this patch? >>>>>> >>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>> >>>>> >>>>> Looks good. I think this is a nice simplification, especially for >>>>> G1's concurrent parts. Just two comments: >>>>> >>>>> genCollectedHeap.cpp: >>>>> --------------------- >>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>> different scopes. Do we really want that second mark? >>>> >>>> >>>> We potentially do two GCs in GenCollectedHeap::do_collection(). >>>> First a young GC and then potentially a full GC. These two should >>>> have different GC IDs. So, yes, we need two GCIdMarks in this >>>> method. The scoping could be better, but I think that is a >>>> refactoring that should be done separately from my current patch. >>>> I'll talk to Jesper about it since he has been cleaning up this >>>> code lately. >>>> >>>>> >>>>> >>>>> gcId.cpp: >>>>> -------- >>>>> - I think you might have left currentNamedthread() in there. You >>>>> probably just want to use Thread::current() instead. >>>> >>>> Yes, good catch. Thanks. >>>> >>>> However, after thinking some more about the implications of moving >>>> the GC ID out from the GCTracer I figured it would be possible to >>>> just store the previous GC ID in the GCIdMark and then restore it >>>> in the destructor of that class. That way we don't have to store >>>> anything in the Thread class but can still have both a concurrent >>>> GC ID and a STW GC ID active at the same time. This also removes >>>> the need to copy the GC ID to the worker tasks. >>>> >>>> Here's an updated webrev with a solution that does not add anything >>>> to the Thread class: >>>> >>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>> >>>> And here's a diff compared to the last version: >>>> >>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>> >>>> Unfotunately the webrev tool had some bad luck when creating the >>>> diff webrev so at least the g1CollectedHeap.cpp seem to contain the >>>> complete changes rather than just the diff compared to the 01 >>>> version. The rest of the diff looks correct as far as I can tell. >>>> >>>> Thanks, >>>> Bengt >>>>> >>>>> >>>>> cheers, >>>>> /Per >>>>> >>>>> >>>>>> Currently the GC ID, that is used for event tracing and logging, is >>>>>> stored in the GCTracer object. That has caused some minor >>>>>> problems since >>>>>> there are multiple GCTracers active at the same time. The correct >>>>>> GC IDs >>>>>> need to be passed around in many places. >>>>>> >>>>>> For some upcoming GC logging changes I would like to have a more >>>>>> consistent way of finding the currently active GC ID. I've played >>>>>> around >>>>>> with a couple of different solutions, but eventually I found that >>>>>> the >>>>>> simplest solution is to store the current GC ID in the thread. >>>>>> That way >>>>>> there is a single way to look it up in all places. It is also fairly >>>>>> straight forward to set it up. >>>>>> >>>>>> I've reworked the GCId class a bit to support this and I've >>>>>> introduced a >>>>>> GCIdMark class that is a scoped object that helps to set up and tear >>>>>> down a current GC ID. By moving the GC ID out from the GCTracer >>>>>> class I >>>>>> got rid of a few minor issues as well. One is that we no longer >>>>>> need to >>>>>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>>>>> before since we did logging *after* we had told the corresponding >>>>>> GCTracer that the concurrent cycle was aborted and it had thrown >>>>>> its GC >>>>>> ID away. Now the GC ID can stay in scope until we have completed all >>>>>> logging. >>>>>> >>>>>> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC >>>>>> used >>>>>> to have to create a new GC ID since their logging was done before >>>>>> we had >>>>>> the correct GCTracer set up. Now the GC ID can be available early >>>>>> enough >>>>>> for this logging to be using the same (and correct) GC ID as the >>>>>> rest of >>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>> PrintClassHistogramAfterFullGC. >>>>>> >>>>>> I've added an uint to the Thread class to keep track of the >>>>>> currently >>>>>> active GC ID. In the current code there are actually only >>>>>> NamedThreads >>>>>> that need the GC ID. So, I'm not sure where the best place is for >>>>>> this >>>>>> field is. But it seems like the Thread class contains most of the >>>>>> data, >>>>>> even data that is only used by some subclasses, so I opted for >>>>>> putting >>>>>> the field in Thread as opposed to in NamedThread. This opens up for >>>>>> possible future extensions when Java threads may do part of the >>>>>> GC work. >>>>>> However, I'm open to moving the field down to NamedThread if that is >>>>>> seen as better. >>>>>> >>>>>> In the GCTracer class there were many asserts that were checking >>>>>> that >>>>>> the GC ID was properly set up. Mostly these assert verify that >>>>>> start/end >>>>>> is called correctly and that the other methods are called inside >>>>>> of the >>>>>> start/end scope. Now that the GC ID is moved out of the GCTracer >>>>>> class >>>>>> these asserts had to be dealt with. I figured there were three >>>>>> ways to >>>>>> handle it; just remove them, replace them with check that the GC >>>>>> ID from >>>>>> the current thread is correct, or implement a new status field to >>>>>> keep >>>>>> track of GC start/end state. Personally I'm not sure these >>>>>> asserts are >>>>>> valuable enough to spend time on, so I went for the first approach: >>>>>> removing them. I don't think making them use the thread GC ID >>>>>> will be >>>>>> appropriate. But if anyone feels strongly about these asserts I can >>>>>> implement a separate start/end state. >>>>>> >>>>>> There are a few "Tasks" in the GC code that are executed by worker >>>>>> threads. To make the worker threads use the correct GC ID I make >>>>>> sure >>>>>> that the Task constructors copy the GC ID from the initiating thread >>>>>> into a local variable. When the task is executed in its worker >>>>>> thread it >>>>>> sets up the GC ID based on the local variable. >>>>>> >>>>>> The proposed change does not alter any logging (except for the >>>>>> small bug >>>>>> fix for >>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>> This >>>>>> means that no existing tests need to be updated. In particular the >>>>>> test/gc/logging/TestGCId.java test passes even after these changes. >>>>>> >>>>>> >>>>>> A big thanks to Per, who pre-reviewed these changes and came with >>>>>> some >>>>>> really good feedback. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>> >>> >> > From kim.barrett at oracle.com Mon Sep 28 14:48:04 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 28 Sep 2015 10:48:04 -0400 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <56013DCC.4060307@oracle.com> References: <56013DCC.4060307@oracle.com> Message-ID: This is the last of my comments on: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ A few more minor formatting nits, and a couple of things that should probably be left for later. ------------------------------------------------------------------------------ src/share/vm/oops/klassVtable.cpp 1464 fatal("klass %s: klass object too short (vtable extends beyond " 1465 "end)", _klass->internal_name()); Remove line break in format string. ------------------------------------------------------------------------------ src/share/vm/oops/method.cpp 249 { ResourceMark rm; 250 assert(is_native() && bcp == code_base() || contains(bcp) || is_error_reported(), 251 "bcp doesn't belong to this method: bcp: " INTPTR_FORMAT ", method: %s", bcp, name_and_sig_as_C_string()); 252 } [Pre-existing] The assert isn't properly indented within the nested block. Note that I think the ResourceMark is required, due to name_and_sig_as_C_String(). [Recall that failed assert can still return, though I think only in non-PRODUCT mode.] ------------------------------------------------------------------------------ src/share/vm/oops/method.cpp 282 assert((is_native() && bci == 0) || (!is_native() && 0 <= bci && bci < code_size()), "illegal bci: %d", bci); [Pre-existing] Extra space before "||". ------------------------------------------------------------------------------ src/share/vm/oops/method.cpp 575 if (class_access_flags.is_interface()) { 576 assert(is_nonv == is_static(), "is_nonv=%s", name_and_sig_as_C_string()); 577 } [Pre-existing] Indentation of assert statement should be 2, not 4. ------------------------------------------------------------------------------ src/share/vm/oops/oop.cpp 126 guarantee(obj->is_oop_or_null(), "invalid oop: " INTPTR_FORMAT, p2i((oopDesc*) obj)); [Pre-existing] I don't think the cast is needed here. [Maybe this sort of thing should be left to separate cleanup of p2i usage. See below.] ------------------------------------------------------------------------------ src/share/vm/oops/oop.inline.hpp 192 assert(check_obj_alignment(result), "address not aligned: " INTPTR_FORMAT, p2i((void*) result)); [Pre-existing] I don't think the cast is needed here. [Maybe this sort of thing should be left to a separate cleanup of p2i usage. With two in a row now, I'm inclined to suggest that.] ------------------------------------------------------------------------------ src/share/vm/opto/castnode.cpp 132 fatal("unexpected comparison %s", ss.as_string()); [Pre-existing. Any change related to this should be separate from the assert formatting arguments change.] [Note that this change removed err_msg_res.] There's no obvious associated ResourceMark. But maybe there should be, since ss.as_string() performs resource allocation. Recall that fatal() and other related macros can return under some circumstances. (Though I think only non-PRODUCT.) ------------------------------------------------------------------------------ src/share/vm/opto/graphKit.hpp 356 assert(value->bottom_type()->basic_type() == T_INT, 357 "wrong type: %s", type2name(value->bottom_type()->basic_type())); ... 361 assert(value->bottom_type()->basic_type() == T_LONG, 362 "wrong type: %s", type2name(value->bottom_type()->basic_type())); Indentation of message. ------------------------------------------------------------------------------ From gerard.ziemski at oracle.com Mon Sep 28 14:55:35 2015 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 28 Sep 2015 09:55:35 -0500 Subject: Speeding up the build process of OpenJDK In-Reply-To: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> References: <662A816D-42A9-4DE9-A08A-8F8E37026584@gmail.com> Message-ID: <560954E7.1070300@oracle.com> hi Manas, How do you configure the jdk for a build? In particular are you enabling the multithreading option (ie. --with-jobs=x) to match your Mac's number of cores? In my case (5 year old MacBook Pro) here is what my configure looks like: ./configure --with-jobs=8 --disable-warnings-as-errors --with-jvm-variants=server --enable-deploy=no --enable-installer=no Secondly, I noticed that using SSD instead of regular HDD gives a very nice performance boost. Lastly, just for a ballpark comparison, since it's been a while since I build jdk8, my jdk9 built time for "make images" using the configure option I showed above is about 10 minutes. Subsequent builds of "make images" when touching 1 file from hotspot is about 1 minute. ps. I do have an Xcode project that lets me edit hotspot from Xcode and rebuilt just the hotspot lib that only takes about 20 seconds. I will see if I can make it public, if you're interested. cheers On 09/28/2015 03:59 AM, Manas Thakur wrote: > Hello all, > > I use OpenJDK 8 built on Mac OS X Yosemite. The compilation process after doing even minor changes is very slow (takes about 15-20 minutes; with ccache enabled). Is it because the gcc version I have to use on Yosemite is older (gcc 4.2 supplied with Xcode 4.6.3)? If yes, any ideas on speeding up the incremental builds? > > Regards, > Manas > From per.liden at oracle.com Mon Sep 28 14:57:44 2015 From: per.liden at oracle.com (Per Liden) Date: Mon, 28 Sep 2015 16:57:44 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <56093EC1.9090109@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> Message-ID: <56095568.2010805@oracle.com> Hi Bengt, On 2015-09-28 15:21, Bengt Rutisson wrote: > > Hi Jon and Per, > > I've been trying to get the version proposed in webrev.02 to work but I > ran into a show stopper for this approach. The G1 concurrent mark thread > is at times executing (and logging) at the same time as a young or mixed > GC is executing. There is no good way of handling this with a common > place to store the GC ID. Instead I've decided to go back to storing the > current GC ID in the thread. That way the G1 concurrent marking and the > young/mixed GCs will have their own GC IDs. > > Here's an updated webrev: > http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ > > This is the same as webrev.01 that both of you already looked at. I've > made two minor adjustments: > > - I removed the currentNamedthread() function in gcId.cpp that Per > pointed out in his review of webrev.01. Instead I use Thread::current() > directly. > > - I made GCIdMark a StackObj. > > Otherwise the patch is the same as in webrev.01. > Looks good. /Per > Thanks, > Bengt > > On 2015-09-10 14:37, Bengt Rutisson wrote: >> >> Hi Jon, >> >> Thanks for looking at this! >> >> On 2015-09-10 01:00, Jon Masamitsu wrote: >>> Bengt, >>> >>> When a CMS concurrent collection hands off to a STW foreground >>> collection, >>> (done in acquire_control_and_collect()), I expected a new GCId would be >>> used. Did I miss it? That STW collection does not go through >>> do_collection(). >> >> The call to acquire_control_and_collect() originates from >> GenCollectedHeap::collect_generation() and there is a new GCIdMark in >> there that will create a new GCId for the STW collection. That's the >> "extra" GCIdMark that Per asked about in his review. >> >> *But* I wanted to verify that it worked properly since you asked about >> it and I realized that there is another bug. >> >> The GCId that is set up for the concurrent cycle, the one set up in >> ConcurrentMarkSweepThread::run(), will is still active over the >> complete STW foreground collection. That's fine in my model since the >> new GCIdMark in GenCollectedHeap::collect_generation() will cache that >> GCId. But the ConcurrentMarkSweep thread is not completely idle even >> though control has been left to the foreground collection in >> acquire_control_and_collect(). So, there is some logging done (by fore >> example CMSPhaseAccounting) that is done at the same time as the >> foreground collection. This logging will now use the foreground GCId >> instead of the CMS GCId. >> >> I haven't had time to dig in to the details of that just yet. But this >> is an unintended change of the logging, so I would like to fix it. >> Hopefully I'll have an updated webrev tomorrow. >> >> Thanks, >> Bengt >> >> >>> >>> Jon >>> >>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>> >>>> Hi again, >>>> >>>> I found an issue with how G1 young collections kick off concurrent >>>> marks. There is a short window where we might have two active GC IDs >>>> at the same time. I've fixed this and updated the webrevs. In case >>>> anyone had already started looking at the webrevs you need to do a >>>> refresh now. The fix is in >>>> G1CollectedHeap::do_collection_pause_at_safepoint(). >>>> >>>> Thanks, >>>> Bengt >>>> >>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>> >>>>> Hi Per, >>>>> >>>>> Thanks for looking at this! >>>>> >>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi everyone, >>>>>>> >>>>>>> This is mostly a GC related patch, but it adds a field to the Thread >>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>> >>>>>>> Could I have a couple of reviews for this patch? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>> >>>>>> >>>>>> Looks good. I think this is a nice simplification, especially for >>>>>> G1's concurrent parts. Just two comments: >>>>>> >>>>>> genCollectedHeap.cpp: >>>>>> --------------------- >>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>> different scopes. Do we really want that second mark? >>>>> >>>>> >>>>> We potentially do two GCs in GenCollectedHeap::do_collection(). >>>>> First a young GC and then potentially a full GC. These two should >>>>> have different GC IDs. So, yes, we need two GCIdMarks in this >>>>> method. The scoping could be better, but I think that is a >>>>> refactoring that should be done separately from my current patch. >>>>> I'll talk to Jesper about it since he has been cleaning up this >>>>> code lately. >>>>> >>>>>> >>>>>> >>>>>> gcId.cpp: >>>>>> -------- >>>>>> - I think you might have left currentNamedthread() in there. You >>>>>> probably just want to use Thread::current() instead. >>>>> >>>>> Yes, good catch. Thanks. >>>>> >>>>> However, after thinking some more about the implications of moving >>>>> the GC ID out from the GCTracer I figured it would be possible to >>>>> just store the previous GC ID in the GCIdMark and then restore it >>>>> in the destructor of that class. That way we don't have to store >>>>> anything in the Thread class but can still have both a concurrent >>>>> GC ID and a STW GC ID active at the same time. This also removes >>>>> the need to copy the GC ID to the worker tasks. >>>>> >>>>> Here's an updated webrev with a solution that does not add anything >>>>> to the Thread class: >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>> >>>>> And here's a diff compared to the last version: >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>> >>>>> Unfotunately the webrev tool had some bad luck when creating the >>>>> diff webrev so at least the g1CollectedHeap.cpp seem to contain the >>>>> complete changes rather than just the diff compared to the 01 >>>>> version. The rest of the diff looks correct as far as I can tell. >>>>> >>>>> Thanks, >>>>> Bengt >>>>>> >>>>>> >>>>>> cheers, >>>>>> /Per >>>>>> >>>>>> >>>>>>> Currently the GC ID, that is used for event tracing and logging, is >>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>> problems since >>>>>>> there are multiple GCTracers active at the same time. The correct >>>>>>> GC IDs >>>>>>> need to be passed around in many places. >>>>>>> >>>>>>> For some upcoming GC logging changes I would like to have a more >>>>>>> consistent way of finding the currently active GC ID. I've played >>>>>>> around >>>>>>> with a couple of different solutions, but eventually I found that >>>>>>> the >>>>>>> simplest solution is to store the current GC ID in the thread. >>>>>>> That way >>>>>>> there is a single way to look it up in all places. It is also fairly >>>>>>> straight forward to set it up. >>>>>>> >>>>>>> I've reworked the GCId class a bit to support this and I've >>>>>>> introduced a >>>>>>> GCIdMark class that is a scoped object that helps to set up and tear >>>>>>> down a current GC ID. By moving the GC ID out from the GCTracer >>>>>>> class I >>>>>>> got rid of a few minor issues as well. One is that we no longer >>>>>>> need to >>>>>>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>>>>>> before since we did logging *after* we had told the corresponding >>>>>>> GCTracer that the concurrent cycle was aborted and it had thrown >>>>>>> its GC >>>>>>> ID away. Now the GC ID can stay in scope until we have completed all >>>>>>> logging. >>>>>>> >>>>>>> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC >>>>>>> used >>>>>>> to have to create a new GC ID since their logging was done before >>>>>>> we had >>>>>>> the correct GCTracer set up. Now the GC ID can be available early >>>>>>> enough >>>>>>> for this logging to be using the same (and correct) GC ID as the >>>>>>> rest of >>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>> PrintClassHistogramAfterFullGC. >>>>>>> >>>>>>> I've added an uint to the Thread class to keep track of the >>>>>>> currently >>>>>>> active GC ID. In the current code there are actually only >>>>>>> NamedThreads >>>>>>> that need the GC ID. So, I'm not sure where the best place is for >>>>>>> this >>>>>>> field is. But it seems like the Thread class contains most of the >>>>>>> data, >>>>>>> even data that is only used by some subclasses, so I opted for >>>>>>> putting >>>>>>> the field in Thread as opposed to in NamedThread. This opens up for >>>>>>> possible future extensions when Java threads may do part of the >>>>>>> GC work. >>>>>>> However, I'm open to moving the field down to NamedThread if that is >>>>>>> seen as better. >>>>>>> >>>>>>> In the GCTracer class there were many asserts that were checking >>>>>>> that >>>>>>> the GC ID was properly set up. Mostly these assert verify that >>>>>>> start/end >>>>>>> is called correctly and that the other methods are called inside >>>>>>> of the >>>>>>> start/end scope. Now that the GC ID is moved out of the GCTracer >>>>>>> class >>>>>>> these asserts had to be dealt with. I figured there were three >>>>>>> ways to >>>>>>> handle it; just remove them, replace them with check that the GC >>>>>>> ID from >>>>>>> the current thread is correct, or implement a new status field to >>>>>>> keep >>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>> asserts are >>>>>>> valuable enough to spend time on, so I went for the first approach: >>>>>>> removing them. I don't think making them use the thread GC ID >>>>>>> will be >>>>>>> appropriate. But if anyone feels strongly about these asserts I can >>>>>>> implement a separate start/end state. >>>>>>> >>>>>>> There are a few "Tasks" in the GC code that are executed by worker >>>>>>> threads. To make the worker threads use the correct GC ID I make >>>>>>> sure >>>>>>> that the Task constructors copy the GC ID from the initiating thread >>>>>>> into a local variable. When the task is executed in its worker >>>>>>> thread it >>>>>>> sets up the GC ID based on the local variable. >>>>>>> >>>>>>> The proposed change does not alter any logging (except for the >>>>>>> small bug >>>>>>> fix for >>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>> This >>>>>>> means that no existing tests need to be updated. In particular the >>>>>>> test/gc/logging/TestGCId.java test passes even after these changes. >>>>>>> >>>>>>> >>>>>>> A big thanks to Per, who pre-reviewed these changes and came with >>>>>>> some >>>>>>> really good feedback. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>> >>>> >>> >> > From david.lindholm at oracle.com Mon Sep 28 15:37:15 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Mon, 28 Sep 2015 17:37:15 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: References: <56013DCC.4060307@oracle.com> Message-ID: <56095EAB.5030805@oracle.com> Kim and Per, Thank you! I have made fixes for all your comments, the latest webrev is available here: http://cr.openjdk.java.net/~david/JDK-8080775/webrev.03/ Kim, I have left the casts in the code, as you say this should be fixed in another change. Thanks, David On 2015-09-28 16:48, Kim Barrett wrote: > This is the last of my comments on: > http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ > > A few more minor formatting nits, and a couple of things that should > probably be left for later. > > ------------------------------------------------------------------------------ > src/share/vm/oops/klassVtable.cpp > 1464 fatal("klass %s: klass object too short (vtable extends beyond " > 1465 "end)", _klass->internal_name()); > > Remove line break in format string. > > ------------------------------------------------------------------------------ > src/share/vm/oops/method.cpp > 249 { ResourceMark rm; > 250 assert(is_native() && bcp == code_base() || contains(bcp) || is_error_reported(), > 251 "bcp doesn't belong to this method: bcp: " INTPTR_FORMAT ", method: %s", bcp, name_and_sig_as_C_string()); > 252 } > > [Pre-existing] > The assert isn't properly indented within the nested block. Note that > I think the ResourceMark is required, due to > name_and_sig_as_C_String(). [Recall that failed assert can still > return, though I think only in non-PRODUCT mode.] > > ------------------------------------------------------------------------------ > src/share/vm/oops/method.cpp > 282 assert((is_native() && bci == 0) || (!is_native() && 0 <= bci && bci < code_size()), "illegal bci: %d", bci); > > [Pre-existing] > Extra space before "||". > > ------------------------------------------------------------------------------ > src/share/vm/oops/method.cpp > 575 if (class_access_flags.is_interface()) { > 576 assert(is_nonv == is_static(), "is_nonv=%s", name_and_sig_as_C_string()); > 577 } > > [Pre-existing] > Indentation of assert statement should be 2, not 4. > > ------------------------------------------------------------------------------ > src/share/vm/oops/oop.cpp > 126 guarantee(obj->is_oop_or_null(), "invalid oop: " INTPTR_FORMAT, p2i((oopDesc*) obj)); > > [Pre-existing] > I don't think the cast is needed here. > > [Maybe this sort of thing should be left to separate cleanup of p2i > usage. See below.] > > ------------------------------------------------------------------------------ > src/share/vm/oops/oop.inline.hpp > 192 assert(check_obj_alignment(result), "address not aligned: " INTPTR_FORMAT, p2i((void*) result)); > > [Pre-existing] > I don't think the cast is needed here. > > [Maybe this sort of thing should be left to a separate cleanup of p2i > usage. With two in a row now, I'm inclined to suggest that.] > > ------------------------------------------------------------------------------ > src/share/vm/opto/castnode.cpp > 132 fatal("unexpected comparison %s", ss.as_string()); > > [Pre-existing. Any change related to this should be separate from the > assert formatting arguments change.] > > [Note that this change removed err_msg_res.] > > There's no obvious associated ResourceMark. But maybe there should > be, since ss.as_string() performs resource allocation. > > Recall that fatal() and other related macros can return under some > circumstances. (Though I think only non-PRODUCT.) > > ------------------------------------------------------------------------------ > src/share/vm/opto/graphKit.hpp > 356 assert(value->bottom_type()->basic_type() == T_INT, > 357 "wrong type: %s", type2name(value->bottom_type()->basic_type())); > ... > 361 assert(value->bottom_type()->basic_type() == T_LONG, > 362 "wrong type: %s", type2name(value->bottom_type()->basic_type())); > > Indentation of message. > > ------------------------------------------------------------------------------ > From kim.barrett at oracle.com Mon Sep 28 17:11:23 2015 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 28 Sep 2015 13:11:23 -0400 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <56095EAB.5030805@oracle.com> References: <56013DCC.4060307@oracle.com> <56095EAB.5030805@oracle.com> Message-ID: <297F25EB-03B6-48E2-9BEF-E9AC3E1C6515@oracle.com> On Sep 28, 2015, at 11:37 AM, David Lindholm wrote: > > Kim and Per, > > Thank you! I have made fixes for all your comments, the latest webrev is available here: > > http://cr.openjdk.java.net/~david/JDK-8080775/webrev.03/ > > Kim, I have left the casts in the code, as you say this should be fixed in another change. ------------------------------------------------------------------------------ > src/share/vm/gc/g1/g1BlockOffsetTable.cpp > 364 assert((_array->offset_array(orig_index) == 0 && > 365 blk_start == boundary) || > 366 (_array->offset_array(orig_index) > 0 && > 367 _array->offset_array(orig_index) <= N_words), > > Indentation of 366 and 367 are wrong. webrev.03 has 364 assert((_array->offset_array(orig_index) == 0 && 365 blk_start == boundary) || 366 (_array->offset_array(orig_index) > 0 && 367 _array->offset_array(orig_index) <= N_words), Lines 365 and 367 should be indented one more space. ------------------------------------------------------------------------------ Otherwise, looks good. I don't need a new webrev for the above. From jon.masamitsu at oracle.com Mon Sep 28 17:19:06 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Mon, 28 Sep 2015 10:19:06 -0700 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <56093EC1.9090109@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> Message-ID: <5609768A.5030209@oracle.com> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: > > Hi Jon and Per, > > I've been trying to get the version proposed in webrev.02 to work but > I ran into a show stopper for this approach. The G1 concurrent mark > thread is at times executing (and logging) at the same time as a young > or mixed GC is executing. There is no good way of handling this with a > common place to store the GC ID. Instead I've decided to go back to > storing the current GC ID in the thread. That way the G1 concurrent > marking and the young/mixed GCs will have their own GC IDs. > > Here's an updated webrev: > http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ Bengt, The _gc_id is needed in a JavaThread because a JavaThread might do work for G1 (concurrent refinement for example) and needs a GC ID? But a WatcherThread would never need one, right? Jon > > This is the same as webrev.01 that both of you already looked at. I've > made two minor adjustments: > > - I removed the currentNamedthread() function in gcId.cpp that Per > pointed out in his review of webrev.01. Instead I use > Thread::current() directly. > > - I made GCIdMark a StackObj. > > Otherwise the patch is the same as in webrev.01. > > Thanks, > Bengt > > On 2015-09-10 14:37, Bengt Rutisson wrote: >> >> Hi Jon, >> >> Thanks for looking at this! >> >> On 2015-09-10 01:00, Jon Masamitsu wrote: >>> Bengt, >>> >>> When a CMS concurrent collection hands off to a STW foreground >>> collection, >>> (done in acquire_control_and_collect()), I expected a new GCId would be >>> used. Did I miss it? That STW collection does not go through >>> do_collection(). >> >> The call to acquire_control_and_collect() originates from >> GenCollectedHeap::collect_generation() and there is a new GCIdMark in >> there that will create a new GCId for the STW collection. That's the >> "extra" GCIdMark that Per asked about in his review. >> >> *But* I wanted to verify that it worked properly since you asked >> about it and I realized that there is another bug. >> >> The GCId that is set up for the concurrent cycle, the one set up in >> ConcurrentMarkSweepThread::run(), will is still active over the >> complete STW foreground collection. That's fine in my model since the >> new GCIdMark in GenCollectedHeap::collect_generation() will cache >> that GCId. But the ConcurrentMarkSweep thread is not completely idle >> even though control has been left to the foreground collection in >> acquire_control_and_collect(). So, there is some logging done (by >> fore example CMSPhaseAccounting) that is done at the same time as the >> foreground collection. This logging will now use the foreground GCId >> instead of the CMS GCId. >> >> I haven't had time to dig in to the details of that just yet. But >> this is an unintended change of the logging, so I would like to fix >> it. Hopefully I'll have an updated webrev tomorrow. >> >> Thanks, >> Bengt >> >> >>> >>> Jon >>> >>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>> >>>> Hi again, >>>> >>>> I found an issue with how G1 young collections kick off concurrent >>>> marks. There is a short window where we might have two active GC >>>> IDs at the same time. I've fixed this and updated the webrevs. In >>>> case anyone had already started looking at the webrevs you need to >>>> do a refresh now. The fix is in >>>> G1CollectedHeap::do_collection_pause_at_safepoint(). >>>> >>>> Thanks, >>>> Bengt >>>> >>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>> >>>>> Hi Per, >>>>> >>>>> Thanks for looking at this! >>>>> >>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>> Hi Bengt, >>>>>> >>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi everyone, >>>>>>> >>>>>>> This is mostly a GC related patch, but it adds a field to the >>>>>>> Thread >>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>> >>>>>>> Could I have a couple of reviews for this patch? >>>>>>> >>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>> >>>>>> >>>>>> Looks good. I think this is a nice simplification, especially for >>>>>> G1's concurrent parts. Just two comments: >>>>>> >>>>>> genCollectedHeap.cpp: >>>>>> --------------------- >>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>> different scopes. Do we really want that second mark? >>>>> >>>>> >>>>> We potentially do two GCs in GenCollectedHeap::do_collection(). >>>>> First a young GC and then potentially a full GC. These two should >>>>> have different GC IDs. So, yes, we need two GCIdMarks in this >>>>> method. The scoping could be better, but I think that is a >>>>> refactoring that should be done separately from my current patch. >>>>> I'll talk to Jesper about it since he has been cleaning up this >>>>> code lately. >>>>> >>>>>> >>>>>> >>>>>> gcId.cpp: >>>>>> -------- >>>>>> - I think you might have left currentNamedthread() in there. You >>>>>> probably just want to use Thread::current() instead. >>>>> >>>>> Yes, good catch. Thanks. >>>>> >>>>> However, after thinking some more about the implications of moving >>>>> the GC ID out from the GCTracer I figured it would be possible to >>>>> just store the previous GC ID in the GCIdMark and then restore it >>>>> in the destructor of that class. That way we don't have to store >>>>> anything in the Thread class but can still have both a concurrent >>>>> GC ID and a STW GC ID active at the same time. This also removes >>>>> the need to copy the GC ID to the worker tasks. >>>>> >>>>> Here's an updated webrev with a solution that does not add >>>>> anything to the Thread class: >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>> >>>>> And here's a diff compared to the last version: >>>>> >>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>> >>>>> Unfotunately the webrev tool had some bad luck when creating the >>>>> diff webrev so at least the g1CollectedHeap.cpp seem to contain >>>>> the complete changes rather than just the diff compared to the 01 >>>>> version. The rest of the diff looks correct as far as I can tell. >>>>> >>>>> Thanks, >>>>> Bengt >>>>>> >>>>>> >>>>>> cheers, >>>>>> /Per >>>>>> >>>>>> >>>>>>> Currently the GC ID, that is used for event tracing and logging, is >>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>> problems since >>>>>>> there are multiple GCTracers active at the same time. The >>>>>>> correct GC IDs >>>>>>> need to be passed around in many places. >>>>>>> >>>>>>> For some upcoming GC logging changes I would like to have a more >>>>>>> consistent way of finding the currently active GC ID. I've >>>>>>> played around >>>>>>> with a couple of different solutions, but eventually I found >>>>>>> that the >>>>>>> simplest solution is to store the current GC ID in the thread. >>>>>>> That way >>>>>>> there is a single way to look it up in all places. It is also >>>>>>> fairly >>>>>>> straight forward to set it up. >>>>>>> >>>>>>> I've reworked the GCId class a bit to support this and I've >>>>>>> introduced a >>>>>>> GCIdMark class that is a scoped object that helps to set up and >>>>>>> tear >>>>>>> down a current GC ID. By moving the GC ID out from the GCTracer >>>>>>> class I >>>>>>> got rid of a few minor issues as well. One is that we no longer >>>>>>> need to >>>>>>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>>>>>> before since we did logging *after* we had told the corresponding >>>>>>> GCTracer that the concurrent cycle was aborted and it had thrown >>>>>>> its GC >>>>>>> ID away. Now the GC ID can stay in scope until we have completed >>>>>>> all >>>>>>> logging. >>>>>>> >>>>>>> Also the HeapDumpBeforeFullGC and >>>>>>> PrintClassHistogramBeforeFullGC used >>>>>>> to have to create a new GC ID since their logging was done >>>>>>> before we had >>>>>>> the correct GCTracer set up. Now the GC ID can be available >>>>>>> early enough >>>>>>> for this logging to be using the same (and correct) GC ID as the >>>>>>> rest of >>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>> PrintClassHistogramAfterFullGC. >>>>>>> >>>>>>> I've added an uint to the Thread class to keep track of the >>>>>>> currently >>>>>>> active GC ID. In the current code there are actually only >>>>>>> NamedThreads >>>>>>> that need the GC ID. So, I'm not sure where the best place is >>>>>>> for this >>>>>>> field is. But it seems like the Thread class contains most of >>>>>>> the data, >>>>>>> even data that is only used by some subclasses, so I opted for >>>>>>> putting >>>>>>> the field in Thread as opposed to in NamedThread. This opens up for >>>>>>> possible future extensions when Java threads may do part of the >>>>>>> GC work. >>>>>>> However, I'm open to moving the field down to NamedThread if >>>>>>> that is >>>>>>> seen as better. >>>>>>> >>>>>>> In the GCTracer class there were many asserts that were checking >>>>>>> that >>>>>>> the GC ID was properly set up. Mostly these assert verify that >>>>>>> start/end >>>>>>> is called correctly and that the other methods are called inside >>>>>>> of the >>>>>>> start/end scope. Now that the GC ID is moved out of the GCTracer >>>>>>> class >>>>>>> these asserts had to be dealt with. I figured there were three >>>>>>> ways to >>>>>>> handle it; just remove them, replace them with check that the GC >>>>>>> ID from >>>>>>> the current thread is correct, or implement a new status field >>>>>>> to keep >>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>> asserts are >>>>>>> valuable enough to spend time on, so I went for the first approach: >>>>>>> removing them. I don't think making them use the thread GC ID >>>>>>> will be >>>>>>> appropriate. But if anyone feels strongly about these asserts I can >>>>>>> implement a separate start/end state. >>>>>>> >>>>>>> There are a few "Tasks" in the GC code that are executed by worker >>>>>>> threads. To make the worker threads use the correct GC ID I make >>>>>>> sure >>>>>>> that the Task constructors copy the GC ID from the initiating >>>>>>> thread >>>>>>> into a local variable. When the task is executed in its worker >>>>>>> thread it >>>>>>> sets up the GC ID based on the local variable. >>>>>>> >>>>>>> The proposed change does not alter any logging (except for the >>>>>>> small bug >>>>>>> fix for >>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>> This >>>>>>> means that no existing tests need to be updated. In particular the >>>>>>> test/gc/logging/TestGCId.java test passes even after these changes. >>>>>>> >>>>>>> >>>>>>> A big thanks to Per, who pre-reviewed these changes and came >>>>>>> with some >>>>>>> really good feedback. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>> >>>> >>> >> > From ysr1729 at gmail.com Mon Sep 28 19:39:43 2015 From: ysr1729 at gmail.com (Srinivas Ramakrishna) Date: Mon, 28 Sep 2015 12:39:43 -0700 Subject: Version of jtreg to use for testing of specific JDK 8u-dev releases? Message-ID: Hello all -- What's the version iof jtreg to use for JDK 8u testing? I am seeing two failures in HotSpot regression tests that appear to be because of using a somewhat older version of jtreg and wanted to check if there's any way to tell what the correct vintage of jtreg recommended for testing specific jdk8u releases? Would it be safe to always get the latest jtreg for testing at any time, or are there tagged builds of jtreeg suitable for specific versions of jdk8u-dev testing? thanks for any pointers! -- ramki From jonathan.gibbons at oracle.com Mon Sep 28 20:39:23 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 28 Sep 2015 13:39:23 -0700 Subject: Version of jtreg to use for testing of specific JDK 8u-dev releases? In-Reply-To: References: Message-ID: <5609A57B.7050406@oracle.com> Hi, Generally, the recommended version of jtreg for most releases is a build from the latest sources with a tag of the form jtreg4.1-bNN. You can see the tags here: http://hg.openjdk.java.net/code-tools/jtreg/tags You can generally get a copy of a build of the latest version by following the links on the OpenJDK jtreg page, or by going to the Adopt OpenJDK group's pages. http://openjdk.java.net/jtreg/ https://adopt-openjdk.ci.cloudbees.com/job/jtreg/lastSuccessfulBuild/artifact/ (Oracle folk can also get builds from Oracle's internal servers.) The primary exception to the rule to use the latest tagged version is with regard to running tests in the OpenJDK jigsaw/jake forest. In this case, the recommendation is to use a build from the tip of the repository. Since you're asking about jdk8u, this does not apply to you. Hope that helps, -- Jon On 09/28/2015 12:39 PM, Srinivas Ramakrishna wrote: > Hello all -- > > What's the version iof jtreg to use for JDK 8u testing? > I am seeing two failures in HotSpot regression tests that appear to be > because of using a somewhat older version of jtreg and wanted to > check if there's any way to tell what the correct vintage of jtreg > recommended > for testing specific jdk8u releases? > > Would it be safe to always get the latest jtreg for testing at any time, or > are there > tagged builds of jtreeg suitable for specific versions of jdk8u-dev testing? > > thanks for any pointers! > -- ramki From gerard.ziemski at oracle.com Mon Sep 28 21:55:52 2015 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 28 Sep 2015 16:55:52 -0500 Subject: RFR (S): JDK-8129855: -XX:+IgnoreUnrecognizedVMOptions hides out of range VM options. Message-ID: <5609B768.8010109@oracle.com> hi all, We are fixing how IgnoreUnrecognizedVMOptions treats those flags whose values are out of range. Before the fix, the VM would continue even if flag?s value was out of range, ex: java_old -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version size_t MinTLABSize=0 is outside the allowed range [ 1 ... 4294967295 ] java version "1.9.0-internal-fastdebug" Java(TM) SE Runtime Environment (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00) Java HotSpot(TM) Server VM (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00, mixed mode) now, we correctly exit the VM with an error, ex: java_new -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version size_t MinTLABSize=0 is outside the allowed range [ 1 ... 18446744073709551615 ] Improperly specified VM option 'MinTLABSize=0' Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. In addition IgnoreUnrecognizedVMOptions nows strictly follows the spec as outlined in JDK-8129855. The behavior changes depending on whether the build is product or debug. We also introduce a new test that verifies all known use cases that we care about. References: bugid: https://bugs.openjdk.java.net/browse/JDK-8129855 webrev: http://cr.openjdk.java.net/~gziemski/8129855_rev0 discussion: http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-June/019213.html Passes "JPRT hotspot" and "RBT testlist,noncolo.testlist quick" cheers From gerard.ziemski at oracle.com Mon Sep 28 22:00:11 2015 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 28 Sep 2015 17:00:11 -0500 Subject: (Closed) RFR (XXS): JDK-8129855: -XX:+IgnoreUnrecognizedVMOptions hides out of range VM options. Message-ID: <5609B86B.9080705@oracle.com> hi all, Please review this closed world change that goes with the fix for JDK-8129855 We must trivially fix basicvm_test, now that IgnoreUnrecognizedVMOption implements a strict spec as defined in JDK-8129855 References: bugid: https://bugs.openjdk.java.net/browse/JDK-8129855 webrev: http://javaweb.us.oracle.com/~gziemski/8129855_closed_rev0/ cheers From christian.thalinger at oracle.com Tue Sep 29 01:12:07 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 28 Sep 2015 15:12:07 -1000 Subject: RFR (XXL): JEP 243: Java-Level JVM Compiler Interface In-Reply-To: <56090777.2090807@oracle.com> References: <2B401EAC-B386-4E42-BC4C-DD267FFD4970@oracle.com> <55F9671C.4000208@oracle.com> <99042B61-9F43-42CE-835C-095CD2858CB1@oracle.com> <55F9F08E.8010506@oracle.com> <08A9DB82-58D0-41F8-97F5-23AD78205263@oracle.com> <067C7DA1-7422-411E-A678-91FBDECEBE1D@oracle.com> <8E92D69B-7207-4CDC-80F7-3AA65AC5C482@oracle.com> <97150580-7329-438F-9B64-F47E3BFE9F08@oracle.com> <56051FD0.4010303@oracle.com> <1F931EBE-F2C0-413D-BFE4-787362577384@oracle.com> <56090777.2090807@oracle.com> Message-ID: <2C323894-C058-442D-92CF-BDF969CD6504@oracle.com> > On Sep 27, 2015, at 11:25 PM, Magnus Ihse Bursie wrote: > > On 2015-09-25 22:00, Christian Thalinger wrote: >> >>> On Sep 25, 2015, at 12:20 AM, Magnus Ihse Bursie > wrote: >>> >>> On 2015-09-24 20:57, Christian Thalinger wrote: >>>>> On Sep 21, 2015, at 12:24 PM, Christian Thalinger > wrote: >>>>> >>>>> >>>>>> On Sep 18, 2015, at 2:19 PM, Christian Thalinger > wrote: >>>>>> >>>>>> >>>>>>> On Sep 18, 2015, at 7:00 AM, Christian Thalinger > wrote: >>>>>>> >>>>>>> >>>>>>>> On Sep 17, 2015, at 11:00 PM, Volker Simonis > wrote: >>>>>>>> >>>>>>>> To which repository will you finally push these changes? >>>>>>>> >>>>>>>> The current webrevs are against jdk9/jdk9/hotspot but I doubt that >>>>>>>> they will be integrated there. >>>>>>> Good point. >>>>>>> >>>>>>>> Unfortunately the patches don't apply to jdk9/dev/hotspot anymore >>>>>>>> because of '8135067: Preparatory refactorings for compiler control' >>>>>>>> and "8134626: Misc cleanups after generation array removal": >>>>>>>> >>>>>>>> 3 out of 18 hunks FAILED -- saving rejects to file >>>>>>>> src/share/vm/compiler/compileBroker.cpp.rej >>>>>>>> 1 out of 5 hunks FAILED -- saving rejects to file >>>>>>>> src/share/vm/compiler/compileBroker.hpp.rej >>>>>>>> 6 out of 40 hunks FAILED -- saving rejects to file >>>>>>>> src/share/vm/runtime/vmStructs.cpp.rej >>>>>>>> >>>>>>>> Will you update the patches and if yes against which repositories? >>>>>>> Yes. Let me update the graal-jvmci-9 repository to hs-comp. >>>>>> Done. Now I have to create new webrevs? >>>>> Here are new webrevs against hs-comp: >>>>> >>>>> http://cr.openjdk.java.net/~twisti/8136421/webrev/ > >>>>> http://cr.openjdk.java.net/~twisti/8136421/hotspot/webrev/ > >>>> I have refreshed these webrevs. They contain all the changes we discussed here plus a bunch of new tests that SQE finished. >>> >>> Hi Christian, >>> >>> I didn't see Volkers proposed fix about undefined-bool-conversion: >>> >>>> If this comes only from adlc, you could fix for adlc only by adding >>>> >>>> WARNINGS_ARE_ERRORS += -Wno-undefined-bool-conversion >>>> >>>> to make/linux/makefiles/adlc.make instead of make/linux/makefiles/gcc.make >>> >>> While this is really a fix for an unrelated issue (support for a newer version of clang), I do believe Volker's suggestion to only add it to adlc is much better, if you really need to take this in as part of the JVMCI integration. >> >> Sorry I dropped this. After Gilles? hint I remembered the code in question was removed with: >> >> [JDK-8041620] Solaris Studio 12.4 C++ 5.13 change in behavior for placing friend declarations within surrounding scope - Java Bug System >> >> and the remaining ones were fixed with: >> >> [JDK-8077364] "if( !this )" construct prevents build on Xcode 6.3 - Java Bug System >> >> I?ve removed the change: >> >> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6de55f563ee1 > Good! > >> >>> >>> Apart from this, the makefile changes (still) look good. >> >> Btw. we found a bug in creating the OptionDescriptors files; we get duplicate entries: >> >> $ cat build/macosx-x86_64-normal-server-release/jdk/modules/java.base/META-INF/services/jdk.internal.jvmci.options.OptionDescriptors >> jdk.internal.jvmci.compiler.Compiler_OptionDescriptors >> jdk.internal.jvmci.hotspot.HotSpotConstantReflectionProvider_OptionDescriptors >> jdk.internal.jvmci.hotspot.HotSpotResolvedJavaFieldImpl_OptionDescriptors >> jdk.internal.jvmci.hotspot.HotSpotResolvedJavaMethodImpl_OptionDescriptors >> jdk.internal.jvmci.compiler.Compiler_OptionDescriptors >> jdk.internal.jvmci.hotspot.HotSpotConstantReflectionProvider_OptionDescriptors >> jdk.internal.jvmci.hotspot.HotSpotResolvedJavaFieldImpl_OptionDescriptors >> jdk.internal.jvmci.hotspot.HotSpotResolvedJavaMethodImpl_OptionDescriptors >> ? >> >> Would this be the right fix? >> >> diff -r db1a815d2f6c make/gensrc/Gensrc-java.base.gmk >> --- a/make/gensrc/Gensrc-java.base.gmkThu Sep 24 15:35:49 2015 -1000 >> +++ b/make/gensrc/Gensrc-java.base.gmkFri Sep 25 18:18:35 2015 +0200 >> @@ -94,6 +94,7 @@ >> $(GENSRC_DIR)/_gensrc_proc_done >> $(MKDIR) -p $(@D) >> ($(CD) $(GENSRC_DIR)/META-INF/jvmci.options && \ >> + $(RM) -f $@; \ >> for i in $$(ls); do \ >> echo $${i}_OptionDescriptors >> $@; \ >> done) >> > That seems like a reasonable fix, yes. Thanks, but? (see below) > > /Magnus > >> And I see the same behavior for HotSpotJVMCIBackendFactory: >> >> $ cat build/macosx-x86_64-normal-server-release/jdk/modules/java.base/META-INF/services/jdk.internal.jvmci.hotspot.HotSpotJVMCIBackendFactory >> jdk.internal.jvmci.hotspot.amd64.AMD64HotSpotJVMCIBackendFactory >> jdk.internal.jvmci.hotspot.sparc.SPARCHotSpotJVMCIBackendFactory >> jdk.internal.jvmci.hotspot.amd64.AMD64HotSpotJVMCIBackendFactory >> jdk.internal.jvmci.hotspot.sparc.SPARCHotSpotJVMCIBackendFactory >> ? >> >> So I think a similar fix needs to be applied there. ?I?ve look at the code that creates this file and it isn?t obvious to me how to fix it. Any good ideas? >> >>> >>> /Magnus >>> >>>> >>>>>>>> If I want to work on the ppc64 implementation, which repository should I use? >>>>>>> graal-jvmci-9. Are you working on this for fun or do you want to have that integrated with this JEP? >>>>>>> >>>>>>>> Thank you and best regards, >>>>>>>> Volker >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Sep 18, 2015 at 10:20 AM, Volker Simonis >>>>>>>> > wrote: >>>>>>>>> For the top-level change I always get a strange error when importing it: >>>>>>>>> >>>>>>>>> patch failed, unable to continue (try -v) >>>>>>>>> patch failed, rejects left in working dir >>>>>>>>> errors during apply, please fix and refresh 8062493_jvmci_top_0911.v1.patch >>>>>>>>> >>>>>>>>> It is because of the strange path syntax of the last hunk in the patch file: >>>>>>>>> >>>>>>>>> --- old/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>>>>> +++ new/./modules.xml 2015-09-16 15:11:14.000000000 -0700 >>>>>>>>> @@ -254,6 +254,14 @@ >>>>>>>>> jdk.jfr >>>>>>>>> >>>>>>>>> >>>>>>>>> + jdk.internal.jvmci.hotspot >>>>>>>>> + jdk.jfr >>>>>>>>> + >>>>>>>>> + >>>>>>>>> + jdk.internal.jvmci.hotspot.events >>>>>>>>> + jdk.jfr >>>>>>>>> + >>>>>>>>> + >>>>>>>>> sun.misc >>>>>>>>> java.corba >>>>>>>>> java.desktop >>>>>>>>> >>>>>>>>> >>>>>>>>> Notice the strange syntax "old/./modules.xml" and "new/./modules.xml". >>>>>>>>> If I remove the redundant './' from the path, everything works fine. >>>>>>>>> For some reason only the diffs for modules.xml has this strange path >>>>>>>>> syntax in the patch. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Volker >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Sep 17, 2015 at 10:32 PM, Christian Thalinger >>>>>>>>> > wrote: >>>>>>>>>> Since there are no objections I?m going to push this? >>>>>>>>>> >>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/6a6766a8cbbf >>>>>>>>>> >>>>>>>>>>> On Sep 16, 2015, at 3:05 PM, Christian Thalinger wrote: >>>>>>>>>>> >>>>>>>>>>> I would like to add this change: >>>>>>>>>>> >>>>>>>>>>> diff -r 2134e08cc132 src/share/vm/utilities/vmError.cpp >>>>>>>>>>> --- a/src/share/vm/utilities/vmError.cpp Wed Sep 16 14:28:33 2015 -1000 >>>>>>>>>>> +++ b/src/share/vm/utilities/vmError.cpp Wed Sep 16 15:04:02 2015 -1000 >>>>>>>>>>> @@ -517,6 +517,10 @@ void VMError::report(outputStream* st) { >>>>>>>>>>> Abstract_VM_Version::vm_release(), >>>>>>>>>>> Abstract_VM_Version::vm_info_string(), >>>>>>>>>>> TieredCompilation ? ", tiered" : "", >>>>>>>>>>> +#if INCLUDE_JVMCI >>>>>>>>>>> + EnableJVMCI ? ", jvmci" : "", >>>>>>>>>>> + UseJVMCICompiler ? ", jvmci compiler" : "", >>>>>>>>>>> +#endif >>>>>>>>>>> UseCompressedOops ? ", compressed oops" : "", >>>>>>>>>>> gc_mode(), >>>>>>>>>>> Abstract_VM_Version::vm_platform_string() >>>>>>>>>>> >>>>>>>>>>> It would be useful to see in the crash logs if this experimental feature was turned on. >>>>>>>>>>> >>>>>>>>>>>> On Sep 16, 2015, at 12:43 PM, Vladimir Kozlov wrote: >>>>>>>>>>>> >>>>>>>>>>>> I updated top and hotspot webrev with latest (build) changes. >>>>>>>>>>>> >>>>>>>>>>>> Vladimir >>>>>>>>>>>> >>>>>>>>>>>> On 9/16/15 2:28 PM, Christian Thalinger wrote: >>>>>>>>>>>>>> On Sep 16, 2015, at 6:52 AM, Christian Thalinger wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sep 16, 2015, at 2:57 AM, Magnus Ihse Bursie wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 2015-09-14 09:24, Christian Thalinger wrote: >>>>>>>>>>>>>>>> The JEP itself can be found here: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8062493 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Here are the webrevs: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.top/ >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~kvn/JVMCI/webrev.hotspot/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The change has already undergone a few iterations of internal review by different people of different teams. Most comments and suggestions were handled accordingly. Although there is one open item we agreed we will address after the integration of JEP 243 and that work is captured in RFE: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134994 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> SQE is still working on the tests and all test tasks can be seen as sub-tasks of the JEP. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The integration will happen under the bug number: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8136421 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Christian, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> (Adding build-dev since this review includes some major build changes.) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The makefile changes looks good in general to me. I have not reviewed the source code changes. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Some comments: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> * modules.xml: >>>>>>>>>>>>>>> Make sure you get sign-off from a Jigsaw developer for modifying this file. >>>>>>>>>>>>>> I?ve asked Alan to take a look. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> * hotspot/make/linux/makefiles/gcc.make: >>>>>>>>>>>>>>> Seems unfortunate to have to disable a new warning (undefined-bool-conversion) for newly incorporated code. Is it not possible to fix the code emitting this warning instead? >>>>>>>>>>>>>> We had this question before. It?s not obvious because you can?t see it in the regular diff views but this is under: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ifeq ($(USE_CLANG), true) >>>>>>>>>>>>>> >>>>>>>>>>>>>>> * make/common/MakeBase.gmk and hotspot/make/gensrc/Gensrc-java.base.gmk: >>>>>>>>>>>>>>> I see a coming merge conflict. In jdk9/dev, there is now a new function that performs the same function as CreatePath, but it's named PathList. (It's a bit unfortunate that this code snippet has bounced around as patches without a definite name.) I assume you are going to push this through a hotspot forest. If the PathList patch reaches the hotspot repo before this, please remove the CreatePath from MakeBase, and change the calls to CreatePath to PathList instead. (I could only find such calls in hotspot/make/gensrc/Gensrc-java.base.gmk). If this patch goes in before that, we'll need to give a heads-up to the integrator about this conflict. >>>>>>>>>>>>>> Thanks for the heads up. >>>>>>>>>>>>> Erik sent me a patch to avoid merge conflicts. I?ve integrated two changesets: >>>>>>>>>>>>> >>>>>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/rev/ddedccc5c0ab > >>>>>>>>>>>>> http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/fee6b89199c9 > >>>>>>>>>>>>> >>>>>>>>>>>>>>> Another potential coming merge conflict relates to ListPathsSafely in Gensrc-java.base.gmk. There is currenlty a review out from Erik which modifies the API for ListPathsSafely. If/when it goes in, the call to ListPathsSafely in Gensrc-java.base.gmk will need to be modified (Erik can give advice on how). Depending on timing, this too might hit the integrator rather than your push. >>>>>>>>>>>>>> Ok, thanks. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> /Magnus >> > From per.liden at oracle.com Tue Sep 29 07:55:04 2015 From: per.liden at oracle.com (Per Liden) Date: Tue, 29 Sep 2015 09:55:04 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <56095EAB.5030805@oracle.com> References: <56013DCC.4060307@oracle.com> <56095EAB.5030805@oracle.com> Message-ID: <560A43D8.3050001@oracle.com> Hi David, On 2015-09-28 17:37, David Lindholm wrote: > Kim and Per, > > Thank you! I have made fixes for all your comments, the latest webrev is > available here: > > http://cr.openjdk.java.net/~david/JDK-8080775/webrev.03/ Looks good. /Per > > Kim, I have left the casts in the code, as you say this should be fixed > in another change. > > > Thanks, > David > > On 2015-09-28 16:48, Kim Barrett wrote: >> This is the last of my comments on: >> http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >> >> A few more minor formatting nits, and a couple of things that should >> probably be left for later. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/oops/klassVtable.cpp >> 1464 fatal("klass %s: klass object too short (vtable extends beyond " >> 1465 "end)", _klass->internal_name()); >> >> Remove line break in format string. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/oops/method.cpp >> 249 { ResourceMark rm; >> 250 assert(is_native() && bcp == code_base() || contains(bcp) || >> is_error_reported(), >> 251 "bcp doesn't belong to this method: bcp: " >> INTPTR_FORMAT ", method: %s", bcp, name_and_sig_as_C_string()); >> 252 } >> >> [Pre-existing] >> The assert isn't properly indented within the nested block. Note that >> I think the ResourceMark is required, due to >> name_and_sig_as_C_String(). [Recall that failed assert can still >> return, though I think only in non-PRODUCT mode.] >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/oops/method.cpp >> 282 assert((is_native() && bci == 0) || (!is_native() && 0 <= bci >> && bci < code_size()), "illegal bci: %d", bci); >> >> [Pre-existing] >> Extra space before "||". >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/oops/method.cpp >> 575 if (class_access_flags.is_interface()) { >> 576 assert(is_nonv == is_static(), "is_nonv=%s", >> name_and_sig_as_C_string()); >> 577 } >> >> [Pre-existing] >> Indentation of assert statement should be 2, not 4. >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/oops/oop.cpp >> 126 guarantee(obj->is_oop_or_null(), "invalid oop: " >> INTPTR_FORMAT, p2i((oopDesc*) obj)); >> >> [Pre-existing] >> I don't think the cast is needed here. >> >> [Maybe this sort of thing should be left to separate cleanup of p2i >> usage. See below.] >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/oops/oop.inline.hpp >> 192 assert(check_obj_alignment(result), "address not aligned: " >> INTPTR_FORMAT, p2i((void*) result)); >> >> [Pre-existing] >> I don't think the cast is needed here. >> >> [Maybe this sort of thing should be left to a separate cleanup of p2i >> usage. With two in a row now, I'm inclined to suggest that.] >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/opto/castnode.cpp >> 132 fatal("unexpected comparison %s", ss.as_string()); >> >> [Pre-existing. Any change related to this should be separate from the >> assert formatting arguments change.] >> >> [Note that this change removed err_msg_res.] >> >> There's no obvious associated ResourceMark. But maybe there should >> be, since ss.as_string() performs resource allocation. >> >> Recall that fatal() and other related macros can return under some >> circumstances. (Though I think only non-PRODUCT.) >> >> ------------------------------------------------------------------------------ >> >> src/share/vm/opto/graphKit.hpp >> 356 assert(value->bottom_type()->basic_type() == T_INT, >> 357 "wrong type: %s", >> type2name(value->bottom_type()->basic_type())); >> ... >> 361 assert(value->bottom_type()->basic_type() == T_LONG, >> 362 "wrong type: %s", >> type2name(value->bottom_type()->basic_type())); >> >> Indentation of message. >> >> ------------------------------------------------------------------------------ >> >> > From bengt.rutisson at oracle.com Tue Sep 29 08:09:07 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 29 Sep 2015 10:09:07 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <5609768A.5030209@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> Message-ID: <560A4723.90604@oracle.com> Hi Jon, On 2015-09-28 19:19, Jon Masamitsu wrote: > > > On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >> >> Hi Jon and Per, >> >> I've been trying to get the version proposed in webrev.02 to work but >> I ran into a show stopper for this approach. The G1 concurrent mark >> thread is at times executing (and logging) at the same time as a >> young or mixed GC is executing. There is no good way of handling this >> with a common place to store the GC ID. Instead I've decided to go >> back to storing the current GC ID in the thread. That way the G1 >> concurrent marking and the young/mixed GCs will have their own GC IDs. >> >> Here's an updated webrev: >> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ > Bengt, > > The _gc_id is needed in a JavaThread because a JavaThread > might do work for G1 (concurrent refinement for example) and > needs a GC ID? The Java threads currently don't log anything when they do GC work. But in the future it is likely that they will. > But a WatcherThread would never need one, > right? Right. Initially I only added the _gc_id field to NamedThread. But Per thought it would be better to have it in Thread. Both for making it possible for future improvements (such as logging GC work from JavaThreads) and because it seems like we normally keep storage in the Thread class even for things that are not used by all subclasses. For example TLABs and allocated_bytes() are only relevant for JavaThreads but have their storage in Thread. Thanks, Bengt > > Jon > >> >> This is the same as webrev.01 that both of you already looked at. >> I've made two minor adjustments: >> >> - I removed the currentNamedthread() function in gcId.cpp that Per >> pointed out in his review of webrev.01. Instead I use >> Thread::current() directly. >> >> - I made GCIdMark a StackObj. >> >> Otherwise the patch is the same as in webrev.01. >> >> Thanks, >> Bengt >> >> On 2015-09-10 14:37, Bengt Rutisson wrote: >>> >>> Hi Jon, >>> >>> Thanks for looking at this! >>> >>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>> Bengt, >>>> >>>> When a CMS concurrent collection hands off to a STW foreground >>>> collection, >>>> (done in acquire_control_and_collect()), I expected a new GCId >>>> would be >>>> used. Did I miss it? That STW collection does not go through >>>> do_collection(). >>> >>> The call to acquire_control_and_collect() originates from >>> GenCollectedHeap::collect_generation() and there is a new GCIdMark >>> in there that will create a new GCId for the STW collection. That's >>> the "extra" GCIdMark that Per asked about in his review. >>> >>> *But* I wanted to verify that it worked properly since you asked >>> about it and I realized that there is another bug. >>> >>> The GCId that is set up for the concurrent cycle, the one set up in >>> ConcurrentMarkSweepThread::run(), will is still active over the >>> complete STW foreground collection. That's fine in my model since >>> the new GCIdMark in GenCollectedHeap::collect_generation() will >>> cache that GCId. But the ConcurrentMarkSweep thread is not >>> completely idle even though control has been left to the foreground >>> collection in acquire_control_and_collect(). So, there is some >>> logging done (by fore example CMSPhaseAccounting) that is done at >>> the same time as the foreground collection. This logging will now >>> use the foreground GCId instead of the CMS GCId. >>> >>> I haven't had time to dig in to the details of that just yet. But >>> this is an unintended change of the logging, so I would like to fix >>> it. Hopefully I'll have an updated webrev tomorrow. >>> >>> Thanks, >>> Bengt >>> >>> >>>> >>>> Jon >>>> >>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>> >>>>> Hi again, >>>>> >>>>> I found an issue with how G1 young collections kick off concurrent >>>>> marks. There is a short window where we might have two active GC >>>>> IDs at the same time. I've fixed this and updated the webrevs. In >>>>> case anyone had already started looking at the webrevs you need to >>>>> do a refresh now. The fix is in >>>>> G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Per, >>>>>> >>>>>> Thanks for looking at this! >>>>>> >>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>> Hi Bengt, >>>>>>> >>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> This is mostly a GC related patch, but it adds a field to the >>>>>>>> Thread >>>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>>> >>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>> >>>>>>> >>>>>>> Looks good. I think this is a nice simplification, especially >>>>>>> for G1's concurrent parts. Just two comments: >>>>>>> >>>>>>> genCollectedHeap.cpp: >>>>>>> --------------------- >>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>>> different scopes. Do we really want that second mark? >>>>>> >>>>>> >>>>>> We potentially do two GCs in GenCollectedHeap::do_collection(). >>>>>> First a young GC and then potentially a full GC. These two should >>>>>> have different GC IDs. So, yes, we need two GCIdMarks in this >>>>>> method. The scoping could be better, but I think that is a >>>>>> refactoring that should be done separately from my current patch. >>>>>> I'll talk to Jesper about it since he has been cleaning up this >>>>>> code lately. >>>>>> >>>>>>> >>>>>>> >>>>>>> gcId.cpp: >>>>>>> -------- >>>>>>> - I think you might have left currentNamedthread() in there. You >>>>>>> probably just want to use Thread::current() instead. >>>>>> >>>>>> Yes, good catch. Thanks. >>>>>> >>>>>> However, after thinking some more about the implications of >>>>>> moving the GC ID out from the GCTracer I figured it would be >>>>>> possible to just store the previous GC ID in the GCIdMark and >>>>>> then restore it in the destructor of that class. That way we >>>>>> don't have to store anything in the Thread class but can still >>>>>> have both a concurrent GC ID and a STW GC ID active at the same >>>>>> time. This also removes the need to copy the GC ID to the worker >>>>>> tasks. >>>>>> >>>>>> Here's an updated webrev with a solution that does not add >>>>>> anything to the Thread class: >>>>>> >>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>> >>>>>> And here's a diff compared to the last version: >>>>>> >>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>> >>>>>> Unfotunately the webrev tool had some bad luck when creating the >>>>>> diff webrev so at least the g1CollectedHeap.cpp seem to contain >>>>>> the complete changes rather than just the diff compared to the 01 >>>>>> version. The rest of the diff looks correct as far as I can tell. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>>> >>>>>>> >>>>>>> cheers, >>>>>>> /Per >>>>>>> >>>>>>> >>>>>>>> Currently the GC ID, that is used for event tracing and >>>>>>>> logging, is >>>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>>> problems since >>>>>>>> there are multiple GCTracers active at the same time. The >>>>>>>> correct GC IDs >>>>>>>> need to be passed around in many places. >>>>>>>> >>>>>>>> For some upcoming GC logging changes I would like to have a more >>>>>>>> consistent way of finding the currently active GC ID. I've >>>>>>>> played around >>>>>>>> with a couple of different solutions, but eventually I found >>>>>>>> that the >>>>>>>> simplest solution is to store the current GC ID in the thread. >>>>>>>> That way >>>>>>>> there is a single way to look it up in all places. It is also >>>>>>>> fairly >>>>>>>> straight forward to set it up. >>>>>>>> >>>>>>>> I've reworked the GCId class a bit to support this and I've >>>>>>>> introduced a >>>>>>>> GCIdMark class that is a scoped object that helps to set up and >>>>>>>> tear >>>>>>>> down a current GC ID. By moving the GC ID out from the GCTracer >>>>>>>> class I >>>>>>>> got rid of a few minor issues as well. One is that we no longer >>>>>>>> need to >>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was >>>>>>>> necessary >>>>>>>> before since we did logging *after* we had told the corresponding >>>>>>>> GCTracer that the concurrent cycle was aborted and it had >>>>>>>> thrown its GC >>>>>>>> ID away. Now the GC ID can stay in scope until we have >>>>>>>> completed all >>>>>>>> logging. >>>>>>>> >>>>>>>> Also the HeapDumpBeforeFullGC and >>>>>>>> PrintClassHistogramBeforeFullGC used >>>>>>>> to have to create a new GC ID since their logging was done >>>>>>>> before we had >>>>>>>> the correct GCTracer set up. Now the GC ID can be available >>>>>>>> early enough >>>>>>>> for this logging to be using the same (and correct) GC ID as >>>>>>>> the rest of >>>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>>> PrintClassHistogramAfterFullGC. >>>>>>>> >>>>>>>> I've added an uint to the Thread class to keep track of the >>>>>>>> currently >>>>>>>> active GC ID. In the current code there are actually only >>>>>>>> NamedThreads >>>>>>>> that need the GC ID. So, I'm not sure where the best place is >>>>>>>> for this >>>>>>>> field is. But it seems like the Thread class contains most of >>>>>>>> the data, >>>>>>>> even data that is only used by some subclasses, so I opted for >>>>>>>> putting >>>>>>>> the field in Thread as opposed to in NamedThread. This opens up >>>>>>>> for >>>>>>>> possible future extensions when Java threads may do part of the >>>>>>>> GC work. >>>>>>>> However, I'm open to moving the field down to NamedThread if >>>>>>>> that is >>>>>>>> seen as better. >>>>>>>> >>>>>>>> In the GCTracer class there were many asserts that were >>>>>>>> checking that >>>>>>>> the GC ID was properly set up. Mostly these assert verify that >>>>>>>> start/end >>>>>>>> is called correctly and that the other methods are called >>>>>>>> inside of the >>>>>>>> start/end scope. Now that the GC ID is moved out of the >>>>>>>> GCTracer class >>>>>>>> these asserts had to be dealt with. I figured there were three >>>>>>>> ways to >>>>>>>> handle it; just remove them, replace them with check that the >>>>>>>> GC ID from >>>>>>>> the current thread is correct, or implement a new status field >>>>>>>> to keep >>>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>>> asserts are >>>>>>>> valuable enough to spend time on, so I went for the first >>>>>>>> approach: >>>>>>>> removing them. I don't think making them use the thread GC ID >>>>>>>> will be >>>>>>>> appropriate. But if anyone feels strongly about these asserts I >>>>>>>> can >>>>>>>> implement a separate start/end state. >>>>>>>> >>>>>>>> There are a few "Tasks" in the GC code that are executed by worker >>>>>>>> threads. To make the worker threads use the correct GC ID I >>>>>>>> make sure >>>>>>>> that the Task constructors copy the GC ID from the initiating >>>>>>>> thread >>>>>>>> into a local variable. When the task is executed in its worker >>>>>>>> thread it >>>>>>>> sets up the GC ID based on the local variable. >>>>>>>> >>>>>>>> The proposed change does not alter any logging (except for the >>>>>>>> small bug >>>>>>>> fix for >>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>>> This >>>>>>>> means that no existing tests need to be updated. In particular the >>>>>>>> test/gc/logging/TestGCId.java test passes even after these >>>>>>>> changes. >>>>>>>> >>>>>>>> >>>>>>>> A big thanks to Per, who pre-reviewed these changes and came >>>>>>>> with some >>>>>>>> really good feedback. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>> >>>>> >>>> >>> >> > From david.lindholm at oracle.com Tue Sep 29 09:24:32 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Tue, 29 Sep 2015 11:24:32 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <297F25EB-03B6-48E2-9BEF-E9AC3E1C6515@oracle.com> References: <56013DCC.4060307@oracle.com> <56095EAB.5030805@oracle.com> <297F25EB-03B6-48E2-9BEF-E9AC3E1C6515@oracle.com> Message-ID: <560A58D0.9040708@oracle.com> Kim, Thank you for the review! /David On 2015-09-28 19:11, Kim Barrett wrote: > On Sep 28, 2015, at 11:37 AM, David Lindholm wrote: >> Kim and Per, >> >> Thank you! I have made fixes for all your comments, the latest webrev is available here: >> >> http://cr.openjdk.java.net/~david/JDK-8080775/webrev.03/ >> >> Kim, I have left the casts in the code, as you say this should be fixed in another change. > ------------------------------------------------------------------------------ >> src/share/vm/gc/g1/g1BlockOffsetTable.cpp >> 364 assert((_array->offset_array(orig_index) == 0 && >> 365 blk_start == boundary) || >> 366 (_array->offset_array(orig_index) > 0 && >> 367 _array->offset_array(orig_index) <= N_words), >> >> Indentation of 366 and 367 are wrong. > webrev.03 has > 364 assert((_array->offset_array(orig_index) == 0 && > 365 blk_start == boundary) || > 366 (_array->offset_array(orig_index) > 0 && > 367 _array->offset_array(orig_index) <= N_words), > > Lines 365 and 367 should be indented one more space. > > ------------------------------------------------------------------------------ > > Otherwise, looks good. I don't need a new webrev for the above. > From david.lindholm at oracle.com Tue Sep 29 09:24:54 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Tue, 29 Sep 2015 11:24:54 +0200 Subject: RFR: 8080775: Better argument formatting for assert() and friends In-Reply-To: <560A43D8.3050001@oracle.com> References: <56013DCC.4060307@oracle.com> <56095EAB.5030805@oracle.com> <560A43D8.3050001@oracle.com> Message-ID: <560A58E6.6010401@oracle.com> Per, Thank you for the review! /David On 2015-09-29 09:55, Per Liden wrote: > Hi David, > > On 2015-09-28 17:37, David Lindholm wrote: >> Kim and Per, >> >> Thank you! I have made fixes for all your comments, the latest webrev is >> available here: >> >> http://cr.openjdk.java.net/~david/JDK-8080775/webrev.03/ > > Looks good. > > /Per > >> >> Kim, I have left the casts in the code, as you say this should be fixed >> in another change. >> >> >> Thanks, >> David >> >> On 2015-09-28 16:48, Kim Barrett wrote: >>> This is the last of my comments on: >>> http://cr.openjdk.java.net/~david/JDK-8080775/webrev.00/ >>> >>> A few more minor formatting nits, and a couple of things that should >>> probably be left for later. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/oops/klassVtable.cpp >>> 1464 fatal("klass %s: klass object too short (vtable extends >>> beyond " >>> 1465 "end)", _klass->internal_name()); >>> >>> Remove line break in format string. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/oops/method.cpp >>> 249 { ResourceMark rm; >>> 250 assert(is_native() && bcp == code_base() || contains(bcp) || >>> is_error_reported(), >>> 251 "bcp doesn't belong to this method: bcp: " >>> INTPTR_FORMAT ", method: %s", bcp, name_and_sig_as_C_string()); >>> 252 } >>> >>> [Pre-existing] >>> The assert isn't properly indented within the nested block. Note that >>> I think the ResourceMark is required, due to >>> name_and_sig_as_C_String(). [Recall that failed assert can still >>> return, though I think only in non-PRODUCT mode.] >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/oops/method.cpp >>> 282 assert((is_native() && bci == 0) || (!is_native() && 0 <= bci >>> && bci < code_size()), "illegal bci: %d", bci); >>> >>> [Pre-existing] >>> Extra space before "||". >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/oops/method.cpp >>> 575 if (class_access_flags.is_interface()) { >>> 576 assert(is_nonv == is_static(), "is_nonv=%s", >>> name_and_sig_as_C_string()); >>> 577 } >>> >>> [Pre-existing] >>> Indentation of assert statement should be 2, not 4. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/oops/oop.cpp >>> 126 guarantee(obj->is_oop_or_null(), "invalid oop: " >>> INTPTR_FORMAT, p2i((oopDesc*) obj)); >>> >>> [Pre-existing] >>> I don't think the cast is needed here. >>> >>> [Maybe this sort of thing should be left to separate cleanup of p2i >>> usage. See below.] >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/oops/oop.inline.hpp >>> 192 assert(check_obj_alignment(result), "address not aligned: " >>> INTPTR_FORMAT, p2i((void*) result)); >>> >>> [Pre-existing] >>> I don't think the cast is needed here. >>> >>> [Maybe this sort of thing should be left to a separate cleanup of p2i >>> usage. With two in a row now, I'm inclined to suggest that.] >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/opto/castnode.cpp >>> 132 fatal("unexpected comparison %s", ss.as_string()); >>> >>> [Pre-existing. Any change related to this should be separate from the >>> assert formatting arguments change.] >>> >>> [Note that this change removed err_msg_res.] >>> >>> There's no obvious associated ResourceMark. But maybe there should >>> be, since ss.as_string() performs resource allocation. >>> >>> Recall that fatal() and other related macros can return under some >>> circumstances. (Though I think only non-PRODUCT.) >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> src/share/vm/opto/graphKit.hpp >>> 356 assert(value->bottom_type()->basic_type() == T_INT, >>> 357 "wrong type: %s", >>> type2name(value->bottom_type()->basic_type())); >>> ... >>> 361 assert(value->bottom_type()->basic_type() == T_LONG, >>> 362 "wrong type: %s", >>> type2name(value->bottom_type()->basic_type())); >>> >>> Indentation of message. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> >> From dmitry.dmitriev at oracle.com Tue Sep 29 12:50:33 2015 From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev) Date: Tue, 29 Sep 2015 15:50:33 +0300 Subject: RFR (S): JDK-8129855: -XX:+IgnoreUnrecognizedVMOptions hides out of range VM options. In-Reply-To: <5609B768.8010109@oracle.com> References: <5609B768.8010109@oracle.com> Message-ID: <560A8919.8050700@oracle.com> Hi Gerard, Hotspot changes look good to me. Comments about new test. 1) I think that test should be placed in "test/runtime/CommandLine", because recently we don't use bugid folder to put tests. 2) Since this is a new test, then we need only 2015 in copyright year at line 2. 3) Extra spaces between "@bug" and "8129855" on line 28. 4) Lines 42-49: use Platform.isDebugBuild() from test library(test/testlibrary/jdk/test/lib/Platform.java). It do the same thing. 5) Test perform similar things each step: create process builder, start it and verify exit value. I think you can introduce additional function which perform these things and in this case you can remove duplicated code. For example, you can introduce following function: private static void ranJavaCheckExitValue(boolean shouldSuccess, String... args) { ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(args); OutputAnalyzer output = new OutputAnalyzer(pb.start()); if (shouldSuccess) { output.shouldHaveExitValue(0); } else { output.shouldHaveExitValue(1); } } And test will looks like this(lines 51-69 in the original test): /* #1.1 normal flag: exists, invalid value does not exist -XX:MinHeapFreeRatio=notnum -XX:THIS_FLAG_DOESNT_EXIST -IgnoreUnrecognizedVMOptions ERR ERR +IgnoreUnrecognizedVMOptions ERR OK */ ranJavaCheckExitValue(false, "-XX:-IgnoreUnrecognizedVMOptions", "-XX:MinHeapFreeRatio=notnum", "-version"); ranJavaCheckExitValue(false, "-XX:-IgnoreUnrecognizedVMOptions", "-XX:THIS_FLAG_DOESNT_EXIST", "-version"); ranJavaCheckExitValue(false, "-XX:+IgnoreUnrecognizedVMOptions", "-XX:MinHeapFreeRatio=notnum", "-version"); ranJavaCheckExitValue(true, "-XX:+IgnoreUnrecognizedVMOptions", "-XX:THIS_FLAG_DOESNT_EXIST", "-version"); 6) I think that empty line 232 is not needed. Thanks, Dmitry On 29.09.2015 0:55, gerard ziemski wrote: > hi all, > > We are fixing how IgnoreUnrecognizedVMOptions treats those flags whose > values are out of range. Before the fix, the VM would continue even if > flag?s value was out of range, ex: > > java_old -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version > size_t MinTLABSize=0 is outside the allowed range [ 1 ... 4294967295 ] > java version "1.9.0-internal-fastdebug" > Java(TM) SE Runtime Environment (build > 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00) > Java HotSpot(TM) Server VM (build > 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00, mixed mode) > > now, we correctly exit the VM with an error, ex: > > java_new -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version > size_t MinTLABSize=0 is outside the allowed range [ 1 ... > 18446744073709551615 ] > Improperly specified VM option 'MinTLABSize=0' > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > In addition IgnoreUnrecognizedVMOptions nows strictly follows the spec > as outlined in JDK-8129855. The behavior changes depending on whether > the build is product or debug. > > We also introduce a new test that verifies all known use cases that we > care about. > > References: > bugid: https://bugs.openjdk.java.net/browse/JDK-8129855 > webrev: http://cr.openjdk.java.net/~gziemski/8129855_rev0 > discussion: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-June/019213.html > > Passes "JPRT hotspot" and "RBT testlist,noncolo.testlist quick" > > > cheers From david.lindholm at oracle.com Tue Sep 29 12:58:09 2015 From: david.lindholm at oracle.com (David Lindholm) Date: Tue, 29 Sep 2015 14:58:09 +0200 Subject: New assert(), guarantee(), fatal() and vm_exit_out_of_memory() macros. Message-ID: <560A8AE1.3030608@oracle.com> Hi All, I have now pushed the new versions of the macros assert(), guarantee(), fatal() and vm_exit_out_of_memory(). These are now variadic macros, which means that you do not need to format the detailed error message using err_msg() or err_msg_res(). So, for example, guarantee(_saved_index == _index, err_msg("saved index: %d index: %d", _saved_index, _index)); is replaced by guarantee(_saved_index == _index, "saved index: %d index: %d", _saved_index, _index); I have made changes to these macros, and also removed all err_msg() and err_msg_res() in usages of these macros. err_msg_res() is completely removed while err_msg() is still used from other functions. Thanks, David From gerard.ziemski at oracle.com Tue Sep 29 13:57:59 2015 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Tue, 29 Sep 2015 08:57:59 -0500 Subject: RFR (S): JDK-8129855: -XX:+IgnoreUnrecognizedVMOptions hides out of range VM options. In-Reply-To: <560A8919.8050700@oracle.com> References: <5609B768.8010109@oracle.com> <560A8919.8050700@oracle.com> Message-ID: <560A98E7.80707@oracle.com> Excellent suggestions! Thank you Dmitry! cheers On 09/29/2015 07:50 AM, Dmitry Dmitriev wrote: > Hi Gerard, > > Hotspot changes look good to me. Comments about new test. > 1) I think that test should be placed in "test/runtime/CommandLine", because recently we don't use bugid folder to put > tests. > 2) Since this is a new test, then we need only 2015 in copyright year at line 2. > 3) Extra spaces between "@bug" and "8129855" on line 28. > 4) Lines 42-49: use Platform.isDebugBuild() from test library(test/testlibrary/jdk/test/lib/Platform.java). It do the > same thing. > 5) Test perform similar things each step: create process builder, start it and verify exit value. I think you can > introduce additional function which perform these things and in this case you can remove duplicated code. > For example, you can introduce following function: > private static void ranJavaCheckExitValue(boolean shouldSuccess, String... args) { > ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(args); > OutputAnalyzer output = new OutputAnalyzer(pb.start()); > if (shouldSuccess) { > output.shouldHaveExitValue(0); > } else { > output.shouldHaveExitValue(1); > } > } > > And test will looks like this(lines 51-69 in the original test): > /* > #1.1 normal flag: > exists, invalid value does not exist > -XX:MinHeapFreeRatio=notnum -XX:THIS_FLAG_DOESNT_EXIST > -IgnoreUnrecognizedVMOptions ERR ERR > +IgnoreUnrecognizedVMOptions ERR OK > */ > ranJavaCheckExitValue(false, "-XX:-IgnoreUnrecognizedVMOptions", "-XX:MinHeapFreeRatio=notnum", "-version"); > ranJavaCheckExitValue(false, "-XX:-IgnoreUnrecognizedVMOptions", "-XX:THIS_FLAG_DOESNT_EXIST", "-version"); > ranJavaCheckExitValue(false, "-XX:+IgnoreUnrecognizedVMOptions", "-XX:MinHeapFreeRatio=notnum", "-version"); > ranJavaCheckExitValue(true, "-XX:+IgnoreUnrecognizedVMOptions", "-XX:THIS_FLAG_DOESNT_EXIST", "-version"); > > 6) I think that empty line 232 is not needed. > > Thanks, > Dmitry > > On 29.09.2015 0:55, gerard ziemski wrote: >> hi all, >> >> We are fixing how IgnoreUnrecognizedVMOptions treats those flags whose values are out of range. Before the fix, the VM >> would continue even if flag?s value was out of range, ex: >> >> java_old -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version >> size_t MinTLABSize=0 is outside the allowed range [ 1 ... 4294967295 ] >> java version "1.9.0-internal-fastdebug" >> Java(TM) SE Runtime Environment (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00) >> Java HotSpot(TM) Server VM (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00, mixed mode) >> >> now, we correctly exit the VM with an error, ex: >> >> java_new -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version >> size_t MinTLABSize=0 is outside the allowed range [ 1 ... 18446744073709551615 ] >> Improperly specified VM option 'MinTLABSize=0' >> Error: Could not create the Java Virtual Machine. >> Error: A fatal exception has occurred. Program will exit. >> >> In addition IgnoreUnrecognizedVMOptions nows strictly follows the spec as outlined in JDK-8129855. The behavior >> changes depending on whether the build is product or debug. >> >> We also introduce a new test that verifies all known use cases that we care about. >> >> References: >> bugid: https://bugs.openjdk.java.net/browse/JDK-8129855 >> webrev: http://cr.openjdk.java.net/~gziemski/8129855_rev0 >> discussion: http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-June/019213.html >> >> Passes "JPRT hotspot" and "RBT testlist,noncolo.testlist quick" >> >> >> cheers > > From jon.masamitsu at oracle.com Tue Sep 29 15:59:13 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 29 Sep 2015 08:59:13 -0700 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <560A4723.90604@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> <560A4723.90604@oracle.com> Message-ID: <560AB551.6010507@oracle.com> On 9/29/15 1:09 AM, Bengt Rutisson wrote: > > Hi Jon, > > On 2015-09-28 19:19, Jon Masamitsu wrote: >> >> >> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >>> >>> Hi Jon and Per, >>> >>> I've been trying to get the version proposed in webrev.02 to work >>> but I ran into a show stopper for this approach. The G1 concurrent >>> mark thread is at times executing (and logging) at the same time as >>> a young or mixed GC is executing. There is no good way of handling >>> this with a common place to store the GC ID. Instead I've decided to >>> go back to storing the current GC ID in the thread. That way the G1 >>> concurrent marking and the young/mixed GCs will have their own GC IDs. >>> >>> Here's an updated webrev: >>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ >> Bengt, >> >> The _gc_id is needed in a JavaThread because a JavaThread >> might do work for G1 (concurrent refinement for example) and >> needs a GC ID? > > The Java threads currently don't log anything when they do GC work. > But in the future it is likely that they will. > >> But a WatcherThread would never need one, >> right? > > Right. Initially I only added the _gc_id field to NamedThread. But Per > thought it would be better to have it in Thread. Both for making it > possible for future improvements (such as logging GC work from > JavaThreads) and because it seems like we normally keep storage in the > Thread class even for things that are not used by all subclasses. For > example TLABs and allocated_bytes() are only relevant for JavaThreads > but have their storage in Thread. Yes, Thread could be improved but I'm not sure I can stand adding GC data where it's not needed. Unless GC logging from JavaThread is something that's going to happen soon, I'd rather add it when it's needed rather than right now. I'd personally rather duplicate the code in JavaThread if it is needed rather than have WatcherThreads have a _gc_id. Jon > > Thanks, > Bengt > >> >> Jon >> >>> >>> This is the same as webrev.01 that both of you already looked at. >>> I've made two minor adjustments: >>> >>> - I removed the currentNamedthread() function in gcId.cpp that Per >>> pointed out in his review of webrev.01. Instead I use >>> Thread::current() directly. >>> >>> - I made GCIdMark a StackObj. >>> >>> Otherwise the patch is the same as in webrev.01. >>> >>> Thanks, >>> Bengt >>> >>> On 2015-09-10 14:37, Bengt Rutisson wrote: >>>> >>>> Hi Jon, >>>> >>>> Thanks for looking at this! >>>> >>>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>>> Bengt, >>>>> >>>>> When a CMS concurrent collection hands off to a STW foreground >>>>> collection, >>>>> (done in acquire_control_and_collect()), I expected a new GCId >>>>> would be >>>>> used. Did I miss it? That STW collection does not go through >>>>> do_collection(). >>>> >>>> The call to acquire_control_and_collect() originates from >>>> GenCollectedHeap::collect_generation() and there is a new GCIdMark >>>> in there that will create a new GCId for the STW collection. That's >>>> the "extra" GCIdMark that Per asked about in his review. >>>> >>>> *But* I wanted to verify that it worked properly since you asked >>>> about it and I realized that there is another bug. >>>> >>>> The GCId that is set up for the concurrent cycle, the one set up in >>>> ConcurrentMarkSweepThread::run(), will is still active over the >>>> complete STW foreground collection. That's fine in my model since >>>> the new GCIdMark in GenCollectedHeap::collect_generation() will >>>> cache that GCId. But the ConcurrentMarkSweep thread is not >>>> completely idle even though control has been left to the foreground >>>> collection in acquire_control_and_collect(). So, there is some >>>> logging done (by fore example CMSPhaseAccounting) that is done at >>>> the same time as the foreground collection. This logging will now >>>> use the foreground GCId instead of the CMS GCId. >>>> >>>> I haven't had time to dig in to the details of that just yet. But >>>> this is an unintended change of the logging, so I would like to fix >>>> it. Hopefully I'll have an updated webrev tomorrow. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> >>>>> >>>>> Jon >>>>> >>>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>>> >>>>>> Hi again, >>>>>> >>>>>> I found an issue with how G1 young collections kick off >>>>>> concurrent marks. There is a short window where we might have two >>>>>> active GC IDs at the same time. I've fixed this and updated the >>>>>> webrevs. In case anyone had already started looking at the >>>>>> webrevs you need to do a refresh now. The fix is in >>>>>> G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Per, >>>>>>> >>>>>>> Thanks for looking at this! >>>>>>> >>>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>>> Hi Bengt, >>>>>>>> >>>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi everyone, >>>>>>>>> >>>>>>>>> This is mostly a GC related patch, but it adds a field to the >>>>>>>>> Thread >>>>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>>>> >>>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>>> >>>>>>>> >>>>>>>> Looks good. I think this is a nice simplification, especially >>>>>>>> for G1's concurrent parts. Just two comments: >>>>>>>> >>>>>>>> genCollectedHeap.cpp: >>>>>>>> --------------------- >>>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>>>> different scopes. Do we really want that second mark? >>>>>>> >>>>>>> >>>>>>> We potentially do two GCs in GenCollectedHeap::do_collection(). >>>>>>> First a young GC and then potentially a full GC. These two >>>>>>> should have different GC IDs. So, yes, we need two GCIdMarks in >>>>>>> this method. The scoping could be better, but I think that is a >>>>>>> refactoring that should be done separately from my current >>>>>>> patch. I'll talk to Jesper about it since he has been cleaning >>>>>>> up this code lately. >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> gcId.cpp: >>>>>>>> -------- >>>>>>>> - I think you might have left currentNamedthread() in there. >>>>>>>> You probably just want to use Thread::current() instead. >>>>>>> >>>>>>> Yes, good catch. Thanks. >>>>>>> >>>>>>> However, after thinking some more about the implications of >>>>>>> moving the GC ID out from the GCTracer I figured it would be >>>>>>> possible to just store the previous GC ID in the GCIdMark and >>>>>>> then restore it in the destructor of that class. That way we >>>>>>> don't have to store anything in the Thread class but can still >>>>>>> have both a concurrent GC ID and a STW GC ID active at the same >>>>>>> time. This also removes the need to copy the GC ID to the worker >>>>>>> tasks. >>>>>>> >>>>>>> Here's an updated webrev with a solution that does not add >>>>>>> anything to the Thread class: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>>> >>>>>>> And here's a diff compared to the last version: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>>> >>>>>>> Unfotunately the webrev tool had some bad luck when creating the >>>>>>> diff webrev so at least the g1CollectedHeap.cpp seem to contain >>>>>>> the complete changes rather than just the diff compared to the >>>>>>> 01 version. The rest of the diff looks correct as far as I can >>>>>>> tell. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>> cheers, >>>>>>>> /Per >>>>>>>> >>>>>>>> >>>>>>>>> Currently the GC ID, that is used for event tracing and >>>>>>>>> logging, is >>>>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>>>> problems since >>>>>>>>> there are multiple GCTracers active at the same time. The >>>>>>>>> correct GC IDs >>>>>>>>> need to be passed around in many places. >>>>>>>>> >>>>>>>>> For some upcoming GC logging changes I would like to have a more >>>>>>>>> consistent way of finding the currently active GC ID. I've >>>>>>>>> played around >>>>>>>>> with a couple of different solutions, but eventually I found >>>>>>>>> that the >>>>>>>>> simplest solution is to store the current GC ID in the thread. >>>>>>>>> That way >>>>>>>>> there is a single way to look it up in all places. It is also >>>>>>>>> fairly >>>>>>>>> straight forward to set it up. >>>>>>>>> >>>>>>>>> I've reworked the GCId class a bit to support this and I've >>>>>>>>> introduced a >>>>>>>>> GCIdMark class that is a scoped object that helps to set up >>>>>>>>> and tear >>>>>>>>> down a current GC ID. By moving the GC ID out from the >>>>>>>>> GCTracer class I >>>>>>>>> got rid of a few minor issues as well. One is that we no >>>>>>>>> longer need to >>>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was >>>>>>>>> necessary >>>>>>>>> before since we did logging *after* we had told the corresponding >>>>>>>>> GCTracer that the concurrent cycle was aborted and it had >>>>>>>>> thrown its GC >>>>>>>>> ID away. Now the GC ID can stay in scope until we have >>>>>>>>> completed all >>>>>>>>> logging. >>>>>>>>> >>>>>>>>> Also the HeapDumpBeforeFullGC and >>>>>>>>> PrintClassHistogramBeforeFullGC used >>>>>>>>> to have to create a new GC ID since their logging was done >>>>>>>>> before we had >>>>>>>>> the correct GCTracer set up. Now the GC ID can be available >>>>>>>>> early enough >>>>>>>>> for this logging to be using the same (and correct) GC ID as >>>>>>>>> the rest of >>>>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>>>> PrintClassHistogramAfterFullGC. >>>>>>>>> >>>>>>>>> I've added an uint to the Thread class to keep track of the >>>>>>>>> currently >>>>>>>>> active GC ID. In the current code there are actually only >>>>>>>>> NamedThreads >>>>>>>>> that need the GC ID. So, I'm not sure where the best place is >>>>>>>>> for this >>>>>>>>> field is. But it seems like the Thread class contains most of >>>>>>>>> the data, >>>>>>>>> even data that is only used by some subclasses, so I opted for >>>>>>>>> putting >>>>>>>>> the field in Thread as opposed to in NamedThread. This opens >>>>>>>>> up for >>>>>>>>> possible future extensions when Java threads may do part of >>>>>>>>> the GC work. >>>>>>>>> However, I'm open to moving the field down to NamedThread if >>>>>>>>> that is >>>>>>>>> seen as better. >>>>>>>>> >>>>>>>>> In the GCTracer class there were many asserts that were >>>>>>>>> checking that >>>>>>>>> the GC ID was properly set up. Mostly these assert verify that >>>>>>>>> start/end >>>>>>>>> is called correctly and that the other methods are called >>>>>>>>> inside of the >>>>>>>>> start/end scope. Now that the GC ID is moved out of the >>>>>>>>> GCTracer class >>>>>>>>> these asserts had to be dealt with. I figured there were three >>>>>>>>> ways to >>>>>>>>> handle it; just remove them, replace them with check that the >>>>>>>>> GC ID from >>>>>>>>> the current thread is correct, or implement a new status field >>>>>>>>> to keep >>>>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>>>> asserts are >>>>>>>>> valuable enough to spend time on, so I went for the first >>>>>>>>> approach: >>>>>>>>> removing them. I don't think making them use the thread GC ID >>>>>>>>> will be >>>>>>>>> appropriate. But if anyone feels strongly about these asserts >>>>>>>>> I can >>>>>>>>> implement a separate start/end state. >>>>>>>>> >>>>>>>>> There are a few "Tasks" in the GC code that are executed by >>>>>>>>> worker >>>>>>>>> threads. To make the worker threads use the correct GC ID I >>>>>>>>> make sure >>>>>>>>> that the Task constructors copy the GC ID from the initiating >>>>>>>>> thread >>>>>>>>> into a local variable. When the task is executed in its worker >>>>>>>>> thread it >>>>>>>>> sets up the GC ID based on the local variable. >>>>>>>>> >>>>>>>>> The proposed change does not alter any logging (except for the >>>>>>>>> small bug >>>>>>>>> fix for >>>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>>>> This >>>>>>>>> means that no existing tests need to be updated. In particular >>>>>>>>> the >>>>>>>>> test/gc/logging/TestGCId.java test passes even after these >>>>>>>>> changes. >>>>>>>>> >>>>>>>>> >>>>>>>>> A big thanks to Per, who pre-reviewed these changes and came >>>>>>>>> with some >>>>>>>>> really good feedback. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>> >>>>>> >>>>> >>>> >>> >> > From bengt.rutisson at oracle.com Tue Sep 29 15:57:48 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Tue, 29 Sep 2015 17:57:48 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <560AB551.6010507@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> <560A4723.90604@oracle.com> <560AB551.6010507@oracle.com> Message-ID: <560AB4FC.1050805@oracle.com> On 2015-09-29 17:59, Jon Masamitsu wrote: > > > On 9/29/15 1:09 AM, Bengt Rutisson wrote: >> >> Hi Jon, >> >> On 2015-09-28 19:19, Jon Masamitsu wrote: >>> >>> >>> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >>>> >>>> Hi Jon and Per, >>>> >>>> I've been trying to get the version proposed in webrev.02 to work >>>> but I ran into a show stopper for this approach. The G1 concurrent >>>> mark thread is at times executing (and logging) at the same time as >>>> a young or mixed GC is executing. There is no good way of handling >>>> this with a common place to store the GC ID. Instead I've decided >>>> to go back to storing the current GC ID in the thread. That way the >>>> G1 concurrent marking and the young/mixed GCs will have their own >>>> GC IDs. >>>> >>>> Here's an updated webrev: >>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ >>> Bengt, >>> >>> The _gc_id is needed in a JavaThread because a JavaThread >>> might do work for G1 (concurrent refinement for example) and >>> needs a GC ID? >> >> The Java threads currently don't log anything when they do GC work. >> But in the future it is likely that they will. >> >>> But a WatcherThread would never need one, >>> right? >> >> Right. Initially I only added the _gc_id field to NamedThread. But >> Per thought it would be better to have it in Thread. Both for making >> it possible for future improvements (such as logging GC work from >> JavaThreads) and because it seems like we normally keep storage in >> the Thread class even for things that are not used by all subclasses. >> For example TLABs and allocated_bytes() are only relevant for >> JavaThreads but have their storage in Thread. > > Yes, Thread could be improved but I'm not sure I can stand adding GC > data where > it's not needed. Unless GC logging from JavaThread is something > that's going to > happen soon, I'd rather add it when it's needed rather than right > now. I'd personally > rather duplicate the code in JavaThread if it is needed rather than > have WatcherThreads > have a _gc_id. I tend to agree. I'll discuss this with Per tomorrow. If he agrees that we can move the _gc_id down to NamedThread, are you ok with me pushing the fix? Thanks, Bengt > > Jon > >> >> Thanks, >> Bengt >> >>> >>> Jon >>> >>>> >>>> This is the same as webrev.01 that both of you already looked at. >>>> I've made two minor adjustments: >>>> >>>> - I removed the currentNamedthread() function in gcId.cpp that Per >>>> pointed out in his review of webrev.01. Instead I use >>>> Thread::current() directly. >>>> >>>> - I made GCIdMark a StackObj. >>>> >>>> Otherwise the patch is the same as in webrev.01. >>>> >>>> Thanks, >>>> Bengt >>>> >>>> On 2015-09-10 14:37, Bengt Rutisson wrote: >>>>> >>>>> Hi Jon, >>>>> >>>>> Thanks for looking at this! >>>>> >>>>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>>>> Bengt, >>>>>> >>>>>> When a CMS concurrent collection hands off to a STW foreground >>>>>> collection, >>>>>> (done in acquire_control_and_collect()), I expected a new GCId >>>>>> would be >>>>>> used. Did I miss it? That STW collection does not go through >>>>>> do_collection(). >>>>> >>>>> The call to acquire_control_and_collect() originates from >>>>> GenCollectedHeap::collect_generation() and there is a new GCIdMark >>>>> in there that will create a new GCId for the STW collection. >>>>> That's the "extra" GCIdMark that Per asked about in his review. >>>>> >>>>> *But* I wanted to verify that it worked properly since you asked >>>>> about it and I realized that there is another bug. >>>>> >>>>> The GCId that is set up for the concurrent cycle, the one set up >>>>> in ConcurrentMarkSweepThread::run(), will is still active over the >>>>> complete STW foreground collection. That's fine in my model since >>>>> the new GCIdMark in GenCollectedHeap::collect_generation() will >>>>> cache that GCId. But the ConcurrentMarkSweep thread is not >>>>> completely idle even though control has been left to the >>>>> foreground collection in acquire_control_and_collect(). So, there >>>>> is some logging done (by fore example CMSPhaseAccounting) that is >>>>> done at the same time as the foreground collection. This logging >>>>> will now use the foreground GCId instead of the CMS GCId. >>>>> >>>>> I haven't had time to dig in to the details of that just yet. But >>>>> this is an unintended change of the logging, so I would like to >>>>> fix it. Hopefully I'll have an updated webrev tomorrow. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> >>>>>> >>>>>> Jon >>>>>> >>>>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi again, >>>>>>> >>>>>>> I found an issue with how G1 young collections kick off >>>>>>> concurrent marks. There is a short window where we might have >>>>>>> two active GC IDs at the same time. I've fixed this and updated >>>>>>> the webrevs. In case anyone had already started looking at the >>>>>>> webrevs you need to do a refresh now. The fix is in >>>>>>> G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Per, >>>>>>>> >>>>>>>> Thanks for looking at this! >>>>>>>> >>>>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>>>> Hi Bengt, >>>>>>>>> >>>>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi everyone, >>>>>>>>>> >>>>>>>>>> This is mostly a GC related patch, but it adds a field to the >>>>>>>>>> Thread >>>>>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>>>>> >>>>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>>>> >>>>>>>>> >>>>>>>>> Looks good. I think this is a nice simplification, especially >>>>>>>>> for G1's concurrent parts. Just two comments: >>>>>>>>> >>>>>>>>> genCollectedHeap.cpp: >>>>>>>>> --------------------- >>>>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>>>>> different scopes. Do we really want that second mark? >>>>>>>> >>>>>>>> >>>>>>>> We potentially do two GCs in GenCollectedHeap::do_collection(). >>>>>>>> First a young GC and then potentially a full GC. These two >>>>>>>> should have different GC IDs. So, yes, we need two GCIdMarks in >>>>>>>> this method. The scoping could be better, but I think that is a >>>>>>>> refactoring that should be done separately from my current >>>>>>>> patch. I'll talk to Jesper about it since he has been cleaning >>>>>>>> up this code lately. >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> gcId.cpp: >>>>>>>>> -------- >>>>>>>>> - I think you might have left currentNamedthread() in there. >>>>>>>>> You probably just want to use Thread::current() instead. >>>>>>>> >>>>>>>> Yes, good catch. Thanks. >>>>>>>> >>>>>>>> However, after thinking some more about the implications of >>>>>>>> moving the GC ID out from the GCTracer I figured it would be >>>>>>>> possible to just store the previous GC ID in the GCIdMark and >>>>>>>> then restore it in the destructor of that class. That way we >>>>>>>> don't have to store anything in the Thread class but can still >>>>>>>> have both a concurrent GC ID and a STW GC ID active at the same >>>>>>>> time. This also removes the need to copy the GC ID to the >>>>>>>> worker tasks. >>>>>>>> >>>>>>>> Here's an updated webrev with a solution that does not add >>>>>>>> anything to the Thread class: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>>>> >>>>>>>> And here's a diff compared to the last version: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>>>> >>>>>>>> Unfotunately the webrev tool had some bad luck when creating >>>>>>>> the diff webrev so at least the g1CollectedHeap.cpp seem to >>>>>>>> contain the complete changes rather than just the diff compared >>>>>>>> to the 01 version. The rest of the diff looks correct as far as >>>>>>>> I can tell. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>>> >>>>>>>>> >>>>>>>>> cheers, >>>>>>>>> /Per >>>>>>>>> >>>>>>>>> >>>>>>>>>> Currently the GC ID, that is used for event tracing and >>>>>>>>>> logging, is >>>>>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>>>>> problems since >>>>>>>>>> there are multiple GCTracers active at the same time. The >>>>>>>>>> correct GC IDs >>>>>>>>>> need to be passed around in many places. >>>>>>>>>> >>>>>>>>>> For some upcoming GC logging changes I would like to have a more >>>>>>>>>> consistent way of finding the currently active GC ID. I've >>>>>>>>>> played around >>>>>>>>>> with a couple of different solutions, but eventually I found >>>>>>>>>> that the >>>>>>>>>> simplest solution is to store the current GC ID in the >>>>>>>>>> thread. That way >>>>>>>>>> there is a single way to look it up in all places. It is also >>>>>>>>>> fairly >>>>>>>>>> straight forward to set it up. >>>>>>>>>> >>>>>>>>>> I've reworked the GCId class a bit to support this and I've >>>>>>>>>> introduced a >>>>>>>>>> GCIdMark class that is a scoped object that helps to set up >>>>>>>>>> and tear >>>>>>>>>> down a current GC ID. By moving the GC ID out from the >>>>>>>>>> GCTracer class I >>>>>>>>>> got rid of a few minor issues as well. One is that we no >>>>>>>>>> longer need to >>>>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was >>>>>>>>>> necessary >>>>>>>>>> before since we did logging *after* we had told the >>>>>>>>>> corresponding >>>>>>>>>> GCTracer that the concurrent cycle was aborted and it had >>>>>>>>>> thrown its GC >>>>>>>>>> ID away. Now the GC ID can stay in scope until we have >>>>>>>>>> completed all >>>>>>>>>> logging. >>>>>>>>>> >>>>>>>>>> Also the HeapDumpBeforeFullGC and >>>>>>>>>> PrintClassHistogramBeforeFullGC used >>>>>>>>>> to have to create a new GC ID since their logging was done >>>>>>>>>> before we had >>>>>>>>>> the correct GCTracer set up. Now the GC ID can be available >>>>>>>>>> early enough >>>>>>>>>> for this logging to be using the same (and correct) GC ID as >>>>>>>>>> the rest of >>>>>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>>>>> PrintClassHistogramAfterFullGC. >>>>>>>>>> >>>>>>>>>> I've added an uint to the Thread class to keep track of the >>>>>>>>>> currently >>>>>>>>>> active GC ID. In the current code there are actually only >>>>>>>>>> NamedThreads >>>>>>>>>> that need the GC ID. So, I'm not sure where the best place is >>>>>>>>>> for this >>>>>>>>>> field is. But it seems like the Thread class contains most of >>>>>>>>>> the data, >>>>>>>>>> even data that is only used by some subclasses, so I opted >>>>>>>>>> for putting >>>>>>>>>> the field in Thread as opposed to in NamedThread. This opens >>>>>>>>>> up for >>>>>>>>>> possible future extensions when Java threads may do part of >>>>>>>>>> the GC work. >>>>>>>>>> However, I'm open to moving the field down to NamedThread if >>>>>>>>>> that is >>>>>>>>>> seen as better. >>>>>>>>>> >>>>>>>>>> In the GCTracer class there were many asserts that were >>>>>>>>>> checking that >>>>>>>>>> the GC ID was properly set up. Mostly these assert verify >>>>>>>>>> that start/end >>>>>>>>>> is called correctly and that the other methods are called >>>>>>>>>> inside of the >>>>>>>>>> start/end scope. Now that the GC ID is moved out of the >>>>>>>>>> GCTracer class >>>>>>>>>> these asserts had to be dealt with. I figured there were >>>>>>>>>> three ways to >>>>>>>>>> handle it; just remove them, replace them with check that the >>>>>>>>>> GC ID from >>>>>>>>>> the current thread is correct, or implement a new status >>>>>>>>>> field to keep >>>>>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>>>>> asserts are >>>>>>>>>> valuable enough to spend time on, so I went for the first >>>>>>>>>> approach: >>>>>>>>>> removing them. I don't think making them use the thread GC ID >>>>>>>>>> will be >>>>>>>>>> appropriate. But if anyone feels strongly about these asserts >>>>>>>>>> I can >>>>>>>>>> implement a separate start/end state. >>>>>>>>>> >>>>>>>>>> There are a few "Tasks" in the GC code that are executed by >>>>>>>>>> worker >>>>>>>>>> threads. To make the worker threads use the correct GC ID I >>>>>>>>>> make sure >>>>>>>>>> that the Task constructors copy the GC ID from the initiating >>>>>>>>>> thread >>>>>>>>>> into a local variable. When the task is executed in its >>>>>>>>>> worker thread it >>>>>>>>>> sets up the GC ID based on the local variable. >>>>>>>>>> >>>>>>>>>> The proposed change does not alter any logging (except for >>>>>>>>>> the small bug >>>>>>>>>> fix for >>>>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>>>>> This >>>>>>>>>> means that no existing tests need to be updated. In >>>>>>>>>> particular the >>>>>>>>>> test/gc/logging/TestGCId.java test passes even after these >>>>>>>>>> changes. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> A big thanks to Per, who pre-reviewed these changes and came >>>>>>>>>> with some >>>>>>>>>> really good feedback. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From jon.masamitsu at oracle.com Tue Sep 29 16:08:05 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Tue, 29 Sep 2015 09:08:05 -0700 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <560AB4FC.1050805@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> <560A4723.90604@oracle.com> <560AB551.6010507@oracle.com> <560AB4FC.1050805@oracle.com> Message-ID: <560AB765.9020605@oracle.com> On 9/29/15 8:57 AM, Bengt Rutisson wrote: > > > On 2015-09-29 17:59, Jon Masamitsu wrote: >> >> >> On 9/29/15 1:09 AM, Bengt Rutisson wrote: >>> >>> Hi Jon, >>> >>> On 2015-09-28 19:19, Jon Masamitsu wrote: >>>> >>>> >>>> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >>>>> >>>>> Hi Jon and Per, >>>>> >>>>> I've been trying to get the version proposed in webrev.02 to work >>>>> but I ran into a show stopper for this approach. The G1 concurrent >>>>> mark thread is at times executing (and logging) at the same time >>>>> as a young or mixed GC is executing. There is no good way of >>>>> handling this with a common place to store the GC ID. Instead I've >>>>> decided to go back to storing the current GC ID in the thread. >>>>> That way the G1 concurrent marking and the young/mixed GCs will >>>>> have their own GC IDs. >>>>> >>>>> Here's an updated webrev: >>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ >>>> Bengt, >>>> >>>> The _gc_id is needed in a JavaThread because a JavaThread >>>> might do work for G1 (concurrent refinement for example) and >>>> needs a GC ID? >>> >>> The Java threads currently don't log anything when they do GC work. >>> But in the future it is likely that they will. >>> >>>> But a WatcherThread would never need one, >>>> right? >>> >>> Right. Initially I only added the _gc_id field to NamedThread. But >>> Per thought it would be better to have it in Thread. Both for making >>> it possible for future improvements (such as logging GC work from >>> JavaThreads) and because it seems like we normally keep storage in >>> the Thread class even for things that are not used by all >>> subclasses. For example TLABs and allocated_bytes() are only >>> relevant for JavaThreads but have their storage in Thread. >> >> Yes, Thread could be improved but I'm not sure I can stand adding GC >> data where >> it's not needed. Unless GC logging from JavaThread is something >> that's going to >> happen soon, I'd rather add it when it's needed rather than right >> now. I'd personally >> rather duplicate the code in JavaThread if it is needed rather than >> have WatcherThreads >> have a _gc_id. > > I tend to agree. I'll discuss this with Per tomorrow. If he agrees > that we can move the _gc_id down to NamedThread, are you ok with me > pushing the fix? Yes, I'm good with the fix then. Would it be appropriate for me to file a CR to change NamedThread to something like GCThread? Or ??? Jon > > Thanks, > Bengt > >> >> Jon >> >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Jon >>>> >>>>> >>>>> This is the same as webrev.01 that both of you already looked at. >>>>> I've made two minor adjustments: >>>>> >>>>> - I removed the currentNamedthread() function in gcId.cpp that Per >>>>> pointed out in his review of webrev.01. Instead I use >>>>> Thread::current() directly. >>>>> >>>>> - I made GCIdMark a StackObj. >>>>> >>>>> Otherwise the patch is the same as in webrev.01. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> On 2015-09-10 14:37, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Jon, >>>>>> >>>>>> Thanks for looking at this! >>>>>> >>>>>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>>>>> Bengt, >>>>>>> >>>>>>> When a CMS concurrent collection hands off to a STW foreground >>>>>>> collection, >>>>>>> (done in acquire_control_and_collect()), I expected a new GCId >>>>>>> would be >>>>>>> used. Did I miss it? That STW collection does not go through >>>>>>> do_collection(). >>>>>> >>>>>> The call to acquire_control_and_collect() originates from >>>>>> GenCollectedHeap::collect_generation() and there is a new >>>>>> GCIdMark in there that will create a new GCId for the STW >>>>>> collection. That's the "extra" GCIdMark that Per asked about in >>>>>> his review. >>>>>> >>>>>> *But* I wanted to verify that it worked properly since you asked >>>>>> about it and I realized that there is another bug. >>>>>> >>>>>> The GCId that is set up for the concurrent cycle, the one set up >>>>>> in ConcurrentMarkSweepThread::run(), will is still active over >>>>>> the complete STW foreground collection. That's fine in my model >>>>>> since the new GCIdMark in GenCollectedHeap::collect_generation() >>>>>> will cache that GCId. But the ConcurrentMarkSweep thread is not >>>>>> completely idle even though control has been left to the >>>>>> foreground collection in acquire_control_and_collect(). So, there >>>>>> is some logging done (by fore example CMSPhaseAccounting) that is >>>>>> done at the same time as the foreground collection. This logging >>>>>> will now use the foreground GCId instead of the CMS GCId. >>>>>> >>>>>> I haven't had time to dig in to the details of that just yet. But >>>>>> this is an unintended change of the logging, so I would like to >>>>>> fix it. Hopefully I'll have an updated webrev tomorrow. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>> >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi again, >>>>>>>> >>>>>>>> I found an issue with how G1 young collections kick off >>>>>>>> concurrent marks. There is a short window where we might have >>>>>>>> two active GC IDs at the same time. I've fixed this and updated >>>>>>>> the webrevs. In case anyone had already started looking at the >>>>>>>> webrevs you need to do a refresh now. The fix is in >>>>>>>> G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi Per, >>>>>>>>> >>>>>>>>> Thanks for looking at this! >>>>>>>>> >>>>>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>>>>> Hi Bengt, >>>>>>>>>> >>>>>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> Hi everyone, >>>>>>>>>>> >>>>>>>>>>> This is mostly a GC related patch, but it adds a field to >>>>>>>>>>> the Thread >>>>>>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>>>>>> >>>>>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Looks good. I think this is a nice simplification, especially >>>>>>>>>> for G1's concurrent parts. Just two comments: >>>>>>>>>> >>>>>>>>>> genCollectedHeap.cpp: >>>>>>>>>> --------------------- >>>>>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>>>>>> different scopes. Do we really want that second mark? >>>>>>>>> >>>>>>>>> >>>>>>>>> We potentially do two GCs in >>>>>>>>> GenCollectedHeap::do_collection(). First a young GC and then >>>>>>>>> potentially a full GC. These two should have different GC IDs. >>>>>>>>> So, yes, we need two GCIdMarks in this method. The scoping >>>>>>>>> could be better, but I think that is a refactoring that should >>>>>>>>> be done separately from my current patch. I'll talk to Jesper >>>>>>>>> about it since he has been cleaning up this code lately. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> gcId.cpp: >>>>>>>>>> -------- >>>>>>>>>> - I think you might have left currentNamedthread() in there. >>>>>>>>>> You probably just want to use Thread::current() instead. >>>>>>>>> >>>>>>>>> Yes, good catch. Thanks. >>>>>>>>> >>>>>>>>> However, after thinking some more about the implications of >>>>>>>>> moving the GC ID out from the GCTracer I figured it would be >>>>>>>>> possible to just store the previous GC ID in the GCIdMark and >>>>>>>>> then restore it in the destructor of that class. That way we >>>>>>>>> don't have to store anything in the Thread class but can still >>>>>>>>> have both a concurrent GC ID and a STW GC ID active at the >>>>>>>>> same time. This also removes the need to copy the GC ID to the >>>>>>>>> worker tasks. >>>>>>>>> >>>>>>>>> Here's an updated webrev with a solution that does not add >>>>>>>>> anything to the Thread class: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>>>>> >>>>>>>>> And here's a diff compared to the last version: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>>>>> >>>>>>>>> Unfotunately the webrev tool had some bad luck when creating >>>>>>>>> the diff webrev so at least the g1CollectedHeap.cpp seem to >>>>>>>>> contain the complete changes rather than just the diff >>>>>>>>> compared to the 01 version. The rest of the diff looks correct >>>>>>>>> as far as I can tell. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> cheers, >>>>>>>>>> /Per >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Currently the GC ID, that is used for event tracing and >>>>>>>>>>> logging, is >>>>>>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>>>>>> problems since >>>>>>>>>>> there are multiple GCTracers active at the same time. The >>>>>>>>>>> correct GC IDs >>>>>>>>>>> need to be passed around in many places. >>>>>>>>>>> >>>>>>>>>>> For some upcoming GC logging changes I would like to have a >>>>>>>>>>> more >>>>>>>>>>> consistent way of finding the currently active GC ID. I've >>>>>>>>>>> played around >>>>>>>>>>> with a couple of different solutions, but eventually I found >>>>>>>>>>> that the >>>>>>>>>>> simplest solution is to store the current GC ID in the >>>>>>>>>>> thread. That way >>>>>>>>>>> there is a single way to look it up in all places. It is >>>>>>>>>>> also fairly >>>>>>>>>>> straight forward to set it up. >>>>>>>>>>> >>>>>>>>>>> I've reworked the GCId class a bit to support this and I've >>>>>>>>>>> introduced a >>>>>>>>>>> GCIdMark class that is a scoped object that helps to set up >>>>>>>>>>> and tear >>>>>>>>>>> down a current GC ID. By moving the GC ID out from the >>>>>>>>>>> GCTracer class I >>>>>>>>>>> got rid of a few minor issues as well. One is that we no >>>>>>>>>>> longer need to >>>>>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was >>>>>>>>>>> necessary >>>>>>>>>>> before since we did logging *after* we had told the >>>>>>>>>>> corresponding >>>>>>>>>>> GCTracer that the concurrent cycle was aborted and it had >>>>>>>>>>> thrown its GC >>>>>>>>>>> ID away. Now the GC ID can stay in scope until we have >>>>>>>>>>> completed all >>>>>>>>>>> logging. >>>>>>>>>>> >>>>>>>>>>> Also the HeapDumpBeforeFullGC and >>>>>>>>>>> PrintClassHistogramBeforeFullGC used >>>>>>>>>>> to have to create a new GC ID since their logging was done >>>>>>>>>>> before we had >>>>>>>>>>> the correct GCTracer set up. Now the GC ID can be available >>>>>>>>>>> early enough >>>>>>>>>>> for this logging to be using the same (and correct) GC ID as >>>>>>>>>>> the rest of >>>>>>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>>>>>> PrintClassHistogramAfterFullGC. >>>>>>>>>>> >>>>>>>>>>> I've added an uint to the Thread class to keep track of the >>>>>>>>>>> currently >>>>>>>>>>> active GC ID. In the current code there are actually only >>>>>>>>>>> NamedThreads >>>>>>>>>>> that need the GC ID. So, I'm not sure where the best place >>>>>>>>>>> is for this >>>>>>>>>>> field is. But it seems like the Thread class contains most >>>>>>>>>>> of the data, >>>>>>>>>>> even data that is only used by some subclasses, so I opted >>>>>>>>>>> for putting >>>>>>>>>>> the field in Thread as opposed to in NamedThread. This opens >>>>>>>>>>> up for >>>>>>>>>>> possible future extensions when Java threads may do part of >>>>>>>>>>> the GC work. >>>>>>>>>>> However, I'm open to moving the field down to NamedThread if >>>>>>>>>>> that is >>>>>>>>>>> seen as better. >>>>>>>>>>> >>>>>>>>>>> In the GCTracer class there were many asserts that were >>>>>>>>>>> checking that >>>>>>>>>>> the GC ID was properly set up. Mostly these assert verify >>>>>>>>>>> that start/end >>>>>>>>>>> is called correctly and that the other methods are called >>>>>>>>>>> inside of the >>>>>>>>>>> start/end scope. Now that the GC ID is moved out of the >>>>>>>>>>> GCTracer class >>>>>>>>>>> these asserts had to be dealt with. I figured there were >>>>>>>>>>> three ways to >>>>>>>>>>> handle it; just remove them, replace them with check that >>>>>>>>>>> the GC ID from >>>>>>>>>>> the current thread is correct, or implement a new status >>>>>>>>>>> field to keep >>>>>>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>>>>>> asserts are >>>>>>>>>>> valuable enough to spend time on, so I went for the first >>>>>>>>>>> approach: >>>>>>>>>>> removing them. I don't think making them use the thread GC >>>>>>>>>>> ID will be >>>>>>>>>>> appropriate. But if anyone feels strongly about these >>>>>>>>>>> asserts I can >>>>>>>>>>> implement a separate start/end state. >>>>>>>>>>> >>>>>>>>>>> There are a few "Tasks" in the GC code that are executed by >>>>>>>>>>> worker >>>>>>>>>>> threads. To make the worker threads use the correct GC ID I >>>>>>>>>>> make sure >>>>>>>>>>> that the Task constructors copy the GC ID from the >>>>>>>>>>> initiating thread >>>>>>>>>>> into a local variable. When the task is executed in its >>>>>>>>>>> worker thread it >>>>>>>>>>> sets up the GC ID based on the local variable. >>>>>>>>>>> >>>>>>>>>>> The proposed change does not alter any logging (except for >>>>>>>>>>> the small bug >>>>>>>>>>> fix for >>>>>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>>>>>> This >>>>>>>>>>> means that no existing tests need to be updated. In >>>>>>>>>>> particular the >>>>>>>>>>> test/gc/logging/TestGCId.java test passes even after these >>>>>>>>>>> changes. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> A big thanks to Per, who pre-reviewed these changes and came >>>>>>>>>>> with some >>>>>>>>>>> really good feedback. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Bengt >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From aph at redhat.com Tue Sep 29 17:17:10 2015 From: aph at redhat.com (Andrew Haley) Date: Tue, 29 Sep 2015 18:17:10 +0100 Subject: RFR: AARCH64: 8138575: Improve generated code for profile counters Message-ID: <560AC796.9030301@redhat.com> AArch64 generates suboptimal code for profile counters. This add xscratch2, x0, #0x218 ldr xscratch1, [xscratch2] add xscratch1, xscratch1, #0x1 str xscratch1, [xscratch2] can be this: ldr xscratch1, [x0, #0x218] add xscratch1, xscratch1, #0x1 str xscratch1, [x0, #0x218] Although this looks like a very minor improvement, the same pattern is repeated many times in C1-generated code. http://cr.openjdk.java.net/~aph/8138575/ Thanks, Andrew. From gerard.ziemski at oracle.com Tue Sep 29 18:09:06 2015 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Tue, 29 Sep 2015 13:09:06 -0500 Subject: RFR (S): JDK-8129855: -XX:+IgnoreUnrecognizedVMOptions hides out of range VM options. In-Reply-To: <5609B768.8010109@oracle.com> References: <5609B768.8010109@oracle.com> Message-ID: <560AD3C2.7090905@oracle.com> Here is a webrev1 updated with fixes based on Dmitry's feedback (ie. changes to the java test only): http://cr.openjdk.java.net/~gziemski/8129855_rev1 cheers On 09/28/2015 04:55 PM, gerard ziemski wrote: > hi all, > > We are fixing how IgnoreUnrecognizedVMOptions treats those flags whose values are out of range. Before the fix, the VM > would continue even if flag?s value was out of range, ex: > > java_old -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version > size_t MinTLABSize=0 is outside the allowed range [ 1 ... 4294967295 ] > java version "1.9.0-internal-fastdebug" > Java(TM) SE Runtime Environment (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00) > Java HotSpot(TM) Server VM (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00, mixed mode) > > now, we correctly exit the VM with an error, ex: > > java_new -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version > size_t MinTLABSize=0 is outside the allowed range [ 1 ... 18446744073709551615 ] > Improperly specified VM option 'MinTLABSize=0' > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > In addition IgnoreUnrecognizedVMOptions nows strictly follows the spec as outlined in JDK-8129855. The behavior changes > depending on whether the build is product or debug. > > We also introduce a new test that verifies all known use cases that we care about. > > References: > bugid: https://bugs.openjdk.java.net/browse/JDK-8129855 > webrev: http://cr.openjdk.java.net/~gziemski/8129855_rev0 > discussion: http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-June/019213.html > > Passes "JPRT hotspot" and "RBT testlist,noncolo.testlist quick" > > > cheers > From ioi.lam at oracle.com Tue Sep 29 19:18:28 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 29 Sep 2015 12:18:28 -0700 Subject: RFR (XS) 8078295 - hotspot test_env.sh can set VM_CPU incorrectly Message-ID: <560AE404.4050503@oracle.com> Please review a very small fix: http://cr.openjdk.java.net/~iklam/8078295_test_env_sh.01/ Bug: [TESTBUG] hotspot test_env.sh can set VM_CPU incorrectly https://bugs.openjdk.java.net/browse/JDK-8078295 Summary of fix: test_env.sh uses the following to get information such as VM_CPU ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion > vm_version.out 2>&1 VM_CPU="unknown" grep "arm" vm_version.out > ${NULL} if [ $? = 0 ] then VM_CPU="arm" fi However, the internal version may contain arbitrary text (from the user name and pwd) such as "arm", which would cause VM_CPU to have incorrect value. The fix is to filter out the arbitrary text using a sed script (please see the bug report for details and sample output): ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion \ | sed -e 's/[(][^)]*[)]//g' -e 's/ by "[^"]*"//g' \ > vm_version.out 2>&1 Tests: JPRT RBT, with the following tests hotspot/test/:hotspot_all vm.parallel_class_loading.testlist vm.regression.testlist vm.runtime.testlist vm.tmtools.testlist) Thanks - Ioi From christian.thalinger at oracle.com Tue Sep 29 23:11:39 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 29 Sep 2015 13:11:39 -1000 Subject: RFR: JDK-5108778 Too many instances of java.lang.Boolean created in Java application In-Reply-To: <5608201C.4020109@gmx.de> References: <5608201C.4020109@gmx.de> Message-ID: Thanks for volunteering to look into such old bugs! Is this the only occurrence of that pattern in the hotspot repository? If you want to fix this in hotspot only without the jdk changes described in the RFE (I just changed the type from Bug to Enhancement) then we have to file a new RFE because we can?t close the original one. > On Sep 27, 2015, at 6:58 AM, Sebastian Sickelmann wrote: > > Hello, > > my name is Sebastian Sickelmann and i signed the OCA. > > Actually I am searching through the JBS for low hanging fruits. > Right now i am looking through the openjdk-sources and try to evaluate > if i can make something about JDK-5108778. > > As I am not an author, I am actually not able to host webrevs on > cr.openjdk.java.net. > > Is there someone who would support at hosting the hotspot-part of > JDK-5108778 on cr.openjdk.java.net for reviewing? > > I placed the hotspot part in my dropbox at: > https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev/index.html > or as zip: > https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev.zip > > I executed all jtreg tests in hotspot/test and get no more errors than before the change. > > -- Sebastian > From sebastian.sickelmann at gmx.de Wed Sep 30 03:38:33 2015 From: sebastian.sickelmann at gmx.de (Sebastian Sickelmann) Date: Wed, 30 Sep 2015 05:38:33 +0200 Subject: RFR: JDK-5108778 Too many instances of java.lang.Boolean created in Java application In-Reply-To: References: <5608201C.4020109@gmx.de> Message-ID: <560B5939.6010100@gmx.de> Yes it is the only (non-test) source i could find in hotspot, but i want to change it in all openjdk sources i can find it. I thought i really must discuss it part by part in the mailing-lists. Actually i am working on the issue to save against regression on this. Thanks Alexandr for this input. So there will be something that integrates into jtreg for this too. -- Sebastian On 09/30/2015 01:11 AM, Christian Thalinger wrote: > Thanks for volunteering to look into such old bugs! > > Is this the only occurrence of that pattern in the hotspot repository? If you want to fix this in hotspot only without the jdk changes described in the RFE (I just changed the type from Bug to Enhancement) then we have to file a new RFE because we can?t close the original one. > >> On Sep 27, 2015, at 6:58 AM, Sebastian Sickelmann wrote: >> >> Hello, >> >> my name is Sebastian Sickelmann and i signed the OCA. >> >> Actually I am searching through the JBS for low hanging fruits. >> Right now i am looking through the openjdk-sources and try to evaluate >> if i can make something about JDK-5108778. >> >> As I am not an author, I am actually not able to host webrevs on >> cr.openjdk.java.net. >> >> Is there someone who would support at hosting the hotspot-part of >> JDK-5108778 on cr.openjdk.java.net for reviewing? >> >> I placed the hotspot part in my dropbox at: >> https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev/index.html >> or as zip: >> https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev.zip >> >> I executed all jtreg tests in hotspot/test and get no more errors than before the change. >> >> -- Sebastian >> > From christian.thalinger at oracle.com Wed Sep 30 05:50:52 2015 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Tue, 29 Sep 2015 19:50:52 -1000 Subject: RFR: JDK-5108778 Too many instances of java.lang.Boolean created in Java application In-Reply-To: <560B5939.6010100@gmx.de> References: <5608201C.4020109@gmx.de> <560B5939.6010100@gmx.de> Message-ID: <118235E4-AD96-4666-8A63-58FB403324BC@oracle.com> Sounds good. If you have the final patch, let us know. > On Sep 29, 2015, at 5:38 PM, Sebastian Sickelmann wrote: > > Yes it is the only (non-test) source i could find in hotspot, but i want > to change it in all openjdk sources i can find it. > I thought i really must discuss it part by part in the mailing-lists. > Actually i am working on the issue to save against regression on this. > Thanks Alexandr for this input. So there will be something that > integrates into jtreg for this too. > > -- Sebastian > > > On 09/30/2015 01:11 AM, Christian Thalinger wrote: >> Thanks for volunteering to look into such old bugs! >> >> Is this the only occurrence of that pattern in the hotspot repository? If you want to fix this in hotspot only without the jdk changes described in the RFE (I just changed the type from Bug to Enhancement) then we have to file a new RFE because we can?t close the original one. >> >>> On Sep 27, 2015, at 6:58 AM, Sebastian Sickelmann wrote: >>> >>> Hello, >>> >>> my name is Sebastian Sickelmann and i signed the OCA. >>> >>> Actually I am searching through the JBS for low hanging fruits. >>> Right now i am looking through the openjdk-sources and try to evaluate >>> if i can make something about JDK-5108778. >>> >>> As I am not an author, I am actually not able to host webrevs on >>> cr.openjdk.java.net. >>> >>> Is there someone who would support at hosting the hotspot-part of >>> JDK-5108778 on cr.openjdk.java.net for reviewing? >>> >>> I placed the hotspot part in my dropbox at: >>> https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev/index.html >>> or as zip: >>> https://dl.dropboxusercontent.com/u/43692695/oss-patches/openjdk/jdk-5108778/hotspot_0/webrev.zip >>> >>> I executed all jtreg tests in hotspot/test and get no more errors than before the change. >>> >>> -- Sebastian >>> >> > From kirk.pepperdine at gmail.com Wed Sep 30 06:59:25 2015 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Wed, 30 Sep 2015 07:59:25 +0100 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <560AB4FC.1050805@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> <560A4723.90604@oracle.com> <560AB551.6010507@oracle.com> <560AB4FC.1050805@oracle.com> Message-ID: <5332C52E-0DE2-4937-94F2-629DBD6D3CEB@gmail.com> Hi all, My 2 cents.. Is gc_id only needed for gc logging? If so I would think that it?s only needed if gc log entries from gc event get entangled with another gc event of the same type. I?ve never seen this happen. Further more, the log entries progress in such a way that you always know when one event has ended and another begins without the need for hints in the log. If I ever needed to know which GC event I was looking at, I could simply count them and I?ve never ever needed to know the number of the GC event I was looking at. So, unless there is an internal debugging need to id the GC event, I don?t really see this as a requirement which suggests that gc id could be dropped from the GC log without loss and without conflating concerns. Kind regards, Kirk > On Sep 29, 2015, at 4:57 PM, Bengt Rutisson wrote: > > > > On 2015-09-29 17:59, Jon Masamitsu wrote: >> >> >> On 9/29/15 1:09 AM, Bengt Rutisson wrote: >>> >>> Hi Jon, >>> >>> On 2015-09-28 19:19, Jon Masamitsu wrote: >>>> >>>> >>>> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >>>>> >>>>> Hi Jon and Per, >>>>> >>>>> I've been trying to get the version proposed in webrev.02 to work but I ran into a show stopper for this approach. The G1 concurrent mark thread is at times executing (and logging) at the same time as a young or mixed GC is executing. There is no good way of handling this with a common place to store the GC ID. Instead I've decided to go back to storing the current GC ID in the thread. That way the G1 concurrent marking and the young/mixed GCs will have their own GC IDs. >>>>> >>>>> Here's an updated webrev: >>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ >>>> Bengt, >>>> >>>> The _gc_id is needed in a JavaThread because a JavaThread >>>> might do work for G1 (concurrent refinement for example) and >>>> needs a GC ID? >>> >>> The Java threads currently don't log anything when they do GC work. But in the future it is likely that they will. >>> >>>> But a WatcherThread would never need one, >>>> right? >>> >>> Right. Initially I only added the _gc_id field to NamedThread. But Per thought it would be better to have it in Thread. Both for making it possible for future improvements (such as logging GC work from JavaThreads) and because it seems like we normally keep storage in the Thread class even for things that are not used by all subclasses. For example TLABs and allocated_bytes() are only relevant for JavaThreads but have their storage in Thread. >> >> Yes, Thread could be improved but I'm not sure I can stand adding GC data where >> it's not needed. Unless GC logging from JavaThread is something that's going to >> happen soon, I'd rather add it when it's needed rather than right now. I'd personally >> rather duplicate the code in JavaThread if it is needed rather than have WatcherThreads >> have a _gc_id. > > I tend to agree. I'll discuss this with Per tomorrow. If he agrees that we can move the _gc_id down to NamedThread, are you ok with me pushing the fix? > > Thanks, > Bengt > >> >> Jon >> >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Jon >>>> >>>>> >>>>> This is the same as webrev.01 that both of you already looked at. I've made two minor adjustments: >>>>> >>>>> - I removed the currentNamedthread() function in gcId.cpp that Per pointed out in his review of webrev.01. Instead I use Thread::current() directly. >>>>> >>>>> - I made GCIdMark a StackObj. >>>>> >>>>> Otherwise the patch is the same as in webrev.01. >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>> On 2015-09-10 14:37, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Jon, >>>>>> >>>>>> Thanks for looking at this! >>>>>> >>>>>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>>>>> Bengt, >>>>>>> >>>>>>> When a CMS concurrent collection hands off to a STW foreground collection, >>>>>>> (done in acquire_control_and_collect()), I expected a new GCId would be >>>>>>> used. Did I miss it? That STW collection does not go through do_collection(). >>>>>> >>>>>> The call to acquire_control_and_collect() originates from GenCollectedHeap::collect_generation() and there is a new GCIdMark in there that will create a new GCId for the STW collection. That's the "extra" GCIdMark that Per asked about in his review. >>>>>> >>>>>> *But* I wanted to verify that it worked properly since you asked about it and I realized that there is another bug. >>>>>> >>>>>> The GCId that is set up for the concurrent cycle, the one set up in ConcurrentMarkSweepThread::run(), will is still active over the complete STW foreground collection. That's fine in my model since the new GCIdMark in GenCollectedHeap::collect_generation() will cache that GCId. But the ConcurrentMarkSweep thread is not completely idle even though control has been left to the foreground collection in acquire_control_and_collect(). So, there is some logging done (by fore example CMSPhaseAccounting) that is done at the same time as the foreground collection. This logging will now use the foreground GCId instead of the CMS GCId. >>>>>> >>>>>> I haven't had time to dig in to the details of that just yet. But this is an unintended change of the logging, so I would like to fix it. Hopefully I'll have an updated webrev tomorrow. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>> >>>>>>> >>>>>>> Jon >>>>>>> >>>>>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi again, >>>>>>>> >>>>>>>> I found an issue with how G1 young collections kick off concurrent marks. There is a short window where we might have two active GC IDs at the same time. I've fixed this and updated the webrevs. In case anyone had already started looking at the webrevs you need to do a refresh now. The fix is in G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi Per, >>>>>>>>> >>>>>>>>> Thanks for looking at this! >>>>>>>>> >>>>>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>>>>> Hi Bengt, >>>>>>>>>> >>>>>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> Hi everyone, >>>>>>>>>>> >>>>>>>>>>> This is mostly a GC related patch, but it adds a field to the Thread >>>>>>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>>>>>> >>>>>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Looks good. I think this is a nice simplification, especially for G1's concurrent parts. Just two comments: >>>>>>>>>> >>>>>>>>>> genCollectedHeap.cpp: >>>>>>>>>> --------------------- >>>>>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in different scopes. Do we really want that second mark? >>>>>>>>> >>>>>>>>> >>>>>>>>> We potentially do two GCs in GenCollectedHeap::do_collection(). First a young GC and then potentially a full GC. These two should have different GC IDs. So, yes, we need two GCIdMarks in this method. The scoping could be better, but I think that is a refactoring that should be done separately from my current patch. I'll talk to Jesper about it since he has been cleaning up this code lately. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> gcId.cpp: >>>>>>>>>> -------- >>>>>>>>>> - I think you might have left currentNamedthread() in there. You probably just want to use Thread::current() instead. >>>>>>>>> >>>>>>>>> Yes, good catch. Thanks. >>>>>>>>> >>>>>>>>> However, after thinking some more about the implications of moving the GC ID out from the GCTracer I figured it would be possible to just store the previous GC ID in the GCIdMark and then restore it in the destructor of that class. That way we don't have to store anything in the Thread class but can still have both a concurrent GC ID and a STW GC ID active at the same time. This also removes the need to copy the GC ID to the worker tasks. >>>>>>>>> >>>>>>>>> Here's an updated webrev with a solution that does not add anything to the Thread class: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>>>>> >>>>>>>>> And here's a diff compared to the last version: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>>>>> >>>>>>>>> Unfotunately the webrev tool had some bad luck when creating the diff webrev so at least the g1CollectedHeap.cpp seem to contain the complete changes rather than just the diff compared to the 01 version. The rest of the diff looks correct as far as I can tell. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> cheers, >>>>>>>>>> /Per >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Currently the GC ID, that is used for event tracing and logging, is >>>>>>>>>>> stored in the GCTracer object. That has caused some minor problems since >>>>>>>>>>> there are multiple GCTracers active at the same time. The correct GC IDs >>>>>>>>>>> need to be passed around in many places. >>>>>>>>>>> >>>>>>>>>>> For some upcoming GC logging changes I would like to have a more >>>>>>>>>>> consistent way of finding the currently active GC ID. I've played around >>>>>>>>>>> with a couple of different solutions, but eventually I found that the >>>>>>>>>>> simplest solution is to store the current GC ID in the thread. That way >>>>>>>>>>> there is a single way to look it up in all places. It is also fairly >>>>>>>>>>> straight forward to set it up. >>>>>>>>>>> >>>>>>>>>>> I've reworked the GCId class a bit to support this and I've introduced a >>>>>>>>>>> GCIdMark class that is a scoped object that helps to set up and tear >>>>>>>>>>> down a current GC ID. By moving the GC ID out from the GCTracer class I >>>>>>>>>>> got rid of a few minor issues as well. One is that we no longer need to >>>>>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>>>>>>>>>> before since we did logging *after* we had told the corresponding >>>>>>>>>>> GCTracer that the concurrent cycle was aborted and it had thrown its GC >>>>>>>>>>> ID away. Now the GC ID can stay in scope until we have completed all >>>>>>>>>>> logging. >>>>>>>>>>> >>>>>>>>>>> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC used >>>>>>>>>>> to have to create a new GC ID since their logging was done before we had >>>>>>>>>>> the correct GCTracer set up. Now the GC ID can be available early enough >>>>>>>>>>> for this logging to be using the same (and correct) GC ID as the rest of >>>>>>>>>>> the GC. Same for HeapDumpAfterFullGC and PrintClassHistogramAfterFullGC. >>>>>>>>>>> >>>>>>>>>>> I've added an uint to the Thread class to keep track of the currently >>>>>>>>>>> active GC ID. In the current code there are actually only NamedThreads >>>>>>>>>>> that need the GC ID. So, I'm not sure where the best place is for this >>>>>>>>>>> field is. But it seems like the Thread class contains most of the data, >>>>>>>>>>> even data that is only used by some subclasses, so I opted for putting >>>>>>>>>>> the field in Thread as opposed to in NamedThread. This opens up for >>>>>>>>>>> possible future extensions when Java threads may do part of the GC work. >>>>>>>>>>> However, I'm open to moving the field down to NamedThread if that is >>>>>>>>>>> seen as better. >>>>>>>>>>> >>>>>>>>>>> In the GCTracer class there were many asserts that were checking that >>>>>>>>>>> the GC ID was properly set up. Mostly these assert verify that start/end >>>>>>>>>>> is called correctly and that the other methods are called inside of the >>>>>>>>>>> start/end scope. Now that the GC ID is moved out of the GCTracer class >>>>>>>>>>> these asserts had to be dealt with. I figured there were three ways to >>>>>>>>>>> handle it; just remove them, replace them with check that the GC ID from >>>>>>>>>>> the current thread is correct, or implement a new status field to keep >>>>>>>>>>> track of GC start/end state. Personally I'm not sure these asserts are >>>>>>>>>>> valuable enough to spend time on, so I went for the first approach: >>>>>>>>>>> removing them. I don't think making them use the thread GC ID will be >>>>>>>>>>> appropriate. But if anyone feels strongly about these asserts I can >>>>>>>>>>> implement a separate start/end state. >>>>>>>>>>> >>>>>>>>>>> There are a few "Tasks" in the GC code that are executed by worker >>>>>>>>>>> threads. To make the worker threads use the correct GC ID I make sure >>>>>>>>>>> that the Task constructors copy the GC ID from the initiating thread >>>>>>>>>>> into a local variable. When the task is executed in its worker thread it >>>>>>>>>>> sets up the GC ID based on the local variable. >>>>>>>>>>> >>>>>>>>>>> The proposed change does not alter any logging (except for the small bug >>>>>>>>>>> fix for >>>>>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. This >>>>>>>>>>> means that no existing tests need to be updated. In particular the >>>>>>>>>>> test/gc/logging/TestGCId.java test passes even after these changes. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> A big thanks to Per, who pre-reviewed these changes and came with some >>>>>>>>>>> really good feedback. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Bengt >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From bengt.rutisson at oracle.com Wed Sep 30 07:20:33 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 30 Sep 2015 09:20:33 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <560AB765.9020605@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> <560A4723.90604@oracle.com> <560AB551.6010507@oracle.com> <560AB4FC.1050805@oracle.com> <560AB765.9020605@oracle.com> Message-ID: <560B8D41.10107@oracle.com> On 2015-09-29 18:08, Jon Masamitsu wrote: > > > On 9/29/15 8:57 AM, Bengt Rutisson wrote: >> >> >> On 2015-09-29 17:59, Jon Masamitsu wrote: >>> >>> >>> On 9/29/15 1:09 AM, Bengt Rutisson wrote: >>>> >>>> Hi Jon, >>>> >>>> On 2015-09-28 19:19, Jon Masamitsu wrote: >>>>> >>>>> >>>>> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >>>>>> >>>>>> Hi Jon and Per, >>>>>> >>>>>> I've been trying to get the version proposed in webrev.02 to work >>>>>> but I ran into a show stopper for this approach. The G1 >>>>>> concurrent mark thread is at times executing (and logging) at the >>>>>> same time as a young or mixed GC is executing. There is no good >>>>>> way of handling this with a common place to store the GC ID. >>>>>> Instead I've decided to go back to storing the current GC ID in >>>>>> the thread. That way the G1 concurrent marking and the >>>>>> young/mixed GCs will have their own GC IDs. >>>>>> >>>>>> Here's an updated webrev: >>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ >>>>> Bengt, >>>>> >>>>> The _gc_id is needed in a JavaThread because a JavaThread >>>>> might do work for G1 (concurrent refinement for example) and >>>>> needs a GC ID? >>>> >>>> The Java threads currently don't log anything when they do GC work. >>>> But in the future it is likely that they will. >>>> >>>>> But a WatcherThread would never need one, >>>>> right? >>>> >>>> Right. Initially I only added the _gc_id field to NamedThread. But >>>> Per thought it would be better to have it in Thread. Both for >>>> making it possible for future improvements (such as logging GC work >>>> from JavaThreads) and because it seems like we normally keep >>>> storage in the Thread class even for things that are not used by >>>> all subclasses. For example TLABs and allocated_bytes() are only >>>> relevant for JavaThreads but have their storage in Thread. >>> >>> Yes, Thread could be improved but I'm not sure I can stand adding GC >>> data where >>> it's not needed. Unless GC logging from JavaThread is something >>> that's going to >>> happen soon, I'd rather add it when it's needed rather than right >>> now. I'd personally >>> rather duplicate the code in JavaThread if it is needed rather than >>> have WatcherThreads >>> have a _gc_id. >> >> I tend to agree. I'll discuss this with Per tomorrow. If he agrees >> that we can move the _gc_id down to NamedThread, are you ok with me >> pushing the fix? > > Yes, I'm good with the fix then. Thanks Jon! I talked to Per and moved the _gc_id field down to NamedThread. Pushing it now. > > Would it be appropriate for me to file a CR to change NamedThread to > something > like GCThread? Or ??? I don't think GCThread is the right name since VMThread is a NamedThread and it can schedule tasks that are not GC related. I agree that NamedThread is not a great name. But I actually think that the naming in the Thread hierarchy is just part of the problem. The real problem is that the division of responsibility is not really correctly partitioned among the classes. If we should file a RFE I think it should be to clean up the Thread hierarchy, including finding good names for the classes. Bengt > > Jon > >> >> Thanks, >> Bengt >> >>> >>> Jon >>> >>>> >>>> Thanks, >>>> Bengt >>>> >>>>> >>>>> Jon >>>>> >>>>>> >>>>>> This is the same as webrev.01 that both of you already looked at. >>>>>> I've made two minor adjustments: >>>>>> >>>>>> - I removed the currentNamedthread() function in gcId.cpp that >>>>>> Per pointed out in his review of webrev.01. Instead I use >>>>>> Thread::current() directly. >>>>>> >>>>>> - I made GCIdMark a StackObj. >>>>>> >>>>>> Otherwise the patch is the same as in webrev.01. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>> On 2015-09-10 14:37, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Jon, >>>>>>> >>>>>>> Thanks for looking at this! >>>>>>> >>>>>>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>>>>>> Bengt, >>>>>>>> >>>>>>>> When a CMS concurrent collection hands off to a STW foreground >>>>>>>> collection, >>>>>>>> (done in acquire_control_and_collect()), I expected a new GCId >>>>>>>> would be >>>>>>>> used. Did I miss it? That STW collection does not go through >>>>>>>> do_collection(). >>>>>>> >>>>>>> The call to acquire_control_and_collect() originates from >>>>>>> GenCollectedHeap::collect_generation() and there is a new >>>>>>> GCIdMark in there that will create a new GCId for the STW >>>>>>> collection. That's the "extra" GCIdMark that Per asked about in >>>>>>> his review. >>>>>>> >>>>>>> *But* I wanted to verify that it worked properly since you asked >>>>>>> about it and I realized that there is another bug. >>>>>>> >>>>>>> The GCId that is set up for the concurrent cycle, the one set up >>>>>>> in ConcurrentMarkSweepThread::run(), will is still active over >>>>>>> the complete STW foreground collection. That's fine in my model >>>>>>> since the new GCIdMark in GenCollectedHeap::collect_generation() >>>>>>> will cache that GCId. But the ConcurrentMarkSweep thread is not >>>>>>> completely idle even though control has been left to the >>>>>>> foreground collection in acquire_control_and_collect(). So, >>>>>>> there is some logging done (by fore example CMSPhaseAccounting) >>>>>>> that is done at the same time as the foreground collection. This >>>>>>> logging will now use the foreground GCId instead of the CMS GCId. >>>>>>> >>>>>>> I haven't had time to dig in to the details of that just yet. >>>>>>> But this is an unintended change of the logging, so I would like >>>>>>> to fix it. Hopefully I'll have an updated webrev tomorrow. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Jon >>>>>>>> >>>>>>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>>>>>> >>>>>>>>> Hi again, >>>>>>>>> >>>>>>>>> I found an issue with how G1 young collections kick off >>>>>>>>> concurrent marks. There is a short window where we might have >>>>>>>>> two active GC IDs at the same time. I've fixed this and >>>>>>>>> updated the webrevs. In case anyone had already started >>>>>>>>> looking at the webrevs you need to do a refresh now. The fix >>>>>>>>> is in G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi Per, >>>>>>>>>> >>>>>>>>>> Thanks for looking at this! >>>>>>>>>> >>>>>>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi everyone, >>>>>>>>>>>> >>>>>>>>>>>> This is mostly a GC related patch, but it adds a field to >>>>>>>>>>>> the Thread >>>>>>>>>>>> class, so I'm sending this out on the broader hotspot-dev >>>>>>>>>>>> list. >>>>>>>>>>>> >>>>>>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Looks good. I think this is a nice simplification, >>>>>>>>>>> especially for G1's concurrent parts. Just two comments: >>>>>>>>>>> >>>>>>>>>>> genCollectedHeap.cpp: >>>>>>>>>>> --------------------- >>>>>>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>>>>>>> different scopes. Do we really want that second mark? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> We potentially do two GCs in >>>>>>>>>> GenCollectedHeap::do_collection(). First a young GC and then >>>>>>>>>> potentially a full GC. These two should have different GC >>>>>>>>>> IDs. So, yes, we need two GCIdMarks in this method. The >>>>>>>>>> scoping could be better, but I think that is a refactoring >>>>>>>>>> that should be done separately from my current patch. I'll >>>>>>>>>> talk to Jesper about it since he has been cleaning up this >>>>>>>>>> code lately. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> gcId.cpp: >>>>>>>>>>> -------- >>>>>>>>>>> - I think you might have left currentNamedthread() in there. >>>>>>>>>>> You probably just want to use Thread::current() instead. >>>>>>>>>> >>>>>>>>>> Yes, good catch. Thanks. >>>>>>>>>> >>>>>>>>>> However, after thinking some more about the implications of >>>>>>>>>> moving the GC ID out from the GCTracer I figured it would be >>>>>>>>>> possible to just store the previous GC ID in the GCIdMark and >>>>>>>>>> then restore it in the destructor of that class. That way we >>>>>>>>>> don't have to store anything in the Thread class but can >>>>>>>>>> still have both a concurrent GC ID and a STW GC ID active at >>>>>>>>>> the same time. This also removes the need to copy the GC ID >>>>>>>>>> to the worker tasks. >>>>>>>>>> >>>>>>>>>> Here's an updated webrev with a solution that does not add >>>>>>>>>> anything to the Thread class: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>>>>>> >>>>>>>>>> And here's a diff compared to the last version: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>>>>>> >>>>>>>>>> Unfotunately the webrev tool had some bad luck when creating >>>>>>>>>> the diff webrev so at least the g1CollectedHeap.cpp seem to >>>>>>>>>> contain the complete changes rather than just the diff >>>>>>>>>> compared to the 01 version. The rest of the diff looks >>>>>>>>>> correct as far as I can tell. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> cheers, >>>>>>>>>>> /Per >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Currently the GC ID, that is used for event tracing and >>>>>>>>>>>> logging, is >>>>>>>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>>>>>>> problems since >>>>>>>>>>>> there are multiple GCTracers active at the same time. The >>>>>>>>>>>> correct GC IDs >>>>>>>>>>>> need to be passed around in many places. >>>>>>>>>>>> >>>>>>>>>>>> For some upcoming GC logging changes I would like to have a >>>>>>>>>>>> more >>>>>>>>>>>> consistent way of finding the currently active GC ID. I've >>>>>>>>>>>> played around >>>>>>>>>>>> with a couple of different solutions, but eventually I >>>>>>>>>>>> found that the >>>>>>>>>>>> simplest solution is to store the current GC ID in the >>>>>>>>>>>> thread. That way >>>>>>>>>>>> there is a single way to look it up in all places. It is >>>>>>>>>>>> also fairly >>>>>>>>>>>> straight forward to set it up. >>>>>>>>>>>> >>>>>>>>>>>> I've reworked the GCId class a bit to support this and I've >>>>>>>>>>>> introduced a >>>>>>>>>>>> GCIdMark class that is a scoped object that helps to set up >>>>>>>>>>>> and tear >>>>>>>>>>>> down a current GC ID. By moving the GC ID out from the >>>>>>>>>>>> GCTracer class I >>>>>>>>>>>> got rid of a few minor issues as well. One is that we no >>>>>>>>>>>> longer need to >>>>>>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was >>>>>>>>>>>> necessary >>>>>>>>>>>> before since we did logging *after* we had told the >>>>>>>>>>>> corresponding >>>>>>>>>>>> GCTracer that the concurrent cycle was aborted and it had >>>>>>>>>>>> thrown its GC >>>>>>>>>>>> ID away. Now the GC ID can stay in scope until we have >>>>>>>>>>>> completed all >>>>>>>>>>>> logging. >>>>>>>>>>>> >>>>>>>>>>>> Also the HeapDumpBeforeFullGC and >>>>>>>>>>>> PrintClassHistogramBeforeFullGC used >>>>>>>>>>>> to have to create a new GC ID since their logging was done >>>>>>>>>>>> before we had >>>>>>>>>>>> the correct GCTracer set up. Now the GC ID can be available >>>>>>>>>>>> early enough >>>>>>>>>>>> for this logging to be using the same (and correct) GC ID >>>>>>>>>>>> as the rest of >>>>>>>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>>>>>>> PrintClassHistogramAfterFullGC. >>>>>>>>>>>> >>>>>>>>>>>> I've added an uint to the Thread class to keep track of the >>>>>>>>>>>> currently >>>>>>>>>>>> active GC ID. In the current code there are actually only >>>>>>>>>>>> NamedThreads >>>>>>>>>>>> that need the GC ID. So, I'm not sure where the best place >>>>>>>>>>>> is for this >>>>>>>>>>>> field is. But it seems like the Thread class contains most >>>>>>>>>>>> of the data, >>>>>>>>>>>> even data that is only used by some subclasses, so I opted >>>>>>>>>>>> for putting >>>>>>>>>>>> the field in Thread as opposed to in NamedThread. This >>>>>>>>>>>> opens up for >>>>>>>>>>>> possible future extensions when Java threads may do part of >>>>>>>>>>>> the GC work. >>>>>>>>>>>> However, I'm open to moving the field down to NamedThread >>>>>>>>>>>> if that is >>>>>>>>>>>> seen as better. >>>>>>>>>>>> >>>>>>>>>>>> In the GCTracer class there were many asserts that were >>>>>>>>>>>> checking that >>>>>>>>>>>> the GC ID was properly set up. Mostly these assert verify >>>>>>>>>>>> that start/end >>>>>>>>>>>> is called correctly and that the other methods are called >>>>>>>>>>>> inside of the >>>>>>>>>>>> start/end scope. Now that the GC ID is moved out of the >>>>>>>>>>>> GCTracer class >>>>>>>>>>>> these asserts had to be dealt with. I figured there were >>>>>>>>>>>> three ways to >>>>>>>>>>>> handle it; just remove them, replace them with check that >>>>>>>>>>>> the GC ID from >>>>>>>>>>>> the current thread is correct, or implement a new status >>>>>>>>>>>> field to keep >>>>>>>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>>>>>>> asserts are >>>>>>>>>>>> valuable enough to spend time on, so I went for the first >>>>>>>>>>>> approach: >>>>>>>>>>>> removing them. I don't think making them use the thread GC >>>>>>>>>>>> ID will be >>>>>>>>>>>> appropriate. But if anyone feels strongly about these >>>>>>>>>>>> asserts I can >>>>>>>>>>>> implement a separate start/end state. >>>>>>>>>>>> >>>>>>>>>>>> There are a few "Tasks" in the GC code that are executed by >>>>>>>>>>>> worker >>>>>>>>>>>> threads. To make the worker threads use the correct GC ID I >>>>>>>>>>>> make sure >>>>>>>>>>>> that the Task constructors copy the GC ID from the >>>>>>>>>>>> initiating thread >>>>>>>>>>>> into a local variable. When the task is executed in its >>>>>>>>>>>> worker thread it >>>>>>>>>>>> sets up the GC ID based on the local variable. >>>>>>>>>>>> >>>>>>>>>>>> The proposed change does not alter any logging (except for >>>>>>>>>>>> the small bug >>>>>>>>>>>> fix for >>>>>>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>>>>>>> This >>>>>>>>>>>> means that no existing tests need to be updated. In >>>>>>>>>>>> particular the >>>>>>>>>>>> test/gc/logging/TestGCId.java test passes even after these >>>>>>>>>>>> changes. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> A big thanks to Per, who pre-reviewed these changes and >>>>>>>>>>>> came with some >>>>>>>>>>>> really good feedback. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From bengt.rutisson at oracle.com Wed Sep 30 07:22:39 2015 From: bengt.rutisson at oracle.com (Bengt Rutisson) Date: Wed, 30 Sep 2015 09:22:39 +0200 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <5332C52E-0DE2-4937-94F2-629DBD6D3CEB@gmail.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> <560A4723.90604@oracle.com> <560AB551.6010507@oracle.com> <560AB4FC.1050805@oracle.com> <5332C52E-0DE2-4937-94F2-629DBD6D3CEB@gmail.com> Message-ID: <560B8DBF.2040203@oracle.com> Hi Kirk, On 2015-09-30 08:59, Kirk Pepperdine wrote: > Hi all, > > My 2 cents.. > > Is gc_id only needed for gc logging? If so I would think that it?s only needed if gc log entries from gc event get entangled with another gc event of the same type. I?ve never seen this happen. Further more, the log entries progress in such a way that you always know when one event has ended and another begins without the need for hints in the log. If I ever needed to know which GC event I was looking at, I could simply count them and I?ve never ever needed to know the number of the GC event I was looking at. So, unless there is an internal debugging need to id the GC event, I don?t really see this as a requirement which suggests that gc id could be dropped from the GC log without loss and without conflating concerns. The event tracing uses the GC id to group events that belong together. We also find it useful to have the GC id on all the entires in the log files to group log entires for a particular GC together. In particular when concurrent GCs are being used. Thanks, Bengt > > Kind regards, > Kirk >> On Sep 29, 2015, at 4:57 PM, Bengt Rutisson wrote: >> >> >> >> On 2015-09-29 17:59, Jon Masamitsu wrote: >>> >>> On 9/29/15 1:09 AM, Bengt Rutisson wrote: >>>> Hi Jon, >>>> >>>> On 2015-09-28 19:19, Jon Masamitsu wrote: >>>>> >>>>> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >>>>>> Hi Jon and Per, >>>>>> >>>>>> I've been trying to get the version proposed in webrev.02 to work but I ran into a show stopper for this approach. The G1 concurrent mark thread is at times executing (and logging) at the same time as a young or mixed GC is executing. There is no good way of handling this with a common place to store the GC ID. Instead I've decided to go back to storing the current GC ID in the thread. That way the G1 concurrent marking and the young/mixed GCs will have their own GC IDs. >>>>>> >>>>>> Here's an updated webrev: >>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ >>>>> Bengt, >>>>> >>>>> The _gc_id is needed in a JavaThread because a JavaThread >>>>> might do work for G1 (concurrent refinement for example) and >>>>> needs a GC ID? >>>> The Java threads currently don't log anything when they do GC work. But in the future it is likely that they will. >>>> >>>>> But a WatcherThread would never need one, >>>>> right? >>>> Right. Initially I only added the _gc_id field to NamedThread. But Per thought it would be better to have it in Thread. Both for making it possible for future improvements (such as logging GC work from JavaThreads) and because it seems like we normally keep storage in the Thread class even for things that are not used by all subclasses. For example TLABs and allocated_bytes() are only relevant for JavaThreads but have their storage in Thread. >>> Yes, Thread could be improved but I'm not sure I can stand adding GC data where >>> it's not needed. Unless GC logging from JavaThread is something that's going to >>> happen soon, I'd rather add it when it's needed rather than right now. I'd personally >>> rather duplicate the code in JavaThread if it is needed rather than have WatcherThreads >>> have a _gc_id. >> I tend to agree. I'll discuss this with Per tomorrow. If he agrees that we can move the _gc_id down to NamedThread, are you ok with me pushing the fix? >> >> Thanks, >> Bengt >> >>> Jon >>> >>>> Thanks, >>>> Bengt >>>> >>>>> Jon >>>>> >>>>>> This is the same as webrev.01 that both of you already looked at. I've made two minor adjustments: >>>>>> >>>>>> - I removed the currentNamedthread() function in gcId.cpp that Per pointed out in his review of webrev.01. Instead I use Thread::current() directly. >>>>>> >>>>>> - I made GCIdMark a StackObj. >>>>>> >>>>>> Otherwise the patch is the same as in webrev.01. >>>>>> >>>>>> Thanks, >>>>>> Bengt >>>>>> >>>>>> On 2015-09-10 14:37, Bengt Rutisson wrote: >>>>>>> Hi Jon, >>>>>>> >>>>>>> Thanks for looking at this! >>>>>>> >>>>>>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>>>>>> Bengt, >>>>>>>> >>>>>>>> When a CMS concurrent collection hands off to a STW foreground collection, >>>>>>>> (done in acquire_control_and_collect()), I expected a new GCId would be >>>>>>>> used. Did I miss it? That STW collection does not go through do_collection(). >>>>>>> The call to acquire_control_and_collect() originates from GenCollectedHeap::collect_generation() and there is a new GCIdMark in there that will create a new GCId for the STW collection. That's the "extra" GCIdMark that Per asked about in his review. >>>>>>> >>>>>>> *But* I wanted to verify that it worked properly since you asked about it and I realized that there is another bug. >>>>>>> >>>>>>> The GCId that is set up for the concurrent cycle, the one set up in ConcurrentMarkSweepThread::run(), will is still active over the complete STW foreground collection. That's fine in my model since the new GCIdMark in GenCollectedHeap::collect_generation() will cache that GCId. But the ConcurrentMarkSweep thread is not completely idle even though control has been left to the foreground collection in acquire_control_and_collect(). So, there is some logging done (by fore example CMSPhaseAccounting) that is done at the same time as the foreground collection. This logging will now use the foreground GCId instead of the CMS GCId. >>>>>>> >>>>>>> I haven't had time to dig in to the details of that just yet. But this is an unintended change of the logging, so I would like to fix it. Hopefully I'll have an updated webrev tomorrow. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>> >>>>>>>> Jon >>>>>>>> >>>>>>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>>>>>> Hi again, >>>>>>>>> >>>>>>>>> I found an issue with how G1 young collections kick off concurrent marks. There is a short window where we might have two active GC IDs at the same time. I've fixed this and updated the webrevs. In case anyone had already started looking at the webrevs you need to do a refresh now. The fix is in G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bengt >>>>>>>>> >>>>>>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>>>>>> Hi Per, >>>>>>>>>> >>>>>>>>>> Thanks for looking at this! >>>>>>>>>> >>>>>>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>>>>>> Hi Bengt, >>>>>>>>>>> >>>>>>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>>>>>> Hi everyone, >>>>>>>>>>>> >>>>>>>>>>>> This is mostly a GC related patch, but it adds a field to the Thread >>>>>>>>>>>> class, so I'm sending this out on the broader hotspot-dev list. >>>>>>>>>>>> >>>>>>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>>>>>> >>>>>>>>>>> Looks good. I think this is a nice simplification, especially for G1's concurrent parts. Just two comments: >>>>>>>>>>> >>>>>>>>>>> genCollectedHeap.cpp: >>>>>>>>>>> --------------------- >>>>>>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in different scopes. Do we really want that second mark? >>>>>>>>>> >>>>>>>>>> We potentially do two GCs in GenCollectedHeap::do_collection(). First a young GC and then potentially a full GC. These two should have different GC IDs. So, yes, we need two GCIdMarks in this method. The scoping could be better, but I think that is a refactoring that should be done separately from my current patch. I'll talk to Jesper about it since he has been cleaning up this code lately. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> gcId.cpp: >>>>>>>>>>> -------- >>>>>>>>>>> - I think you might have left currentNamedthread() in there. You probably just want to use Thread::current() instead. >>>>>>>>>> Yes, good catch. Thanks. >>>>>>>>>> >>>>>>>>>> However, after thinking some more about the implications of moving the GC ID out from the GCTracer I figured it would be possible to just store the previous GC ID in the GCIdMark and then restore it in the destructor of that class. That way we don't have to store anything in the Thread class but can still have both a concurrent GC ID and a STW GC ID active at the same time. This also removes the need to copy the GC ID to the worker tasks. >>>>>>>>>> >>>>>>>>>> Here's an updated webrev with a solution that does not add anything to the Thread class: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>>>>>> >>>>>>>>>> And here's a diff compared to the last version: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>>>>>> >>>>>>>>>> Unfotunately the webrev tool had some bad luck when creating the diff webrev so at least the g1CollectedHeap.cpp seem to contain the complete changes rather than just the diff compared to the 01 version. The rest of the diff looks correct as far as I can tell. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>>> cheers, >>>>>>>>>>> /Per >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Currently the GC ID, that is used for event tracing and logging, is >>>>>>>>>>>> stored in the GCTracer object. That has caused some minor problems since >>>>>>>>>>>> there are multiple GCTracers active at the same time. The correct GC IDs >>>>>>>>>>>> need to be passed around in many places. >>>>>>>>>>>> >>>>>>>>>>>> For some upcoming GC logging changes I would like to have a more >>>>>>>>>>>> consistent way of finding the currently active GC ID. I've played around >>>>>>>>>>>> with a couple of different solutions, but eventually I found that the >>>>>>>>>>>> simplest solution is to store the current GC ID in the thread. That way >>>>>>>>>>>> there is a single way to look it up in all places. It is also fairly >>>>>>>>>>>> straight forward to set it up. >>>>>>>>>>>> >>>>>>>>>>>> I've reworked the GCId class a bit to support this and I've introduced a >>>>>>>>>>>> GCIdMark class that is a scoped object that helps to set up and tear >>>>>>>>>>>> down a current GC ID. By moving the GC ID out from the GCTracer class I >>>>>>>>>>>> got rid of a few minor issues as well. One is that we no longer need to >>>>>>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was necessary >>>>>>>>>>>> before since we did logging *after* we had told the corresponding >>>>>>>>>>>> GCTracer that the concurrent cycle was aborted and it had thrown its GC >>>>>>>>>>>> ID away. Now the GC ID can stay in scope until we have completed all >>>>>>>>>>>> logging. >>>>>>>>>>>> >>>>>>>>>>>> Also the HeapDumpBeforeFullGC and PrintClassHistogramBeforeFullGC used >>>>>>>>>>>> to have to create a new GC ID since their logging was done before we had >>>>>>>>>>>> the correct GCTracer set up. Now the GC ID can be available early enough >>>>>>>>>>>> for this logging to be using the same (and correct) GC ID as the rest of >>>>>>>>>>>> the GC. Same for HeapDumpAfterFullGC and PrintClassHistogramAfterFullGC. >>>>>>>>>>>> >>>>>>>>>>>> I've added an uint to the Thread class to keep track of the currently >>>>>>>>>>>> active GC ID. In the current code there are actually only NamedThreads >>>>>>>>>>>> that need the GC ID. So, I'm not sure where the best place is for this >>>>>>>>>>>> field is. But it seems like the Thread class contains most of the data, >>>>>>>>>>>> even data that is only used by some subclasses, so I opted for putting >>>>>>>>>>>> the field in Thread as opposed to in NamedThread. This opens up for >>>>>>>>>>>> possible future extensions when Java threads may do part of the GC work. >>>>>>>>>>>> However, I'm open to moving the field down to NamedThread if that is >>>>>>>>>>>> seen as better. >>>>>>>>>>>> >>>>>>>>>>>> In the GCTracer class there were many asserts that were checking that >>>>>>>>>>>> the GC ID was properly set up. Mostly these assert verify that start/end >>>>>>>>>>>> is called correctly and that the other methods are called inside of the >>>>>>>>>>>> start/end scope. Now that the GC ID is moved out of the GCTracer class >>>>>>>>>>>> these asserts had to be dealt with. I figured there were three ways to >>>>>>>>>>>> handle it; just remove them, replace them with check that the GC ID from >>>>>>>>>>>> the current thread is correct, or implement a new status field to keep >>>>>>>>>>>> track of GC start/end state. Personally I'm not sure these asserts are >>>>>>>>>>>> valuable enough to spend time on, so I went for the first approach: >>>>>>>>>>>> removing them. I don't think making them use the thread GC ID will be >>>>>>>>>>>> appropriate. But if anyone feels strongly about these asserts I can >>>>>>>>>>>> implement a separate start/end state. >>>>>>>>>>>> >>>>>>>>>>>> There are a few "Tasks" in the GC code that are executed by worker >>>>>>>>>>>> threads. To make the worker threads use the correct GC ID I make sure >>>>>>>>>>>> that the Task constructors copy the GC ID from the initiating thread >>>>>>>>>>>> into a local variable. When the task is executed in its worker thread it >>>>>>>>>>>> sets up the GC ID based on the local variable. >>>>>>>>>>>> >>>>>>>>>>>> The proposed change does not alter any logging (except for the small bug >>>>>>>>>>>> fix for >>>>>>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. This >>>>>>>>>>>> means that no existing tests need to be updated. In particular the >>>>>>>>>>>> test/gc/logging/TestGCId.java test passes even after these changes. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> A big thanks to Per, who pre-reviewed these changes and came with some >>>>>>>>>>>> really good feedback. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Bengt From dmitry.dmitriev at oracle.com Wed Sep 30 10:14:03 2015 From: dmitry.dmitriev at oracle.com (Dmitry Dmitriev) Date: Wed, 30 Sep 2015 03:14:03 -0700 (PDT) Subject: RFR (S): JDK-8129855: -XX:+IgnoreUnrecognizedVMOptions hides out of range VM options. Message-ID: Hi Gerard, Looks good to me! Thanks, Dmitry ----- Original Message ----- From: gerard.ziemski at oracle.com To: hotspot-dev at openjdk.java.net, dmitry.dmitriev at oracle.com Sent: Tuesday, September 29, 2015 9:09:10 PM GMT +03:00 Iraq Subject: Re: RFR (S): JDK-8129855: -XX:+IgnoreUnrecognizedVMOptions hides out of range VM options. Here is a webrev1 updated with fixes based on Dmitry's feedback (ie. changes to the java test only): http://cr.openjdk.java.net/~gziemski/8129855_rev1 cheers On 09/28/2015 04:55 PM, gerard ziemski wrote: > hi all, > > We are fixing how IgnoreUnrecognizedVMOptions treats those flags whose values are out of range. Before the fix, the VM > would continue even if flag?s value was out of range, ex: > > java_old -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version > size_t MinTLABSize=0 is outside the allowed range [ 1 ... 4294967295 ] > java version "1.9.0-internal-fastdebug" > Java(TM) SE Runtime Environment (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00) > Java HotSpot(TM) Server VM (build 1.9.0-internal-fastdebug-20150624183527.jesper.main2rt-b00, mixed mode) > > now, we correctly exit the VM with an error, ex: > > java_new -XX:+IgnoreUnrecognizedVMOptions -XX:MinTLABSize=0 -version > size_t MinTLABSize=0 is outside the allowed range [ 1 ... 18446744073709551615 ] > Improperly specified VM option 'MinTLABSize=0' > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > > In addition IgnoreUnrecognizedVMOptions nows strictly follows the spec as outlined in JDK-8129855. The behavior changes > depending on whether the build is product or debug. > > We also introduce a new test that verifies all known use cases that we care about. > > References: > bugid: https://bugs.openjdk.java.net/browse/JDK-8129855 > webrev: http://cr.openjdk.java.net/~gziemski/8129855_rev0 > discussion: http://mail.openjdk.java.net/pipermail/hotspot-dev/2015-June/019213.html > > Passes "JPRT hotspot" and "RBT testlist,noncolo.testlist quick" > > > cheers > From erik.joelsson at oracle.com Wed Sep 30 12:43:44 2015 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 30 Sep 2015 14:43:44 +0200 Subject: [8u] Request for Approval and Review: JDK-8136980: build for 8u65 and 8u66 for solaris platforms is failing Message-ID: <560BD900.9000308@oracle.com> Hello, Please approve and review this fix for 8u. My last fix for this issue, JDK-8136691, was not enough. I made a mistake while verifying the fix and the problem is still there. There are more discrepancies between Solaris and the other platform makefiles. The excludeSrc.gmk file is not included anywhere when building on Solaris. The variable Src_Files_EXCLUDE is overwritten in vm.make instead of appended to, so even when including excludeSrc.gmk, it wasn't enough. The other platform specific buildtree.make files also set INCLUDE_TRACE in the generated makefile. Bug: https://bugs.openjdk.java.net/browse/JDK-8136980 Webrev: http://cr.openjdk.java.net/~erikj/8136980/webrev.hotspot.01/ /Erik From magnus.ihse.bursie at oracle.com Wed Sep 30 13:30:38 2015 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 30 Sep 2015 15:30:38 +0200 Subject: [8u] Request for Approval and Review: JDK-8136980: build for 8u65 and 8u66 for solaris platforms is failing In-Reply-To: <560BD900.9000308@oracle.com> References: <560BD900.9000308@oracle.com> Message-ID: <560BE3FE.7060103@oracle.com> On 2015-09-30 14:43, Erik Joelsson wrote: > Hello, > > Please approve and review this fix for 8u. > > My last fix for this issue, JDK-8136691, was not enough. I made a > mistake while verifying the fix and the problem is still there. > > There are more discrepancies between Solaris and the other platform > makefiles. The excludeSrc.gmk file is not included anywhere when > building on Solaris. The variable Src_Files_EXCLUDE is overwritten in > vm.make instead of appended to, so even when including excludeSrc.gmk, > it wasn't enough. The other platform specific buildtree.make files > also set INCLUDE_TRACE in the generated makefile. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8136980 > Webrev: http://cr.openjdk.java.net/~erikj/8136980/webrev.hotspot.01/ > > /Erik Looks good to me. /Magnus From jon.masamitsu at oracle.com Wed Sep 30 16:52:24 2015 From: jon.masamitsu at oracle.com (Jon Masamitsu) Date: Wed, 30 Sep 2015 09:52:24 -0700 Subject: RFR: JDK-8134953: Make the GC ID available in a central place In-Reply-To: <560B8D41.10107@oracle.com> References: <55EEC7F0.1060701@oracle.com> <55EEE152.3030404@oracle.com> <55F01A53.1070905@oracle.com> <55F044DD.4080301@oracle.com> <55F0BA00.5000704@oracle.com> <55F17982.3080606@oracle.com> <56093EC1.9090109@oracle.com> <5609768A.5030209@oracle.com> <560A4723.90604@oracle.com> <560AB551.6010507@oracle.com> <560AB4FC.1050805@oracle.com> <560AB765.9020605@oracle.com> <560B8D41.10107@oracle.com> Message-ID: <560C1348.6020003@oracle.com> On 9/30/2015 12:20 AM, Bengt Rutisson wrote: > > > On 2015-09-29 18:08, Jon Masamitsu wrote: >> >> >> On 9/29/15 8:57 AM, Bengt Rutisson wrote: >>> >>> >>> On 2015-09-29 17:59, Jon Masamitsu wrote: >>>> >>>> >>>> On 9/29/15 1:09 AM, Bengt Rutisson wrote: >>>>> >>>>> Hi Jon, >>>>> >>>>> On 2015-09-28 19:19, Jon Masamitsu wrote: >>>>>> >>>>>> >>>>>> On 09/28/2015 06:21 AM, Bengt Rutisson wrote: >>>>>>> >>>>>>> Hi Jon and Per, >>>>>>> >>>>>>> I've been trying to get the version proposed in webrev.02 to >>>>>>> work but I ran into a show stopper for this approach. The G1 >>>>>>> concurrent mark thread is at times executing (and logging) at >>>>>>> the same time as a young or mixed GC is executing. There is no >>>>>>> good way of handling this with a common place to store the GC >>>>>>> ID. Instead I've decided to go back to storing the current GC ID >>>>>>> in the thread. That way the G1 concurrent marking and the >>>>>>> young/mixed GCs will have their own GC IDs. >>>>>>> >>>>>>> Here's an updated webrev: >>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.03/ >>>>>> Bengt, >>>>>> >>>>>> The _gc_id is needed in a JavaThread because a JavaThread >>>>>> might do work for G1 (concurrent refinement for example) and >>>>>> needs a GC ID? >>>>> >>>>> The Java threads currently don't log anything when they do GC >>>>> work. But in the future it is likely that they will. >>>>> >>>>>> But a WatcherThread would never need one, >>>>>> right? >>>>> >>>>> Right. Initially I only added the _gc_id field to NamedThread. But >>>>> Per thought it would be better to have it in Thread. Both for >>>>> making it possible for future improvements (such as logging GC >>>>> work from JavaThreads) and because it seems like we normally keep >>>>> storage in the Thread class even for things that are not used by >>>>> all subclasses. For example TLABs and allocated_bytes() are only >>>>> relevant for JavaThreads but have their storage in Thread. >>>> >>>> Yes, Thread could be improved but I'm not sure I can stand adding >>>> GC data where >>>> it's not needed. Unless GC logging from JavaThread is something >>>> that's going to >>>> happen soon, I'd rather add it when it's needed rather than right >>>> now. I'd personally >>>> rather duplicate the code in JavaThread if it is needed rather than >>>> have WatcherThreads >>>> have a _gc_id. >>> >>> I tend to agree. I'll discuss this with Per tomorrow. If he agrees >>> that we can move the _gc_id down to NamedThread, are you ok with me >>> pushing the fix? >> >> Yes, I'm good with the fix then. > > Thanks Jon! > > I talked to Per and moved the _gc_id field down to NamedThread. > Pushing it now. > >> >> Would it be appropriate for me to file a CR to change NamedThread to >> something >> like GCThread? Or ??? > > > I don't think GCThread is the right name since VMThread is a > NamedThread and it can schedule tasks that are not GC related. I agree > that NamedThread is not a great name. But I actually think that the > naming in the Thread hierarchy is just part of the problem. The real > problem is that the division of responsibility is not really correctly > partitioned among the classes. > > If we should file a RFE I think it should be to clean up the Thread > hierarchy, including finding good names for the classes. That's fine. I created https://bugs.openjdk.java.net/browse/JDK-8138655 Jon > > Bengt > > >> >> Jon >> >>> >>> Thanks, >>> Bengt >>> >>>> >>>> Jon >>>> >>>>> >>>>> Thanks, >>>>> Bengt >>>>> >>>>>> >>>>>> Jon >>>>>> >>>>>>> >>>>>>> This is the same as webrev.01 that both of you already looked >>>>>>> at. I've made two minor adjustments: >>>>>>> >>>>>>> - I removed the currentNamedthread() function in gcId.cpp that >>>>>>> Per pointed out in his review of webrev.01. Instead I use >>>>>>> Thread::current() directly. >>>>>>> >>>>>>> - I made GCIdMark a StackObj. >>>>>>> >>>>>>> Otherwise the patch is the same as in webrev.01. >>>>>>> >>>>>>> Thanks, >>>>>>> Bengt >>>>>>> >>>>>>> On 2015-09-10 14:37, Bengt Rutisson wrote: >>>>>>>> >>>>>>>> Hi Jon, >>>>>>>> >>>>>>>> Thanks for looking at this! >>>>>>>> >>>>>>>> On 2015-09-10 01:00, Jon Masamitsu wrote: >>>>>>>>> Bengt, >>>>>>>>> >>>>>>>>> When a CMS concurrent collection hands off to a STW foreground >>>>>>>>> collection, >>>>>>>>> (done in acquire_control_and_collect()), I expected a new GCId >>>>>>>>> would be >>>>>>>>> used. Did I miss it? That STW collection does not go >>>>>>>>> through do_collection(). >>>>>>>> >>>>>>>> The call to acquire_control_and_collect() originates from >>>>>>>> GenCollectedHeap::collect_generation() and there is a new >>>>>>>> GCIdMark in there that will create a new GCId for the STW >>>>>>>> collection. That's the "extra" GCIdMark that Per asked about in >>>>>>>> his review. >>>>>>>> >>>>>>>> *But* I wanted to verify that it worked properly since you >>>>>>>> asked about it and I realized that there is another bug. >>>>>>>> >>>>>>>> The GCId that is set up for the concurrent cycle, the one set >>>>>>>> up in ConcurrentMarkSweepThread::run(), will is still active >>>>>>>> over the complete STW foreground collection. That's fine in my >>>>>>>> model since the new GCIdMark in >>>>>>>> GenCollectedHeap::collect_generation() will cache that GCId. >>>>>>>> But the ConcurrentMarkSweep thread is not completely idle even >>>>>>>> though control has been left to the foreground collection in >>>>>>>> acquire_control_and_collect(). So, there is some logging done >>>>>>>> (by fore example CMSPhaseAccounting) that is done at the same >>>>>>>> time as the foreground collection. This logging will now use >>>>>>>> the foreground GCId instead of the CMS GCId. >>>>>>>> >>>>>>>> I haven't had time to dig in to the details of that just yet. >>>>>>>> But this is an unintended change of the logging, so I would >>>>>>>> like to fix it. Hopefully I'll have an updated webrev tomorrow. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bengt >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Jon >>>>>>>>> >>>>>>>>> On 9/9/2015 7:40 AM, Bengt Rutisson wrote: >>>>>>>>>> >>>>>>>>>> Hi again, >>>>>>>>>> >>>>>>>>>> I found an issue with how G1 young collections kick off >>>>>>>>>> concurrent marks. There is a short window where we might have >>>>>>>>>> two active GC IDs at the same time. I've fixed this and >>>>>>>>>> updated the webrevs. In case anyone had already started >>>>>>>>>> looking at the webrevs you need to do a refresh now. The fix >>>>>>>>>> is in G1CollectedHeap::do_collection_pause_at_safepoint(). >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bengt >>>>>>>>>> >>>>>>>>>> On 09/09/15 13:38, Bengt Rutisson wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Per, >>>>>>>>>>> >>>>>>>>>>> Thanks for looking at this! >>>>>>>>>>> >>>>>>>>>>> On 2015-09-08 15:23, Per Liden wrote: >>>>>>>>>>>> Hi Bengt, >>>>>>>>>>>> >>>>>>>>>>>> On 2015-09-08 13:35, Bengt Rutisson wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>> >>>>>>>>>>>>> This is mostly a GC related patch, but it adds a field to >>>>>>>>>>>>> the Thread >>>>>>>>>>>>> class, so I'm sending this out on the broader hotspot-dev >>>>>>>>>>>>> list. >>>>>>>>>>>>> >>>>>>>>>>>>> Could I have a couple of reviews for this patch? >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01/ >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8134953 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Looks good. I think this is a nice simplification, >>>>>>>>>>>> especially for G1's concurrent parts. Just two comments: >>>>>>>>>>>> >>>>>>>>>>>> genCollectedHeap.cpp: >>>>>>>>>>>> --------------------- >>>>>>>>>>>> - GenCollectedHeap::do_collection() has two GCIdMarks, in >>>>>>>>>>>> different scopes. Do we really want that second mark? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> We potentially do two GCs in >>>>>>>>>>> GenCollectedHeap::do_collection(). First a young GC and then >>>>>>>>>>> potentially a full GC. These two should have different GC >>>>>>>>>>> IDs. So, yes, we need two GCIdMarks in this method. The >>>>>>>>>>> scoping could be better, but I think that is a refactoring >>>>>>>>>>> that should be done separately from my current patch. I'll >>>>>>>>>>> talk to Jesper about it since he has been cleaning up this >>>>>>>>>>> code lately. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> gcId.cpp: >>>>>>>>>>>> -------- >>>>>>>>>>>> - I think you might have left currentNamedthread() in >>>>>>>>>>>> there. You probably just want to use Thread::current() >>>>>>>>>>>> instead. >>>>>>>>>>> >>>>>>>>>>> Yes, good catch. Thanks. >>>>>>>>>>> >>>>>>>>>>> However, after thinking some more about the implications of >>>>>>>>>>> moving the GC ID out from the GCTracer I figured it would be >>>>>>>>>>> possible to just store the previous GC ID in the GCIdMark >>>>>>>>>>> and then restore it in the destructor of that class. That >>>>>>>>>>> way we don't have to store anything in the Thread class but >>>>>>>>>>> can still have both a concurrent GC ID and a STW GC ID >>>>>>>>>>> active at the same time. This also removes the need to copy >>>>>>>>>>> the GC ID to the worker tasks. >>>>>>>>>>> >>>>>>>>>>> Here's an updated webrev with a solution that does not add >>>>>>>>>>> anything to the Thread class: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> And here's a diff compared to the last version: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~brutisso/8134953/webrev.01-02.diff/ >>>>>>>>>>> >>>>>>>>>>> Unfotunately the webrev tool had some bad luck when creating >>>>>>>>>>> the diff webrev so at least the g1CollectedHeap.cpp seem to >>>>>>>>>>> contain the complete changes rather than just the diff >>>>>>>>>>> compared to the 01 version. The rest of the diff looks >>>>>>>>>>> correct as far as I can tell. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Bengt >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> cheers, >>>>>>>>>>>> /Per >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> Currently the GC ID, that is used for event tracing and >>>>>>>>>>>>> logging, is >>>>>>>>>>>>> stored in the GCTracer object. That has caused some minor >>>>>>>>>>>>> problems since >>>>>>>>>>>>> there are multiple GCTracers active at the same time. The >>>>>>>>>>>>> correct GC IDs >>>>>>>>>>>>> need to be passed around in many places. >>>>>>>>>>>>> >>>>>>>>>>>>> For some upcoming GC logging changes I would like to have >>>>>>>>>>>>> a more >>>>>>>>>>>>> consistent way of finding the currently active GC ID. I've >>>>>>>>>>>>> played around >>>>>>>>>>>>> with a couple of different solutions, but eventually I >>>>>>>>>>>>> found that the >>>>>>>>>>>>> simplest solution is to store the current GC ID in the >>>>>>>>>>>>> thread. That way >>>>>>>>>>>>> there is a single way to look it up in all places. It is >>>>>>>>>>>>> also fairly >>>>>>>>>>>>> straight forward to set it up. >>>>>>>>>>>>> >>>>>>>>>>>>> I've reworked the GCId class a bit to support this and >>>>>>>>>>>>> I've introduced a >>>>>>>>>>>>> GCIdMark class that is a scoped object that helps to set >>>>>>>>>>>>> up and tear >>>>>>>>>>>>> down a current GC ID. By moving the GC ID out from the >>>>>>>>>>>>> GCTracer class I >>>>>>>>>>>>> got rid of a few minor issues as well. One is that we no >>>>>>>>>>>>> longer need to >>>>>>>>>>>>> keep track of the G1 "aborted concurrent GC ID". That was >>>>>>>>>>>>> necessary >>>>>>>>>>>>> before since we did logging *after* we had told the >>>>>>>>>>>>> corresponding >>>>>>>>>>>>> GCTracer that the concurrent cycle was aborted and it had >>>>>>>>>>>>> thrown its GC >>>>>>>>>>>>> ID away. Now the GC ID can stay in scope until we have >>>>>>>>>>>>> completed all >>>>>>>>>>>>> logging. >>>>>>>>>>>>> >>>>>>>>>>>>> Also the HeapDumpBeforeFullGC and >>>>>>>>>>>>> PrintClassHistogramBeforeFullGC used >>>>>>>>>>>>> to have to create a new GC ID since their logging was done >>>>>>>>>>>>> before we had >>>>>>>>>>>>> the correct GCTracer set up. Now the GC ID can be >>>>>>>>>>>>> available early enough >>>>>>>>>>>>> for this logging to be using the same (and correct) GC ID >>>>>>>>>>>>> as the rest of >>>>>>>>>>>>> the GC. Same for HeapDumpAfterFullGC and >>>>>>>>>>>>> PrintClassHistogramAfterFullGC. >>>>>>>>>>>>> >>>>>>>>>>>>> I've added an uint to the Thread class to keep track of >>>>>>>>>>>>> the currently >>>>>>>>>>>>> active GC ID. In the current code there are actually only >>>>>>>>>>>>> NamedThreads >>>>>>>>>>>>> that need the GC ID. So, I'm not sure where the best place >>>>>>>>>>>>> is for this >>>>>>>>>>>>> field is. But it seems like the Thread class contains most >>>>>>>>>>>>> of the data, >>>>>>>>>>>>> even data that is only used by some subclasses, so I opted >>>>>>>>>>>>> for putting >>>>>>>>>>>>> the field in Thread as opposed to in NamedThread. This >>>>>>>>>>>>> opens up for >>>>>>>>>>>>> possible future extensions when Java threads may do part >>>>>>>>>>>>> of the GC work. >>>>>>>>>>>>> However, I'm open to moving the field down to NamedThread >>>>>>>>>>>>> if that is >>>>>>>>>>>>> seen as better. >>>>>>>>>>>>> >>>>>>>>>>>>> In the GCTracer class there were many asserts that were >>>>>>>>>>>>> checking that >>>>>>>>>>>>> the GC ID was properly set up. Mostly these assert verify >>>>>>>>>>>>> that start/end >>>>>>>>>>>>> is called correctly and that the other methods are called >>>>>>>>>>>>> inside of the >>>>>>>>>>>>> start/end scope. Now that the GC ID is moved out of the >>>>>>>>>>>>> GCTracer class >>>>>>>>>>>>> these asserts had to be dealt with. I figured there were >>>>>>>>>>>>> three ways to >>>>>>>>>>>>> handle it; just remove them, replace them with check that >>>>>>>>>>>>> the GC ID from >>>>>>>>>>>>> the current thread is correct, or implement a new status >>>>>>>>>>>>> field to keep >>>>>>>>>>>>> track of GC start/end state. Personally I'm not sure these >>>>>>>>>>>>> asserts are >>>>>>>>>>>>> valuable enough to spend time on, so I went for the first >>>>>>>>>>>>> approach: >>>>>>>>>>>>> removing them. I don't think making them use the thread GC >>>>>>>>>>>>> ID will be >>>>>>>>>>>>> appropriate. But if anyone feels strongly about these >>>>>>>>>>>>> asserts I can >>>>>>>>>>>>> implement a separate start/end state. >>>>>>>>>>>>> >>>>>>>>>>>>> There are a few "Tasks" in the GC code that are executed >>>>>>>>>>>>> by worker >>>>>>>>>>>>> threads. To make the worker threads use the correct GC ID >>>>>>>>>>>>> I make sure >>>>>>>>>>>>> that the Task constructors copy the GC ID from the >>>>>>>>>>>>> initiating thread >>>>>>>>>>>>> into a local variable. When the task is executed in its >>>>>>>>>>>>> worker thread it >>>>>>>>>>>>> sets up the GC ID based on the local variable. >>>>>>>>>>>>> >>>>>>>>>>>>> The proposed change does not alter any logging (except for >>>>>>>>>>>>> the small bug >>>>>>>>>>>>> fix for >>>>>>>>>>>>> HeapDumpBefore(After)FullGC/PrintClassHistogramBefore(After)FullGC. >>>>>>>>>>>>> This >>>>>>>>>>>>> means that no existing tests need to be updated. In >>>>>>>>>>>>> particular the >>>>>>>>>>>>> test/gc/logging/TestGCId.java test passes even after these >>>>>>>>>>>>> changes. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> A big thanks to Per, who pre-reviewed these changes and >>>>>>>>>>>>> came with some >>>>>>>>>>>>> really good feedback. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Bengt >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From dean.long at oracle.com Wed Sep 30 18:55:06 2015 From: dean.long at oracle.com (Dean Long) Date: Wed, 30 Sep 2015 11:55:06 -0700 Subject: RFR (XS) 8078295 - hotspot test_env.sh can set VM_CPU incorrectly In-Reply-To: <560AE404.4050503@oracle.com> References: <560AE404.4050503@oracle.com> Message-ID: <560C300A.9050407@oracle.com> Looks fine to me, as long as the new output line 115 echo "INT_VERSION=`cat vm_version.out 2>&1`" isn't going to confuse any scripts. dl On 9/29/2015 12:18 PM, Ioi Lam wrote: > Please review a very small fix: > > http://cr.openjdk.java.net/~iklam/8078295_test_env_sh.01/ > > Bug: [TESTBUG] hotspot test_env.sh can set VM_CPU incorrectly > > https://bugs.openjdk.java.net/browse/JDK-8078295 > > Summary of fix: > > test_env.sh uses the following to get information such as VM_CPU > > ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion > > vm_version.out 2>&1 > > VM_CPU="unknown" > grep "arm" vm_version.out > ${NULL} > if [ $? = 0 ] > then > VM_CPU="arm" > fi > > However, the internal version may contain arbitrary text (from the > user name and pwd) > such as "arm", which would cause VM_CPU to have incorrect value. > > The fix is to filter out the arbitrary text using a sed script > (please see the bug report > for details and sample output): > > ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion \ > | sed -e 's/[(][^)]*[)]//g' -e 's/ by "[^"]*"//g' \ > > vm_version.out 2>&1 > > Tests: > > JPRT > RBT, with the following tests > hotspot/test/:hotspot_all > vm.parallel_class_loading.testlist > vm.regression.testlist > vm.runtime.testlist > vm.tmtools.testlist) > > Thanks > - Ioi From dmitry.samersoff at oracle.com Wed Sep 30 19:32:13 2015 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Wed, 30 Sep 2015 22:32:13 +0300 Subject: RFR (XS) 8078295 - hotspot test_env.sh can set VM_CPU incorrectly In-Reply-To: <560AE404.4050503@oracle.com> References: <560AE404.4050503@oracle.com> Message-ID: <560C38BD.7090701@oracle.com> Ioi, Looks good for me. (Assumed you are tested it under cygwin as well). echo "INT_VERSION=`cat vm_version.out 2>&1`" Actually you need not to redirect stderr to stdout here because cat would print something to stderr only if vm_version.out doesn't exist or unreadable. -Dmitry On 2015-09-29 22:18, Ioi Lam wrote: > Please review a very small fix: > > http://cr.openjdk.java.net/~iklam/8078295_test_env_sh.01/ > > Bug: [TESTBUG] hotspot test_env.sh can set VM_CPU incorrectly > > https://bugs.openjdk.java.net/browse/JDK-8078295 > > Summary of fix: > > test_env.sh uses the following to get information such as VM_CPU > > ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion > > vm_version.out 2>&1 > > VM_CPU="unknown" > grep "arm" vm_version.out > ${NULL} > if [ $? = 0 ] > then > VM_CPU="arm" > fi > > However, the internal version may contain arbitrary text (from the > user name and pwd) > such as "arm", which would cause VM_CPU to have incorrect value. > > The fix is to filter out the arbitrary text using a sed script > (please see the bug report > for details and sample output): > > ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion \ > | sed -e 's/[(][^)]*[)]//g' -e 's/ by "[^"]*"//g' \ > > vm_version.out 2>&1 > > Tests: > > JPRT > RBT, with the following tests > hotspot/test/:hotspot_all > vm.parallel_class_loading.testlist > vm.regression.testlist > vm.runtime.testlist > vm.tmtools.testlist) > > Thanks > - Ioi -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From ioi.lam at oracle.com Wed Sep 30 22:19:17 2015 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 Sep 2015 15:19:17 -0700 Subject: RFR (XS) 8078295 - hotspot test_env.sh can set VM_CPU incorrectly In-Reply-To: <560C38BD.7090701@oracle.com> References: <560AE404.4050503@oracle.com> <560C38BD.7090701@oracle.com> Message-ID: <560C5FE5.6090906@oracle.com> Hi Dean & Dmitry, Thanks for the reviews. I've tested using RBT on windows/mac/solaris/linux, as well as JPRT on all platforms (including embedded), so cygwin has been tested. The INT_VERSION= output doesn't seem to confuse any scripts. I think it's better to have the "2>&1" in there. If vm_version.out is unavailable (due to file system corruption, etc) which would cause these subsequent lines to have incorrect values: echo "VM_TYPE=${VM_TYPE}" echo "VM_BITS=${VM_BITS}" echo "VM_OS=${VM_OS}" echo "VM_CPU=${VM_CPU}" If this happens, the INT_VERSION= output will tells us what went wrong. Thanks - Ioi On 9/30/15 12:32 PM, Dmitry Samersoff wrote: > Ioi, > > Looks good for me. (Assumed you are tested it under cygwin as well). > > echo "INT_VERSION=`cat vm_version.out 2>&1`" > > Actually you need not to redirect stderr to stdout here because cat > would print something to stderr only if vm_version.out doesn't exist > or unreadable. > > -Dmitry > > On 2015-09-29 22:18, Ioi Lam wrote: >> Please review a very small fix: >> >> http://cr.openjdk.java.net/~iklam/8078295_test_env_sh.01/ >> >> Bug: [TESTBUG] hotspot test_env.sh can set VM_CPU incorrectly >> >> https://bugs.openjdk.java.net/browse/JDK-8078295 >> >> Summary of fix: >> >> test_env.sh uses the following to get information such as VM_CPU >> >> ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion > >> vm_version.out 2>&1 >> >> VM_CPU="unknown" >> grep "arm" vm_version.out > ${NULL} >> if [ $? = 0 ] >> then >> VM_CPU="arm" >> fi >> >> However, the internal version may contain arbitrary text (from the >> user name and pwd) >> such as "arm", which would cause VM_CPU to have incorrect value. >> >> The fix is to filter out the arbitrary text using a sed script >> (please see the bug report >> for details and sample output): >> >> ${TESTJAVA}${FS}bin${FS}java ${TESTOPTS} -Xinternalversion \ >> | sed -e 's/[(][^)]*[)]//g' -e 's/ by "[^"]*"//g' \ >> > vm_version.out 2>&1 >> >> Tests: >> >> JPRT >> RBT, with the following tests >> hotspot/test/:hotspot_all >> vm.parallel_class_loading.testlist >> vm.regression.testlist >> vm.runtime.testlist >> vm.tmtools.testlist) >> >> Thanks >> - Ioi > From rob.mckenna at oracle.com Wed Sep 30 13:40:25 2015 From: rob.mckenna at oracle.com (Rob McKenna) Date: Wed, 30 Sep 2015 14:40:25 +0100 Subject: [8u] Request for Approval and Review: JDK-8136980: build for 8u65 and 8u66 for solaris platforms is failing In-Reply-To: <560BE3FE.7060103@oracle.com> References: <560BD900.9000308@oracle.com> <560BE3FE.7060103@oracle.com> Message-ID: <560BE649.7080904@oracle.com> Approved. -Rob On 30/09/15 14:30, Magnus Ihse Bursie wrote: > On 2015-09-30 14:43, Erik Joelsson wrote: >> Hello, >> >> Please approve and review this fix for 8u. >> >> My last fix for this issue, JDK-8136691, was not enough. I made a >> mistake while verifying the fix and the problem is still there. >> >> There are more discrepancies between Solaris and the other platform >> makefiles. The excludeSrc.gmk file is not included anywhere when >> building on Solaris. The variable Src_Files_EXCLUDE is overwritten in >> vm.make instead of appended to, so even when including excludeSrc.gmk, >> it wasn't enough. The other platform specific buildtree.make files >> also set INCLUDE_TRACE in the generated makefile. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8136980 >> Webrev: http://cr.openjdk.java.net/~erikj/8136980/webrev.hotspot.01/ >> >> /Erik > > Looks good to me. > > /Magnus